Install Git for Windows, and make sure to put Git/bin folder in your Windows path variable.

Here is a good link to make your experience of Git and PowerShell better:

http://learnaholic.me/2012/10/12/make-powershell-and-git-suck-less-on-windows/