Mortgage interest rate points

With low interest rates is it a good idea to pay down points at closing? I had several mortgage Companies tell me different things. One is you will get a bigger bang for your money; recoup your money that you paid in points in a few years. Other Mortgage Companies said people will move before they recover their money. Here in the Washington area the average move is around five years or so. Then it would be to the advantage of the borrower to pay down points. A lot of borrowers these days are apprehensive they will only put down the required minimum with no points. What do you think?