Why you should not block pages with robots.txt

People often block pages for many different reasons, such as to hide duplicate content. The problem with doing this is that any links pointing to these pages are wasting link juice.

If you look at the image, you will see that A links to B and B links to C, A is passing link juice to B, inturn the link juice from B is passed to C. Now lets imagine that B is blocked using robots.txt, A passes link juice to B and because it can not be indexed, the search engine will not find the link to C, therfore B will never pass the link juice to C. The link from A to B is passing its link juice to nowhere, its wasted.

What would be a better solutions would be to use a meta tag, no-index,follow, this tells the search engine not to index the page but still follow the links.

An even better solution is to fix you original problem so you do not need to hide the page from the index.

See A simple explanation of PageRank for more on how link juice is passed..