Welcome to the Question2Answer Q&A. There's also a demo if you just want to try it out.
0 votes
1.3k views
in Q2A Core by
Which pages should be blocked to overcome the isssue of duplicate contents.
Q2A version: 1.5.2

1 Answer

0 votes
by

Download Link Sleuth and run it on your site

http://home.snafu.de/tilman/xenulink.html

 

You will get all the links in your site, create robots.txt based on that

On my yet to be published site, I finalized on the following

 

Disallow: /login
Disallow: /ask
Disallow: /forgot
Disallow: /register
Disallow: /questions?sort
Disallow: /admin
Disallow: /*?show=

Also I got the script listed here developed http://www.question2answer.org/qa/15139/fighting-google-panda

Of course, as you can see community did not like the idea showcased in my post but I stand by it as mentioned in the post

http://www.question2answer.org/qa/15139/fighting-google-panda

...