Question2Answer Q&A - Recent questions tagged crawling https://www.question2answer.org/qa/tag/crawling Powered by Question2Answer Do i have to block search engines from crawling the other 3 domain versions (Robots.txt https://www.question2answer.org/qa/73733/block-search-engines-crawling-other-domain-versions-robots I have all 4 domain versions on Google WMT. 3 of them use a permanent redirect to the preferred.<br /> <br /> www and none www &gt; redirects to https none www version Q2A Core https://www.question2answer.org/qa/73733/block-search-engines-crawling-other-domain-versions-robots Thu, 28 Mar 2019 08:28:00 +0000