We know how important it is for you to prevent search engines from displaying your content on Google or other search engines, and we’ve come up with two solutions for you:
- No-index/no-follow header for the Redirect to file option
Since the Download Monitor v4.5.9 update, we’ve added a no-index/no-follow header for the Redirect to file option, meaning that the URL in question has a no-index/no-follow directive in the header:
-> The no-index metatag tells a search engine that even though it can crawl the page, it cannot add the page to its search index,
-> The no-follow tag disallows search engines from crawling the links on that page.
So, having no-index/no-follow directives in the Redirect to file page will prevent access to the search engine’s index and, therefore, won’t show up on search engine results pages. In a nutshell, the Redirect to file page will have all its links ignored by Google and other search engines. - Robots.txt generation/protection setting
You can also find the Robots.txt generation function in Settings > General > Miscellaneous that helps you add the robots.txt. A robots.txt is a text file that tells the search engines where they are not allowed to go on a site.
-> if there is no robots.txt, we will create it for you and add the code;
-> if there is a robots.txt and it’s a real file, then click on Regenerate protection for uploads folder, and we’ll add the code;
Example:User-agent: * Disallow: /dlm_uploads/
-> if it is a virtual file, we’ll warn you to modify it; otherwise, you risk losing what you already have in robots.txt.