Want to Stop Search Engines from Crawling website then you should change some setting which can stop search engine bots to enter in your site. What settings that we are going to tell you here.
Many users don’t want to crawl their sites and for that they have to stop search engine bots to come on their sites. Especially those people who have private sites or blogs. Some time blogger don’t want to index any particular post which has any illegal content. So, during such time you have to make some changes from your dashboard and you can stop search engines to crawl your site.
How to stop Search Engines from Crawling website
This is a one simple method which you can use to stop bots to come on your site. So just update your robots.txt file with given below code
No doubt, Robots.txt file will stop search engine bots to crawl your site but there are some other ways from where search engine bots can come to your site and at that time this will not help. What are those ways? Here are those
- May be it is possible that your domain name was used by someone else before you bought and they have posted links to other sites. So from those sites search engine bots can come to your site
- It is also possible that some one has randomly made the list of domain names on his/ her site and when bots will crawl their site at that time there are chances that bots can come to your site also
So, now what to do? During such time you can use these alternatives.
No index, No follow
If you are using WordPress then you can set this option and to do that follow given below steps
- Go to dashboard of your site
- Under setting tab open Reading option
- Now check the Search Engine Visibility box and you done
You can protect your entire site with password and no one can access it. To set up this password protection follow given below steps
- Log in to your site’s cPanel
- Under Security tab find Password Protect Directories option
- Now in pop up window select the folder where your site is installed (Generally it is public_html) and click on OK button
- Check the Password protect this directory box and save it
- Now you done just go back and set username and password
Your site is now password protected and you can stop accessing it to anyone. So these are the ways to stop search engines from Crawling website. If this is helpful for you then kindly share with others and also let us know through comments.