Hide website from search engines?

httpete · 12-09-2011 9:55pm #1

I have a website that I want to put up but it is not ready for the general public yet so I dont want it showing up in search engines. Is there a way of accomplishing this?

Fanny Cradock · 12-09-2011 10:31pm

Off hand I can think of three methods.

1) place the site in a ridiculously complicated file path - yoursite.com/thisisthesillynameofmyfolder/

2) Change your robots.txt file

3) Host it locally by using wamp, xampp or whatever suits.

dlofnep · 12-09-2011 10:32pm

Robots.txt if not hosting is on a local server.

dlofnep · 12-09-2011 10:32pm

Blast you Fanny.

Fanny Cradock · 12-09-2011 10:36pm

Pixelcraft · 12-09-2011 11:34pm

From google:

To prevent all robots from indexing a page on your site, place the following meta tag into the <head> section of your page:

<meta name="robots" content="noindex">
To allow other robots to index the page on your site, preventing only Google's robots from indexing the page:

<meta name="googlebot" content="noindex">

pixelburp · 12-09-2011 11:42pm

+1 for robots.txt; quick & simple.

Adding to your <head> is a good solution too, but I personally don't like putting in markup or code that's test-environment only, just in case you forget to take it out before going live (hey, we've all done it)

Eoin · 14-09-2011 3:00pm

Out of interest - is it just a matter of adding your site back into Google through the webmaster tools once you want it indexed again? i.e. it's not going to harm future SEO stuff?

dlofnep wrote: »

Blast you Fanny.

snigger

Cathaoirleach · 15-09-2011 8:34pm

For sites I'm developing but don't want to be live, I usually restrict by password or IP using .htaccess
http://home.golden.net/htaccess.html

Or I just include this at the header of each page and put in my IP (and the client's if needed)

[PHP]$userIP = getenv('REMOTE_ADDR');
if ($userIP != '192.168.1.1') {
die("access denied");
}[/PHP]

That way you don't get any bots indexing your unfinished site!

Trojan · 16-09-2011 4:04pm

If it's WordPress, you can install a Maintenance Mode plugin that allows logged in users view the site but noone else.

jmcc · 16-09-2011 4:51pm

Cathaoirleach wrote: »

For sites I'm developing but don't want to be live, I usually restrict by password or IP using .htaccess
http://home.golden.net/htaccess.html

Or I just include this at the header of each page and put in my IP (and the client's if needed)

[PHP]$userIP = getenv('REMOTE_ADDR');
if ($userIP != '192.168.1.1') {
die("access denied");
}[/PHP]

That way you don't get any bots indexing your unfinished site!

You might find it hard to get into a search engine index if you just serve that as content. It should get flagged by the SE's quality control as a dead/denied website and dropped from the crawl. Getting back into the SE's index might take a few months or a few hours depending on the SE but it is a risky thing to do.

The simplest way is just work on the site on your own PC and only put it on the web when you are ready for it to be spidered/crawled.

Regards...jmcc

Hide website from search engines?

Comments