Removing Blacknight test URL from SERPs

yahoo_moe

Hi there,

The site I manage is hosted on Blacknight and I'm wondering about how best to stop the xxxxxxxxx.blacknight.com alias URL that we used to test the site before going live from showing up in search results.

The pages don't have massive pagerank but doing a search for site:xxxxxxxxx.blacknight.com shows that a number of URLs have been indexed and I'd obviously rather that nobody ever saw them there.

I presume it's a robots.txt solution but just wondering how that works when the /webspace/httpdocs directory is the root for both URLs - will search engines understand what's going on?

Thanks...

mneylon

It depends on which CMS you are using.

In any case .. make sure that you have configured it to use the domain and not the alias.

You can then set a .htaccess to push / redirect all traffic from the temporary URL to the actual domain. (see http://www.modrewrite.co.uk/ for some tips)


Of course, if you aren't planning on launching for a while you might want to simply block who can access the site based on IP, which would stop Google et al from spidering

yahoo_moe

Hi, thanks for the quick response.

The site's already live so I'm not going to block completely as that would remove us from SERPs completely.

In terms of doing it with .htaccess, I've already got a few 301s set up from pages on the old site (same domain, different content) to current pages. Can I just add the 301s to the same file? i.e. when search engines crawl, do they consider that the .htaccess for both xxxxxxxxx.blacknight.com and example.com simultaneously?

In that case, given that there's no pagerank I want to retain, am I not better off blocking those pages using robots.txt and then removing the URLs via Gootle Webmaster Tools? Or will that stop example.com rankinga also?

mneylon

If you look at:
http://www.modrewrite.co.uk/mod-rewrite/canonical-urls-with-mod-rewrite.html
It will basically do a redirect on any domain to the main domain

So, for example, if you had domain1.tld and domain2.tld pointing at domain.tld it would push all traffic to domain.tld

A 301 redirect tells Google et al that the content has moved permanently, so two things will happen

1 - anybody clicking through will get to the "real" URL
2 - Google will drop the current URL it has indexed in favour of the "real" one

You could do a removal request via Webmaster tools, but that might not be feasible if there are thousands of URLs

yahoo_moe

Grand so. Just one last question in that case:

modrewrite.co.uk wrote:

RewriteEngine on
RewriteCond %{HTTP_HOST} !^example\.com$ [NC]
RewriteRule ^(.*)$ http://example.com/$1 [L,R=301]

From the link you gave, it seems I don't have to specify the xxxxxxxxx.blacknight.com URL at all? And is that forward slash in the domain name right?

yahoo_moe

Actually, got it sorted there - I just added a rewrite rule for the xxxxxxx.blacknight.com URL and it's working fine just the way the rule for the non-www URL works.

Cheers for your help.