Getting Google to fine forum pages.

beller b

I am using SMF bulletinn Boards but the posts never show in google searches.Is there something that needs to be placed in the meta tags for google to index these pages?

aidan_walsh

You could Sitemap it.

Gordon

Firstly have you submitted it to google?

mneylon

Gordon wrote:

Firstly have you submitted it to google?

You shouldn't have to

Seth Brundle

Michele - do you mean that there should be sufficient links to your site from elsewhere?

mneylon

Yes.

beller b

I've submitted the site...Google finds the site no problem..But the contents of the forums don't show up in any search.. For eg if I search google with BELLER B I will get a load of Boards.ie pages...
I'll give the sitemaps a go. Thanks for your help.

Serbian

From the Google Webmaster Guidelines:

Allow search bots to crawl your sites without session IDs or arguments that track their path through the site. These techniques are useful for tracking individual user behavior, but the access pattern of bots is entirely different. Using these techniques may result in incomplete indexing of your site, as bots may not be able to eliminate URLs that look different but actually point to the same page.

I would imagine that the session ids in your URLs is causing the pages not to be indexed.

seo-ireland

Yes it's your session IDs that are causing the indexing problems. Google see that you have only one file "index.php" so you will need to look in using mod_rewrite to rewrite your dynamic urls into more static, distinct urls.

Example:

index.php?PHPSESSID=0f7bd431f7cd452c7c2d959e39de0e18&topic=25.0

into something like

topic25.html

Gordon

blacknight wrote:

You shouldn't have to

If you have no sites linking to you then Google wouldn't pick you up right?

AARRRGH

Example of url rewriting (from http://www.staff.ie, the company I work for) -

RewriteEngine on
RewriteRule jobs\.html jobseekers_search_results.php?stype=m

This should be in your .htaccess file in the forum root.

As you can see, we are turning the rewrite engine on, and then we are turning http://www.staff.ie/jobseekers_search_results.php?stype=m into http://www.staff.ie/jobs.html. If you go to both URLs are will see it is the same content.

http://www.staff.ie/jobs.html is SEO in two instances -

1. Google can read it properly (Google does not like ?stype=m)
2. We now have the keyword "jobs" in the URL. I could easily make this even more SEO friendly by calling it something like "irish-jobs-ireland" or similar.

See what I mean?

Go to http://forums.seochat.com for more info.

AARRRGH

Gordon wrote:

If you have no sites linking to you then Google wouldn't pick you up right?

Yes. They need to find you somehow.

Seth Brundle

Gordon wrote:

If you have no sites linking to you then Google wouldn't pick you up right?

Also the more sites you have linking to you, the better the Google rank [along with loads of other factors such as content, host location....]!

mneylon

Depending on the software you are using for your forum you should be able to hack it to be more search engine friendly - mod_rewrite alone is not enough

mneylon

kbannon wrote:

Also the more sites you have linking to you, the better the Google rank [along with loads of other factors such as content, host location....]!

Quality is more important than quantity ...

TheWaterboy

dublindude wrote:

Example of url rewriting (from http://www.staff.ie, the company I work for) -

RewriteEngine on
RewriteRule jobs\.html jobseekers_search_results.php?stype=m

This should be in your .htaccess file in the forum root.

As you can see, we are turning the rewrite engine on, and then we are turning http://www.staff.ie/jobseekers_search_results.php?stype=m into http://www.staff.ie/jobs.html. If you go to both URLs are will see it is the same content.

http://www.staff.ie/jobs.html is SEO in two instances -

1. Google can read it properly (Google does not like ?stype=m)
2. We now have the keyword "jobs" in the URL. I could easily make this even more SEO friendly by calling it something like "irish-jobs-ireland" or similar.

See what I mean?

Go to http://forums.seochat.com for more info.

Am i correct in saying that you now have two pages with the exact same continent - google doesnt like duplicate continent and can punish your rankings for this

mneylon

Google is less likely to index the URI with queries etc., than it is the "static" pages, so they're not going to suffer from duplicate content

seo-ireland

Google likes:

- Unique well written content (not overly optimized e.g. repeated keywords)
- Lots and lots (and lots) of quality one-way inbound links. Reciprocal links look to be devalued somewhat.
- Varying text in the inbound links e.g. hundreds of links with the same text 'blue widgets' is going to look fishy to Google, variations like 'view information about blue widgets' and 'For blue widgets click here' are what to strive for.
- Natural use of the title tag and header tags, not stuffed with keywords, make it look natural.
- Validated HTML
- Static URLs
- An easily indexable site. Avoid 'clever' Javascript and Flash navigation system, use standard text links with CSS for style instead. If you have a large site implement Google Sitemaps to allow Google to find all of your pages.
- Use robots.txt, meta tags "noindex, nofollow" and the rel="nofollow" href attribute to guide the Google spider through your site.
- Get links from major directories like Yahoo, DMOZ, Zeal, Joeant, Gimpsy etc..

I have written some SEO articles here that should help you further.

AARRRGH

TheWaterboy wrote:

Am i correct in saying that you now have two pages with the exact same continent - google doesnt like duplicate continent and can punish your rankings for this

No, you're incorrect. As far as Google/your browser are aware, only jobs.html exist.

beller b

Loads to think about there... Thanks a million lads..

beller b

Just tried a few other search engines..
Yahoo MSN & alavista all find the page contents...Could this be google just being slower than the others?
Anyway I've made a few changes & resubmitted to google. We'll see how that goes. Thanks for all the suggestions..

Gordon

Yeah, I got that too with a few sites beller, mind you it can turn up on Google within a day, it just depends on when they do their indexing afaik.

beller b

Thanks Gordon.I'll give it a while & see what happens. If the other engines are seeing it I can't be doing too much wrong!

dahamsta

beller b, you should take the PHP session (PHPSESSID=0f7bd431f7cd452c7c2d959e39de0e18&) out of the link to your forums in your sig. It could lead to someone stealing someone else's session by mistake.

adam

beller b

Thanks for all the help lads. I've done the sitemap & submitted it to Google. It is showing the last upload etc & no errors.But still no other pages being indexed. What confuses me is that Googles robots are crawling the boards constantly.So if they are finding the pages why are they not being indexed.Also MSN is indexing everything fine. What methods do MSN use that would be different to googles?
Any help would be appriciated. Thanks again.

beller b

Just an update on this..I got freefind.com to index my site & all my forun/boards pages show up as discarded (off site) There are links from the boards back to site pages .Why should it see these as off site?

wheres me jumpa

i have always found that my sites are ranked higher in msn, yahoo, alta vista and many others than google. at times the gap between the rankings are huge.

is this an indication as to which engine is better?!

mneylon

wheres me jumpa wrote:

is this an indication as to which engine is better?!

No. It's totally meaningless
Ranked better? For what?

beller b

Just to let you guys know.Google has started indexing my forum pages from today..Is it that Google is just slower than the others or Just more fussy..
A added more hidden links bact to my main page from the forums, that seemed to help..

seo-ireland

Eeek hidden links! You mean links that are invisible? I would get rid of them, Google delves penalties out for them.

mneylon

seo-ireland wrote:

Eeek hidden links! You mean links that are invisible? I would get rid of them, Google delves penalties out for them.

When it finds them ....

A lot of blackhat SEO does actually seem to work.

One of our competitors was using keyword stuffing, multiple gateway sites and a whole load of other techniques and was doing really well for well over a year. Google seems to have started penalising *some* of their sites now, but their primary site seems to have been unaffected
*sigh*

seo-ireland

Believe me Blacknight, I wouldn't touch any blackhat techniques with my least favourite barge pole. It may not be today, it may not be tomorrow but in the future these techniques will be discovered and penalised.

mneylon

Oh I know, but it still makes me sick that they can get away with it..

seo-ireland

Search engine death to spammers!

mneylon

Don't get me started!

beller b

Still havn't got this right..Google indexed a few pages the other day, but the've disappeared again..
I ran the sitemap generator on www.blocklayer.com & I got this on about 200 pages

Link http://****.com/smf/index.php?PHPSESSID=662811ca8df07f5bf2cd3c5c4965a73f&action=who missing www. on page http://www.****.com/smf/index.php (different Domain)
Link http://****.com/smf/index.php?PHPSESSID=662811ca8df07f5bf2cd3c5c4965a73f&action=who missing www. on page http://www.****.com/smf/index.php (different Domain)
Link http://****.com/smf/index.php?PHPSESSID=662811ca8df07f5bf2cd3c5c4965a73f&action=profile;u=41 missing www. on page http://www.****.com/smf/index.php (different Domain)
How can I stop it seeing it as a different domain?

Serbian

beller b wrote:

How can I stop it seeing it as a different domain?

Looks like you need to generate all your links either with the www. or without, not a mixture of. It's a bit stupid that it doesn't recognise www.****.com and ****.com being the same domain, but that's what the problem seems to be.

seo-ireland

Here's a piece of code you can place into the top of your .htaccess file to convert all non-www accesses to the www equivalents in a search engine friendly manner (301 redirects)

RewriteEngine on
RewriteCond %{HTTP_HOST} ^yoursite\.com [NC]
RewriteRule ^(.*)$ http://www.yoursite.com/$1 [R=301,L]

Works like a charm when implemented correctly.

Serbian

seo-ireland wrote:

Here's a piece of code you can place into the top of your .htaccess file to convert all non-www accesses to the www equivalents in a search engine friendly manner (301 redirects)

RewriteEngine on
RewriteCond %{HTTP_HOST} ^yoursite\.com [NC]
RewriteRule ^(.*)$ http://www.yoursite.com/$1 [R=301,L]

Works like a charm when implemented correctly.

That's certainly a useful piece of code, but I don't think it will help with Google sitemaps as it will just read the URL and decide that it's not in the same domain as his site, rather than try and visit it.

seo-ireland

Yes on a re-read I agree. The only thing to do is to make sure all of the links on the site begin with www. This is good practice anyway and the code I posted is essential to avoid link popularity for your site being split between non-www and www versions (the two versions are treated as two different domains)

beller b

Thanks Lads I,ve dont that now. But once you enter the forum pages if loses the www after you leave the smf/index.phb page.. Does it make any difference after that or is there something within the SMF code that would need to be changed?
Thanks again