Need help with google search a site

maurice1

I am trying to search www.rtb.ie

For example: http://www.rtb.ie/dispute-resolution/dispute-resolution/adjudication-orders/2010

down at the bottom shows:
DR690/2009 Applicant Tenants- Ayodeji Algebeleye and Usman Akinade / Respondent Landlords- Peter Creamer and Adam Creamer clearly as text.

I only picked this example as the text was unusual and a couple of years old so web bots would have found it.

SO if I google: site:www.rtb.ie "Ayodeji Algebeleye"
I tried Yahoo, Bing, Duckduckgo also

I get 0 results
I would have expected 2:
1. being the link above
2. the content of the linked pdf

Does anybody know why or where am I going wrong?



Is there a search engine better than google now

PS this question has nothing to do with the RTB adjudication, purely I just want to learn about search engines.

Thanks

maurice1

Have I put this question in the wrong section of Boards?

bonzodog2

A search of that site for "tenant" only returned 420 results for me. Its possible that some parts of their site are restricted as regards being searchable, maybe with a robots.txt file or some other means.

http://www.robotstxt.org/robotstxt.html

golfcaptain

For the example you gave, if you look at the robots.txt for that site (http://www.rtb.ie/robots.txt), you will see "Disallow: /dispute-resolution/disputes/adjudication-orders". Thus, any search engine that respects robots.txt files won't index (i.e. make searchable) the contents of http://www.rtb.ie/dispute-resolution/dispute-resolution/adjudication-orders/2010

maurice1

Thanks Guys for the info.
I had never heard of robot txt before.
I feel that google used to give better search results a couple of years ago.
Is there a better search engine or one that ignores robot.txt?

mach1982

robot.txx are used by web crawlers, they tell what-what part of the website to be ignore Google is still best search engine