Best way to seach google from desktop

techguy

Hi,

I'm currently working on a web crawler and need to search google to get start links.

At the moment i'm screen scraping the result from a simple query but its getting very complicated to just get a clean set of results.

I've been reading up on Google search API but it seems they have discontinued what I am looking for.

Maybe i'm overlooking some new service they have introduced. It would be nice to be able to have something that would return the results in xml.

Páid

DMOZ provide their database of urls free of charge - http://rdf.dmoz.org

Might be easier and faster than querying Google for links.

techguy

How detailed is this DB? I'm going to be searching for pages that are pretty new so would this do the job?

I've been working with Yahoo Boss over the last few days and it seems pretty good.

Thanks.

Páid

It contains a lot more than just urls e.g. hierarchical categories, etc. No idea how up to date it is.

Yahoo BOSS looks good. I noticed that there are unlimited queries presently but that they intend to introduce a fee structure.