IT -Automatic Website usage law.

theTinker

I'm working on a project at the moment that requires me to get a list of different organizations. The information will be used for creating metrics in my company(i work for it, not own it).

The most feasible way of getting the needed information is by using website search features. I'm talking association sites which list their member companies, business phone book sites, etc. Since the search functions are there for public usage, I would assume using them manually and copying information with cut / paste is OK.

Due to the size of the project though and the amount of information, its infeasible to manually do this. Is it legal to write small one shot applications to automatically invoke the search functions using HTTP GET and POST protocols and parse the results into a structure I want?

I'm not sure where clever usage ends and hacking begin. The bandwidth used on thier servers sites would be far less than manually doing it since I'm not downloading the Images, graphics, etc, only text information(HTML).
All the functions I'm invoking are for public usage but this is probably not what they had in mind when setting them up..

Secondly, assuming i run through parameters like "Dublin i" and increase i from 1-24 each time. I know that by just passing parameters like "" (empty values) will get me back much more information. It would all be obtainable if i was to use their selectable values on the drop down menus but doing it this way would save me a lot of time since i can get it all in one go. I get no extra information but technically here I'm doing something that average users couldn't.

Would anyone be kind enough to let me know which parts of this would be legal or illegal?

Thanks.

jhegarty

This not legal advice but...


Every site sets its policy on bots/crawlers in a file called robots.txt

As long as it allows bots you shouldn't have a problem on searching the site.... but of course the site still have copyright of the info on there...

seamus

You may find that you're better off contacting the relevant sites directly to discover if they'll let you do it, or if they'll supply the information to you directly.

They may of course then want money off you.

www.irishstatutebook.ie is down at the moment, so I couldn't search.

It may be worthwhile checking out the relevant laws. My gut says that the act only applies to unlawfully accessing a computer system (or accessing without lawful excuse), or otherwise taking measures in order to actively disable or damage the website (DDOS attacks, etc).

Writing a script which directly accesses a website, I can't see being seen unlawfully accessing the site, or otherwise an attempt to gain any information or access which isn't already published.
*Maybe* if your script caused the site to be disabled (because they didn't count on that volume of data going out), it could be criminal negligence, but I'd be skeptical.

rsynnott

Some countries have laws specifically related to mining other peoples' databases (these are useful for real estate websites and such). I'm not sure whether Ireland is one of them, though.