Pulling publicly available info into a web page

Have you any preferences on language / framework?

Digital_Guy wrote: »

I am sure this is relatively straightforward, but does anyone have any pointers on how to approach building even a beta version of this type of site?

As Graham mentioned considering the copyright implications is important, but on the other hand sometimes you just have to go ahead without any assurances that what you're doing is permissible. It's important to make sure you're not harming the originators business/traffic and even better if you actually provide value to them (e.g. refer business/traffic to them). Try and act in good faith.

You can access another sites data either of three ways:

1. Via an API that the site provides. If a site does provide an API for their data this is an indication that they think it's in their interest disseminating their data.
2. Via a download the site provides. Some sites will provide certain data in a text or CSV file which you can download (in an automated way) and then pull the data from the file.
3. Scraping the website. Where you access the site via a program/service in the same way a person accesses the site. The program then pulls certain bits of information off the site. Scraping is a gray area - some sites are dead against being scraped and other sites don't mind it if the scraping activity is minimal.

You should probably be able to find out if the sites you're targeting provide an API or download. Chances are they won't but if they do the Gods are smiling on you. If you want to look into scraping have a look at something like Portia (to plug an Irish company) to get an idea of how you might go about scraping data.

In terms of building a beta - I imagine the most sensible thing to do is engage with a developer. If this is a passion project you might find a developer who just wants to build it with you in their spare time (i.e. they're also passionate about it as well). Chances are you'll only find someone like that among your friends, colleagues, or wider social circle since developers typically would work on their own ideas if they wanted a side project. Oh, and for this approach you'd need to make sure there's enough upfront work for you to take on so it doesn't look like you're trying to lump all the work onto someone else which can often be the case when a non-developer works with a developer (i.e. someone thinks all they need to be is the "idea guy").

If you're planning on paying someone to build it. Chances are you're looking at a non-trivial sum of money and you'd be best shopping around with a handful of freelancers and studios. Do your due diligence and ideally if you know someone who can refer a freelancer/studio that they used and are happy with that's even better.

If you're looking at this as a potential business it's worth looking into programmes such as New Frontiers if you're new to the world of starting a business. And engage with your local enterprise office to see what they can do for you. If you are looking at paying someone to build the product for you the local enterprise offices offer some fund matching grants you might be able to avail of.

biko wrote: »

I was just going to say this too. If the site(s) provide API or download it should be fine to link to them as they have provided means to do so.

would also make life a bit easier too, potentially