query Internet databases

bpb101

Hey yall ,
just started learing php and started databases a few months ago in college
Im just wondering does anybody know who to do a select statment on websites databases
sites like http://www.checkargos.com/ obviously do a search on argos publicly visitable database except reformats it as to search all stores at once.


I would like to be able to search http://store.steampowered.com/

Is there a way to be able to do this

Cheers lads

Javan

bpb101 wrote: »

Hey yall ,
just started learing php and started databases a few months ago in college
Im just wondering does anybody know who to do a select statment on websites databases
sites like http://www.checkargos.com/ obviously do a search on argos publicly visitable database except reformats it as to search all stores at once.


I would like to be able to search http://store.steampowered.com/

Is there a way to be able to do this

Cheers lads

First: do not try and do this without at getting permission from the site owner. What you are proposing could be construed as unauthorised access which could (worst case) lead to a criminal conviction.

Probably the simplest way to do this is the most obvious: write a script that sends a large number of queries at the target site where each query is one that the target site supports.
In the case of check Argos that means taking the stock code provided from the user and sending requests to the argos.ie site in the same format as the requests you can generate from argos.ie. Then parse the results.

A more interesting way would be to try to access the database directly. Google 'SQL injection attacks'.
... but be aware that this could be considered an illegal / criminal act.

28064212

bpb101 wrote: »

sites like http://www.checkargos.com/ obviously do a search on argos publicly visitable database except reformats it as to search all stores at once.

It doesn't. It's extremely unlikely that any website would allow direct external access to their database, it would be practically impossible to secure.

checkargos.com does essentially the exact same thing as argos.ie does. When you go to a product page on argos.ie and click "Go" beside the "Check stock in your area", it submits data (including the productID and the storeID) to a specific address on argos.ie. The page responds with the stock availability for that store.

checkargos.com just automates that so you don't have to repeat the request 40 times

bpb101

Javan wrote: »

First: do not try and do this without at getting permission from the site owner. What you are proposing could be construed as unauthorised access which could (worst case) lead to a criminal conviction.

Probably the simplest way to do this is the most obvious: write a script that sends a large number of queries at the target site where each query is one that the target site supports.
In the case of check Argos that means taking the stock code provided from the user and sending requests to the argos.ie site in the same format as the requests you can generate from argos.ie. Then parse the results.

A more interesting way would be to try to access the database directly. Google 'SQL injection attacks'.
... but be aware that this could be considered an illegal / criminal act.

Thanks for the reply and word of caution.

Presumable when you search a website you send a query to that website through search, The html then formats that into a way that they think is the best way to view it , clearly if you wanted to take record that info and check to see price reductions and such like http://argos.scanner.today/ie/index.html


what i would like to do is
select title, price , sale price from steamdatabase.
and php it to look out for sales and what not

Mainly as a learning exercise

Steamdb have a load of info on steam as well as other websites who look record info on steam sales and there dosent seem to be any issues though

Javan

bpb101 wrote: »

Thanks for the reply and word of caution.

Presumable when you search a website you send a query to that website through search, The html then formats that into a way that they think is the best way to view it , clearly if you wanted to take record that info and check to see price reductions and such like http://argos.scanner.today/ie/index.html


what i would like to do is
select title, price , sale price from steamdatabase.
and php it to look out for sales and what not

Mainly as a learning exercise

Steamdb have a load of info on steam as well as other websites who look record info on steam sales and there dosent seem to be any issues though

OK, I've had a quick look at the Argos site. If you look at any of the static product pages (click any of the product links on the homepage) you can see that there is a storechecker function. This is in a div called 'deliveryInformation'. Clicking the button on that causes some script to fire a request at the server and get a response. The script then formats that response in another div called 'lightBox'. Since that request/response is handled by client-site script it is probably very well formatted AJAX-like stuff. It would be very easy for a simple PHP script to send that request once per store location and parse the XML response.

The Argos scanner and SteamDB are actually very different propositions. SteamDBs explanation on their homepage is very clear: Valve send updates to the Steam clients with information on offers. SteamDB is a client that receives those updates and makes them easy to search. It does not search a Valve database, it has a DB of its own that is maintained using info published by Valve. I don't know the Argos scanner but it could well operate the same way; receiving updates from one or more feeds ands providing a single view.

28064212

bpb101 wrote: »

what i would like to do is
select title, price , sale price from steamdatabase.
and php it to look out for sales and what not

Mainly as a learning exercise

Steamdb have a load of info on steam as well as other websites who look record info on steam sales and there dosent seem to be any issues though

SteamDB don't query the database directly, they use the SteamKit SDK, which (I assume) uses Steam's API

A better option for you would be to use the Storefront API directly. For example, if you go to http://store.steampowered.com/api/appdetails/?appids=440, you get the details for Team Fortress 2 in JSON format. This includes a lot of information, including the pricing information.

The first step in your application could be to get your PHP to call http://store.steampowered.com/api/appdetails/ with given appids, and parse the returned data into a readable format and display it to the user

bpb101

28064212 wrote: »

SteamDB don't query the database directly, they use the SteamKit SDK, which (I assume) uses Steam's API

A better option for you would be to use the Storefront API directly. For example, if you go to http://store.steampowered.com/api/appdetails/?appids=440, you get the details for Team Fortress 2 in JSON format. This includes a lot of information, including the pricing information.

The first step in your application could be to get your PHP to call http://store.steampowered.com/api/appdetails/ with given appids, and parse the returned data into a readable format and display it to the user

Thats great. thanks very much ,
I could also get that info and trow it into my own database.and work from there.