Advertisement
If you have a new account but are having problems posting or verifying your account, please email us on hello@boards.ie for help. Thanks :)
Hello all! Please ensure that you are posting a new thread or question in the appropriate forum. The Feedback forum is overwhelmed with questions that are having to be moved elsewhere. If you need help to verify your account contact hello@boards.ie
Hi all,
Vanilla are planning an update to the site on April 24th (next Wednesday). It is a major PHP8 update which is expected to boost performance across the site. The site will be down from 7pm and it is expected to take about an hour to complete. We appreciate your patience during the update.
Thanks all.

query Internet databases

Options
  • 28-03-2015 9:43pm
    #1
    bpb101
    Registered Users Posts: 2,827 ✭✭✭


    Hey yall ,
    just started learing php and started databases a few months ago in college
    Im just wondering does anybody know who to do a select statment on websites databases
    sites like http://www.checkargos.com/ obviously do a search on argos publicly visitable database except reformats it as to search all stores at once.


    I would like to be able to search http://store.steampowered.com/

    Is there a way to be able to do this

    Cheers lads
    Tagged:


Comments

  • Options
    #2
    Javan
    Closed Accounts Posts: 265 ✭✭Javan


    bpb101 wrote: »
    Hey yall ,
    just started learing php and started databases a few months ago in college
    Im just wondering does anybody know who to do a select statment on websites databases
    sites like http://www.checkargos.com/ obviously do a search on argos publicly visitable database except reformats it as to search all stores at once.


    I would like to be able to search http://store.steampowered.com/

    Is there a way to be able to do this

    Cheers lads

    First: do not try and do this without at getting permission from the site owner. What you are proposing could be construed as unauthorised access which could (worst case) lead to a criminal conviction.

    Probably the simplest way to do this is the most obvious: write a script that sends a large number of queries at the target site where each query is one that the target site supports.
    In the case of check Argos that means taking the stock code provided from the user and sending requests to the argos.ie site in the same format as the requests you can generate from argos.ie. Then parse the results.

    A more interesting way would be to try to access the database directly. Google 'SQL injection attacks'.
    ... but be aware that this could be considered an illegal / criminal act.


  • Options
    #3
    28064212
    Registered Users Posts: 10,458 ✭✭✭✭28064212


    bpb101 wrote: »
    sites like http://www.checkargos.com/ obviously do a search on argos publicly visitable database except reformats it as to search all stores at once.
    It doesn't. It's extremely unlikely that any website would allow direct external access to their database, it would be practically impossible to secure.

    checkargos.com does essentially the exact same thing as argos.ie does. When you go to a product page on argos.ie and click "Go" beside the "Check stock in your area", it submits data (including the productID and the storeID) to a specific address on argos.ie. The page responds with the stock availability for that store.

    checkargos.com just automates that so you don't have to repeat the request 40 times

    Boardsie Enhancement Suite - a browser extension to make using Boards on desktop a better experience (includes full-width display, keyboard shortcuts, and dark mode). Now available through the extension stores

    Firefox: https://addons.mozilla.org/addon/boardsie-enhancement-suite/

    Chrome/Edge/Opera: https://chromewebstore.google.com/detail/boardsie-enhancement-suit/bbgnmnfagihoohjkofdnofcfmkpdmmce



  • Options
    #4
    bpb101
    Registered Users Posts: 2,827 ✭✭✭bpb101


    Javan wrote: »
    First: do not try and do this without at getting permission from the site owner. What you are proposing could be construed as unauthorised access which could (worst case) lead to a criminal conviction.

    Probably the simplest way to do this is the most obvious: write a script that sends a large number of queries at the target site where each query is one that the target site supports.
    In the case of check Argos that means taking the stock code provided from the user and sending requests to the argos.ie site in the same format as the requests you can generate from argos.ie. Then parse the results.

    A more interesting way would be to try to access the database directly. Google 'SQL injection attacks'.
    ... but be aware that this could be considered an illegal / criminal act.

    Thanks for the reply and word of caution.

    Presumable when you search a website you send a query to that website through search, The html then formats that into a way that they think is the best way to view it , clearly if you wanted to take record that info and check to see price reductions and such like http://argos.scanner.today/ie/index.html


    what i would like to do is
    select title, price , sale price from steamdatabase.
    and php it to look out for sales and what not

    Mainly as a learning exercise

    Steamdb have a load of info on steam as well as other websites who look record info on steam sales and there dosent seem to be any issues though


  • Options
    #5
    Javan
    Closed Accounts Posts: 265 ✭✭Javan


    bpb101 wrote: »
    Thanks for the reply and word of caution.

    Presumable when you search a website you send a query to that website through search, The html then formats that into a way that they think is the best way to view it , clearly if you wanted to take record that info and check to see price reductions and such like http://argos.scanner.today/ie/index.html


    what i would like to do is
    select title, price , sale price from steamdatabase.
    and php it to look out for sales and what not

    Mainly as a learning exercise

    Steamdb have a load of info on steam as well as other websites who look record info on steam sales and there dosent seem to be any issues though

    OK, I've had a quick look at the Argos site. If you look at any of the static product pages (click any of the product links on the homepage) you can see that there is a storechecker function. This is in a div called 'deliveryInformation'. Clicking the button on that causes some script to fire a request at the server and get a response. The script then formats that response in another div called 'lightBox'. Since that request/response is handled by client-site script it is probably very well formatted AJAX-like stuff. It would be very easy for a simple PHP script to send that request once per store location and parse the XML response.

    The Argos scanner and SteamDB are actually very different propositions. SteamDBs explanation on their homepage is very clear: Valve send updates to the Steam clients with information on offers. SteamDB is a client that receives those updates and makes them easy to search. It does not search a Valve database, it has a DB of its own that is maintained using info published by Valve. I don't know the Argos scanner but it could well operate the same way; receiving updates from one or more feeds ands providing a single view.


  • Options
    #6
    28064212
    Registered Users Posts: 10,458 ✭✭✭✭28064212


    bpb101 wrote: »
    what i would like to do is
    select title, price , sale price from steamdatabase.
    and php it to look out for sales and what not

    Mainly as a learning exercise

    Steamdb have a load of info on steam as well as other websites who look record info on steam sales and there dosent seem to be any issues though
    SteamDB don't query the database directly, they use the SteamKit SDK, which (I assume) uses Steam's API

    A better option for you would be to use the Storefront API directly. For example, if you go to http://store.steampowered.com/api/appdetails/?appids=440, you get the details for Team Fortress 2 in JSON format. This includes a lot of information, including the pricing information.

    The first step in your application could be to get your PHP to call http://store.steampowered.com/api/appdetails/ with given appids, and parse the returned data into a readable format and display it to the user

    Boardsie Enhancement Suite - a browser extension to make using Boards on desktop a better experience (includes full-width display, keyboard shortcuts, and dark mode). Now available through the extension stores

    Firefox: https://addons.mozilla.org/addon/boardsie-enhancement-suite/

    Chrome/Edge/Opera: https://chromewebstore.google.com/detail/boardsie-enhancement-suit/bbgnmnfagihoohjkofdnofcfmkpdmmce



  • Advertisement
  • Options
    #7
    bpb101
    Registered Users Posts: 2,827 ✭✭✭bpb101


    28064212 wrote: »
    SteamDB don't query the database directly, they use the SteamKit SDK, which (I assume) uses Steam's API

    A better option for you would be to use the Storefront API directly. For example, if you go to http://store.steampowered.com/api/appdetails/?appids=440, you get the details for Team Fortress 2 in JSON format. This includes a lot of information, including the pricing information.

    The first step in your application could be to get your PHP to call http://store.steampowered.com/api/appdetails/ with given appids, and parse the returned data into a readable format and display it to the user
    Thats great. thanks very much ,
    I could also get that info and trow it into my own database.and work from there.


Advertisement