If you have a new account but are having problems posting or verifying your account, please email us on [email protected] for help. Thanks :)
Hello All, This is just a friendly reminder to read the Forum Charter where you wish to post before posting in it. :)
Hi all, The AutoSave Draft feature is now disabled across the site. The decision to disable the feature was made via a poll last year. The delay in putting it in place was due to a bug/update issue. This should serve as a reminder to manually save your drafts if you wish to keep them. Thanks, The Boards Team.
Hello all! This is just a quick reminder to ensure that you are posting a new thread or question in the appropriate forum. The Feedback forum is overwhelmed with questions that are having to be moved elsewhere.

best way to parse data from webpage

  • 25-05-2009 9:27am
    Registered Users Posts: 8,070 ✭✭✭ Placebo

    Need to retrieve data from a stock site.
    What would we the best way to retrieve data from it [and store it in a db?]

    php - curl ? parse by looking at page structure, i.e if it uses tables etc?



  • Moderators, Category Moderators, Motoring & Transport Moderators Posts: 21,184 CMod ✭✭✭✭ Eoin

    As in a stock price / financial website? Sounds obvious, but see if they have an RSS or atom feed you can use. Or they have a ticker which might be nicely marked up and easy to parse.

  • Registered Users Posts: 8,070 ✭✭✭ Placebo

    Yeah, stock price / financial website.
    No API/RSS/

    ticker =?

  • Moderators, Category Moderators, Motoring & Transport Moderators Posts: 21,184 CMod ✭✭✭✭ Eoin

    You know the scrolly things that show the stock prices? That might use a feed you can use.

  • Closed Accounts Posts: 1 RandiR

    You can automatically get stock prices using a script similar to one below. This script is written in biterscripting. It downloads a stock's price from
    # [B]SCRIPT: Price.txt[/B]
    # Input argument.
    var string symbol
    # Get the first table from for the stock symbol.
    var string URL, table
    set $URL = ""+$symbol
    script ss_webpagetocsv.txt page($URL) number(1) > $table
    # Get the first row.
    var string row ; lex "1" $table > $row
    # Get the first column.
    var string column ; wex "1" $row > $column
    # $column now has stock price in string format. Convert it to real.
    var real price ; set $price = makereal(str($column))
    # Print price. Or insert into DB or excel.
    echo $price

    Store the script in C:\Scripts\Price.txt. Call it from biterscripting as follows.
    script Price.txt symbol("<your stock symbol here>")

    If you want to load the prices into DB instead, insert an sql statement in the above script. You can do this for multiple stock prices by writing a script that calls the above Price script once per stock symbol. Also, you can schedule this activity daily/hourly/etc. using task scheduler or cron.

    As far biterscripting ( ), it is just a general scripting language. It has nothing to do with stock prices. You can use any scripting language. Just code the above functionality in the scripting language of your choice.


    (Make sure any site you use to download prices from, that it is ok with them to download prices using such batch approach.)

  • Registered Users Posts: 1 Josh223

    For Stock prices in Excel, I use iXBRLAnalyst Addin for Excel. It adds a new built-in function "SharePrice(ticker, date)".

  • Advertisement