Advertisement
If you have a new account but are having problems posting or verifying your account, please email us on hello@boards.ie for help. Thanks :)
Hello all! Please ensure that you are posting a new thread or question in the appropriate forum. The Feedback forum is overwhelmed with questions that are having to be moved elsewhere. If you need help to verify your account contact hello@boards.ie
Hi all! We have been experiencing an issue on site where threads have been missing the latest postings. The platform host Vanilla are working on this issue. A workaround that has been used by some is to navigate back from 1 to 10+ pages to re-sync the thread and this will then show the latest posts. Thanks, Mike.
Hi there,
There is an issue with role permissions that is being worked on at the moment.
If you are having trouble with access or permissions on regional forums please post here to get access: https://www.boards.ie/discussion/2058365403/you-do-not-have-permission-for-that#latest

best way to parse data from webpage

  • 25-05-2009 10:27am
    #1
    Registered Users, Registered Users 2 Posts: 8,070 ✭✭✭


    Need to retrieve data from a stock site.
    What would we the best way to retrieve data from it [and store it in a db?]

    php - curl ? parse by looking at page structure, i.e if it uses tables etc?

    Thanks


Comments

  • Registered Users, Registered Users 2 Posts: 21,257 ✭✭✭✭Eoin


    As in a stock price / financial website? Sounds obvious, but see if they have an RSS or atom feed you can use. Or they have a ticker which might be nicely marked up and easy to parse.


  • Registered Users, Registered Users 2 Posts: 8,070 ✭✭✭Placebo


    Yeah, stock price / financial website.
    No API/RSS/

    ticker =?


  • Registered Users, Registered Users 2 Posts: 21,257 ✭✭✭✭Eoin


    You know the scrolly things that show the stock prices? That might use a feed you can use.


  • Closed Accounts Posts: 1 RandiR


    You can automatically get stock prices using a script similar to one below. This script is written in biterscripting. It downloads a stock's price from finance.yahoo.com.
    # [B]SCRIPT: Price.txt[/B]
    # Input argument.
    var string symbol
     
    # Get the first table from finance.yahoo.com for the stock symbol.
    var string URL, table
    set $URL = "http://finance.yahoo.com/q?s="+$symbol
    script ss_webpagetocsv.txt page($URL) number(1) > $table
     
    # Get the first row.
    var string row ; lex "1" $table > $row
    # Get the first column.
    var string column ; wex "1" $row > $column
    # $column now has stock price in string format. Convert it to real.
    var real price ; set $price = makereal(str($column))
     
    # Print price. Or insert into DB or excel.
    echo $price
    

    Store the script in C:\Scripts\Price.txt. Call it from biterscripting as follows.
    script Price.txt symbol("<your stock symbol here>")
    

    If you want to load the prices into DB instead, insert an sql statement in the above script. You can do this for multiple stock prices by writing a script that calls the above Price script once per stock symbol. Also, you can schedule this activity daily/hourly/etc. using task scheduler or cron.

    As far biterscripting ( http://www.biterscripting.com/install.html ), it is just a general scripting language. It has nothing to do with stock prices. You can use any scripting language. Just code the above functionality in the scripting language of your choice.

    Randi

    (Make sure any site you use to download prices from, that it is ok with them to download prices using such batch approach.)


  • Registered Users Posts: 1 Josh223


    For Stock prices in Excel, I use iXBRLAnalyst Addin for Excel. It adds a new built-in function "SharePrice(ticker, date)".


  • Advertisement
Advertisement