Advertisement
If you have a new account but are having problems posting or verifying your account, please email us on hello@boards.ie for help. Thanks :)
Hello all! Please ensure that you are posting a new thread or question in the appropriate forum. The Feedback forum is overwhelmed with questions that are having to be moved elsewhere. If you need help to verify your account contact hello@boards.ie

Ebay/ Ad classification Scrapers

Options
  • 19-03-2015 4:28pm
    #1
    skulltown
    Registered Users Posts: 125 ✭✭


    Hi

    I hope this is the correct tread to use this but I am looking to use some sort of web scraper for marketing. I am looking to get all listing on ebay or add classification sites then export then to excel. Any help would be helpful. I am looking for a free solution. I have tried some tools already put they need lot of manual work. I am looking for something to run every night by itself.

    Thanks
    skulltown
    Tagged:


Comments

  • Options
    #2
    BnB
    Registered Users Posts: 1,081 ✭✭✭BnB


    We looked into this a year ago and spent a good bit of time on it and we came to the conclusion that there was no Off the Shelf tool available that would help us. There are tools out there but they only help you scrape basic lists. The project we were doing was different to yours in that we were scraping websites monitoring changes in legislation etc but the principals of scraping are the same

    If you want to parse through a website and take data off it, you are really looking at writing the code yourself. This is the route we went down and it is still working for us 12 months later.

    One thing that could really help you though (which wasn't available to us) is that some websites (like ebay) have APIs available which allow you to interact directly with the database which would really help development https://go.developer.ebay.com/what-ebay-api

    On the other hand, you may find that some websites do not want to be scraped and will have systems in place to recognise & block bots that are scraping their site.


  • Options
    #3
    skulltown
    Registered Users Posts: 125 ✭✭skulltown


    Thanks for that BnB. I had a look at the Ebay API and I will try to work with that for now. Would you have any suggestions on technologies that I should be looking at as well?

    Thanks again


  • Options
    #4
    jmcc
    Registered Users Posts: 7,334 ✭✭✭jmcc


    skulltown wrote: »
    Thanks for that BnB. I had a look at the Ebay API and I will try to work with that for now. Would you have any suggestions on technologies that I should be looking at as well?

    Thanks again
    Best go the legitimate route. If you have PHP expertise, get a copy of "Webbots, Spiders and Screen Scrapers" by Michael Shrenk. If you have Python expertise, then read up on BeautifulSoup.

    Regards...jmcc


  • Options
    #5
    colosus1908
    Registered Users Posts: 58 ✭✭colosus1908


    Is there a place to get any of this book for free, maybe the first 2 chapters or something like that. Just for review purpose.
    I have found out that when it comes programming. You really need to connect with the writer to get the best out of the book.
    Thanks.
    jmcc wrote: »
    Best go the legitimate route. If you have PHP expertise, get a copy of "Webbots, Spiders and Screen Scrapers" by Michael Shrenk. If you have Python expertise, then read up on BeautifulSoup.

    Regards...jmcc


  • Options
    #6
    skulltown
    Registered Users Posts: 125 ✭✭skulltown


    colosus1908 wrote: »
    Is there a place to get any of this book for free, maybe the first 2 chapters or something like that. Just for review purpose.
    I have found out that when it comes programming. You really need to connect with the writer to get the best out of the book.
    Thanks.

    Amazon will let you look inside a couple of pages of the

    See:
    http://www.amazon.com/gp/product/1593271204/?tag=schrenkcom-20


  • Advertisement
Advertisement