Tinkering; Like Guntering for the Connected Generation.

@rounders I've put together a CSV importer that works with DAFM CSV files, I also applied for and got API access from ICBF which will prove more useful in the near future, but for the moment generates CSV files that can also be uploaded to farmOS which saves the whole process of logging into Agfood/AIM every time I want to sync the herd profile. It still needs some refining but it's getting there.

@patsy_mccabe ICBF API might suit you better than web scraping. I just contacted them and told them what I wanted it for, they asked me for a few details and gave me access within a few days. You need a domain name and static IP though, I use a free domain from DuckDNS and the static IP may depend on your ISP if it can be requested.

The API returns JSON formatted data, it's relatively simple to convert from JSON to CSV which can be opened by Excel.

I'm using Node Red which is a visual programing tool that uses pre-built drag and drop nodes for common tasks and Javascript in special function nodes for anything custom.

The following is an example of converting JSON to CSV using Javascript it is a loop which reads each record and writes them one line at a time to CSV.

let outputMsgs = [];
var csv;
outputMsgs.push({payload:csv});

msg.payload._embedded.live_weight.forEach(data => {
    
    csv = data.animal_id +","+ data.weigh_date + "," + data.weight + "," + data.weight_type + "," + data.price;   
        outputMsgs.push({payload:csv});

    })

return [ outputMsgs ];

There are JSON to CSV nodes too but a little bit of Javascript allows you to extract and format the data as you like.

Authentication is the tricky bit though, but can be handled by Node Red too. The login process starts with a browser web login, then it sends a code to a URL hosted by Node Red on my server, this is used to request an access token which is needed for the API requests and a refresh token which is used to request a new access token and refresh without having to reauthenticate. The access tokens expire after a few minuets but refresh tokens last about 6 months, so you don't need to keep logging in through a web browser once connected.

I'm sure you'd figure out the API if you needed it, but since you have what you need working with web scraping then happy days. I just thought of your ICBF project when I started on mine. I've a tag reader on the way, I am going to try and automate the cattle weighing over the summer.

Yes, I guess so too, only tested at the desk so far. There will be no calves here for about a month.

Not entirely sure what to expect, I'm hoping the jackets themselves trap enough body heat to tell the difference between a calf with elevated temperature and the rest, I expect the base line will need to be set a little lower. Movement tracking can probably tell allot about which calves need extra looking out for too.

There is a robust version too, although I've gone with some of the cheaper ones for now anyway, if they last long enough for proof of concept then the robust version can be considered, but I hope the jackets help protect them too.

Yes Raspberry Pi for now, been experimenting with ESP32's as well, could end up with a Raspberry Pi as primary gateway and ESP32's as additional nodes to extend range and possibly indoor location tracking.

Hope the calves like Marvel 😁 (Pocket may end up needing to be a net material if cloth is to insulating of temperature.)

Very good @_Brian if she's an interest in electronics it's a great place to start, those elegoo sets are good value too.

12V-6V Buck stepdown converter.

12V car batteries, probably won't last as long as electric fence batteries, but obviously they are rechargeable, a small solar panel would keep it going a bit longer though.

Can definitely still get those batteries.

Should be 7V+ charged I'd imagine.