ESB usage data

Happyhouse22

Recently set up ann account with ESB networks to monitor usage. I can download a hdf file which has usage in kilowatts for .5 intervals.

If I add up all these values should it be equivalent to my total usage in kilowatt hours? Or do I need to divide by two as it is in half hour increments?

rob w

https://www.boards.ie/discussion/2058317871/esb-usage-data

Yeah, you need to divide by 2 to give you the kwh value.

kwh = kw x time in hours

so = kw x 0.5hrs OR = kw/2

Wiggy

When I try to download the HDF, it seems to be restricting me to one weeks data per download.

I'm pretty sure this wasn't the case before.

Anyone else seeing this, or am I doing something wrong?

John.G

Can you post a screenshot of a portion of that hdf file.

John.G

https://www.boards.ie/discussion/comment/121231574#Comment_121231574

I wouldn' be too sure that that's the correct intrepretion of it.

IF it shows the energy consumption in kwh every 1/2 hour then to get the average power in kw for that 1/2 hour you X by 2, but to get the total usage you add up the total of the 1/2 hour periods. if you have a record of say the last 48 1/2 hour usage then you summate them to give a "daily" consumption. IF you could switch on a 3kw immersion then (assuning nothing else on) the 1/2 hour reading will be 1.5kwh but the power usage is 1.5X2, 3kw.

https://www.boards.ie/discussion/comment/121340094#Comment_121340094

You are incorrect. The half hour reading for your 3kW heater will be 3kW

Alun

https://www.boards.ie/discussion/comment/121340094#Comment_121340094

That's the thing though, the value in the HDF file isn't the usage in kWh for a 30 minute period as you might expect, it's the average power in kW over that 30 minute period. It's easily verifiable if you have a big load like a car charging and look at the figures in the HDF for that period.

Here's an excerpt from my HDF file when I started a charge at 23:00 on my car (a PHEV so only charges at approx. 3.6 kW) showing it clearly. The "Read Type" column in the HDF file even states explicitly "Active Import Interval (kW)"!

I see a recent small change on the website in that you can now display the usage graphs in "Calculated kWh" instead of kW presumably as a result of people complaining, so they're obviously aware of how it's confusing some people. The HDF file is unaffected though.

Calahonda52

https://www.boards.ie/discussion/comment/121340099#Comment_121340099

Do you want to rethink this?

https://www.boards.ie/discussion/comment/121340165#Comment_121340165

I don't. My EV charges at approx 7.2kW and the half hour data in the HDF file is 7.2kW

Alun

https://www.boards.ie/discussion/comment/121340173#Comment_121340173

No it isn't!! It's 7.2 kW average power over each 30 min period. Like I said it even says so in the text. Going by your reckoning it would have used 14.4 kWh in a one hour period which is impossible.

https://www.boards.ie/discussion/comment/121340181#Comment_121340181

I know. It wasn't my reckoning, it was John.G's above who reckoned you should double each half hour entry instead of halfing it.

TheChizler

This is a classic kW (power) vs kWh (energy) misreading, the assumption was that the ESB give you energy readings, you know,like what the meter directly records. But for some reason they calculate half hourly power readings. So to get the energy used you need to multiply by the time period in hours, the same as dividing by 2, to get the energy used in every half hour period.

Looking at my data there are a lot of intervals missed so I have lots of hour, 2 hour, 6 hour etc periods which makes using average power a pain, if they just gave kWh you could sum everything up without having to think about the time involved.

Alun

https://www.boards.ie/discussion/comment/121340233#Comment_121340233

Exactly, and add to that the odd way they give the power readings for the 30 minute period prior to the timestamp instead of after. You can't even combine all the readings for a specific date and divide by two to get daily consumption because you'll include the last 30 mins of the previous day and miss the last 30 mins of the day you're interested in. They couldn't have made it more confusing if they'd tried.

John.G

Certainly confusing, if you could note your meter reading at the end of any 1/2 hr period and then note it again at the end of say 4 of 1/2 hr periods, then compare the meter consumption with the sum of the 4 periods and see what you get?.

IF one uses 12kwh per day (24hr) then thats a average of 0.5kwh per hour. If the exact power usage was 0.5kw then presumably that power reading above would also read 0.5kw and to arrive at the actual consumption involves dividing by 2? 0.5*1440/30/2 = 12kwh?.

I suppose all its telling you is the average power demand for any 1/2 hour period.

Alun

I suppose all its telling you is the average power demand for any 1/2 hour period.

That's exactly it.

Calahonda52

I suppose all its telling you is the average power demand for any 1/2 hour period

Is it not a bit more than that as this is the number used for billing.

Any of you add up the monthly or two monthly and align with bill data?

Intel and other big users are billed on average power consumption per second.

John.G

And because, as per post #2, kwh = kw x time in hours then kwh = kw*30/60 or kw*0.5=kw/2.

Its a pity the smart meter doesnt display the kw, you can calculate it as the LED flashes 1000 times/kwh, I generally take the time in secs for 10 flashes, the calc is (number of flashes)X3.6/time in secs, for example if it takes 18secs to flash 10 times then the power, kw, = 10*3.6/18, 2.0kw. I think you can buy a device that just reads the LED flashing periods, does the calculation and displays/updates it probably every 30 secs or so.

Alun

For smart tariffs, it's unclear to me exactly how the electricity companies get the data they use for billing.

When I was on a 24hr tariff, I could see every 2-monthly reading on the ESB website, and these, of course, corresponded to the value used for billing purposes, but once I switched to a smart tariff, all that stopped.

The electricity companies clearly have access to the same data as found in the HDF file, so as to be able to provide the fancy usage charts on their websites, but whether they also use this for billing is unclear. I'd say it's unlikely mainly because adding up all those 30 min values over 2 months would lead to cumulative rounding errors, so I assume they receive 2 monthly readings as before, split into day, night and peak usage, but I don't know.

So whereas before I could check to make sure the readings used by my provider were correct, I can't do that any more. I could cobble together a spreadsheet or some other tool to analyse the HDF file but I'm not sure I can be bothered.

John.G

Have you, or anyone, compared the total 2 month billed amount with the actual metered amount used?, at least that would give a good idea of the sampling periods which may be as low as 2secs. I have a smart meter but its still read bimonthly and I also submit the readings, of course since the meter now only displays whole numbers unlike the old Ferranti meters which displayed 1/10ths kwhs so don't even know if the actual meter itself is accurate but assume it is as my billed usage hasn't changed since installation. There is some info on mine when/if I press a button, the total, 1646kwh, and 2 other numbers, T2,1126kwh, T2,140kwh and T1, 379kwh.

TheChizler

https://www.boards.ie/discussion/comment/121341018#Comment_121341018

A bit of guesswork here, but I think the reason it's presented chunks of half hour averages is due to the ad-hoc way the meters report data. AFAIK they report the register values for day night (and peak?) and export total kWh every update but the updates across millions of meters are staggered so not exactly on the half hour, and some updates never make it through due to reception issues or whatever. So if you were to plot the usage between updates over time it would have random timestamps and random length time periods and look messy. They average the data over half hour chunks to tidy it up and make it more user friendly. Personally I'd like access to both data sets.

John.G

Any idea of how often they measure the consumption for billing purposes? Or can they take the actual "readings" directly from the meter somehow or other.

whizbang

Period May-July is spot on in my bill..

takes a lot of messing with Excel to get it in usable format, still havent got it for the total (8 month) Hdf period yet.

staples7

Anyone have working to link to interpret the ESB HDF file?

I used https://onelv.retool.com/embedded/public/ee29e65c-fc32-492f-8baf-e4a3312a22c0 but no longer working

ongarite

Energypal.ie

bog master

I was told by ESB Networks that they supply smart meter readings to Electric Ireland and there have been instances where Electric Ireland made mistakes as my meter reading approx 2 units over four hours when there was a total blackout. You can get meter readings from ESB Networks if you sign up on their home page and readings are kW or Calculated kWh

coylemj

Whether you get kW or kWh in the download seems to depend on which company you're with. I'm with BGE and the 'interval value' number I get for each 30 minutes is described as 'kWh'.

So to see the total energy consumption (in kWh or domestic units) for one hour, I would need to add a pair of the numbers and to see the average power (in kW) I was consuming in one 30 minute period, I would double the number.

Example: if the number from 18:00 to 18:30 is 1.5 kWh, it means that the average power of my combined appliances during that 30 minutes was 3,000W and if the number for the following 30 minutes (18:30-19:00) was 0.25 kWh, it means my consumption dropped to 500W. And during the hour, I consumed a total of 1.75 units (kWh).

ColemanY2K

very odd that the HDF file has not been updated since the 23rd and the days prior to it are incomplete. its as if storm isha scrambled the system. 

coincidentally we received a flogas bill yesterday with no units, just the government credits. not sure if its my smart meter, esbn or flogas that's gone haywire. 

prosaic

I'm grappling with comparisons of HDF to units (kWh) off Electric Irl bill.

So, divide by 2 to convert half-hour power (kW) to 1 hour energy (kWh). But someone up thread said that when there are dropped periods, the calculation might be different? If a few periods are missing, is the avg power given for the period from last timestamp to current timestamp?

Oh, I delight in the calcultations - a new word for the occult process involved!

Am I right so far?

I still don't think my HDF numbers match the bill numbers. Someone else mentioned "when does your meter start?". Is there some synchronisation error? Like ElectricIrl start the day at 9:00 or 12:00 instead of midnight? Anyone got any insight on this?

coylemj

The 'divide by two' instruction is incorrect. The 'interval value' in my (BGE) spreadheeet shows the consumption in kWh for each 30 minute interval. To calculate the average consumption (in kW) for that 30 minute interval, you double the number.

For example, if I consumed 0.8 of a kWh in 30 minutes, that means that the average consumption of my combined appliances during that half hour was 1.6 kW.

kWh is like miles, kW is like miles per hour. A Watt is a rate of energy consumption equal to one Joule per second.

So kWh refers to a total amount of energy consumed while kW is a point in time measure of consumption.

To simplify it, if you run a 1 kW heater and nothing else for one hour, you will consume 1 kWh over that hour. Your report will show that you used 0.5 kWh in each of the 30 minute intervals. So for each 30 minutes interval, you double 0.5 to get 1.0 and that was the average consumption in kW during each half hour.

The smart meter chops up your consumption into 30 minute intervals. I don't know when it sends the data or how often but there is no concept of a start and end time.

prosaic

Coylemj, I think you must be seeing a different data file to me. The field "Read Type" has "Active Import Interval (kW)". Someone mentioned it might depend on which supplier you're with.

You'll see I referred to kW as power and kWh as energy. The divide by 2 is correct to convert from power to energy. If you're file gives kWh, then yes, your multiply by 2 works to get avg power.

What I'm wondering about are the periods where the data isn't reported for several hours at a time. In 2024, there have been a few long periods with no data. On four dates with longest missing data, they were 24hrs, 19hrs, 15hrs, 8 hrs. What happened on those days?

prosaic

My HDF file "kind of" matches what is on the electricity bill. It's close but off by a few kWh in all billing periods and all day/night/boost.

From approx July '23 to April '24, the difference between bill units and ESBN kWh is 45.5 kWh, which looks substantial enough.

TheChizler

https://www.boards.ie/discussion/comment/122326970#Comment_122326970

Sum two consecutive half hour kW values and divide by 2, not a straight divide by 2.

Each half hour kW value sustained for half an hour would have an effective unit of kWh/2, sum the two half hour period to get a full hour then divide by 2 to change the unit to kWh.

coylemj

Each half hour is not kW, it's kWh. A kilowatt is a rate of consumption at a point in time and does not get reported by the meter. Your kW number will go up and down during each half hour, depending on what appliances are in use. The kWh number is the cumulative energy consumed.

The 30 minute number in the downloaded file is the number of kWh you consumed in that half hour.

Using the car analogy, kW is your speed as indicated by the needle on your speedometer, kWh is the number on the odometer.

What the meter measures is total energy consumption in your house, it reports this every 30 minutes, in kWh.

prosaic

I understand what kW and kWh are. kW: thousands of joules per second. kWh: Energy used in 1 hour where the power is 1 kW (or by formula for different power levels (P in kW) and lengths of time (t in hours): E=P*t. 1 watt = 1 joule/s etc etc.

If you want more pedantic, 1 Joule = 1 Newton * 1 meter There are various other derived measures of energy, such as 1 coulomb * 1 volt. The definitions of SI units get a bit confusing here and it has changed a bit over the years since I were a lad. Energy is more fundamental than voltage so the volt is defined in terms of energy/power. Funny that, since we usually think of a volt as more basic than energy. End of pedantry.

The Chizler, you're summing two half hours and dividing by 2 which is the average of the power for the two half hours (assuming the quantity involved is power). To convert from power to energy, E=P*t: E=P*0.5. So it would be divide by 2 to turn it in to energy. If I run 1 kW for an hour, that's 1 kWh. If I run 1 kW for 0.5 hours, that's 0.5 kWh, etc.

The HDF file (that I have downloaded from ESBN), what it has is power, not energy. The top of the column says kW, not kWh. The numbers correspond, approximately, to what I have on my Electric Ireland bill, so I'm happy that the calculations are correct. Coylemj, I think you may be getting a different version of HDF where the column has kWh, where mine has kW.

My question currently is really: why is there a discrepancy between the totals given in the HDF and the units used as given in the electricity bills over a period of nearly 1 year.

In my earlier post I wasn't fully sure I had the HDF figured out, but I think I have it clear now.

coylemj

I discovered the reason for the confusion. While BGE only gives me the kWh number, ESB Networks will allow you to select kW or (Calculated) kWh when you register your MPRN and ask to download your usage data.

So I agree that to convert the 30 minute kW number to kWh, you would need to divide by two. Because if you ran a 1 kW appliance for all of the 30 minutes, the kW number would be 1 but you'd have only consumed 0.5 of a kWh in that 30 minutes.

MickH503

https://www.boards.ie/discussion/comment/122327173#Comment_122327173

Out of curiosity I checked the same thing with my data. I have a smart meter on a standard 24 hour plan. My meter reading gets "taken" in middle of every second month, but my billing period is a calendar month, so all my bills are estimated.

Comparing the ESB Networks HDF file data to the usage in my bills, it is sometimes higher, sometimes lower. The worst it was off by in past 9 months was 43kWh, the closest was 2kWh.

prosaic

Since getting the smart meter set to be used for power export (I think that was the time), it has been recording the meter remotely so we don't seem to get the absolute units reading on the bill any more. We get a number of units used for each of day/night/boost.

The usage period is given on the 1st page of bill. I make comparison in the HDF file for the exact same date range and day/night/boost periods.

I would expect there to be an exact match except for rounding errors. I would expect rounding errors to be fairly small and that these would cancel out over a few months.

TheChizler

https://www.boards.ie/discussion/comment/122329595#Comment_122329595

The meter reports energy consumption in units to the ESB over GSM but not at half hourly periods. Realistically there are going to be gaps in the data when reception is an issue, and they probably stagger transmit time so they don't get 1 million texts all at once whenever they report.

This data wouldn't look very nice presented in its raw form (though it would be nice to have the option) so the ESB average out usage across the day so you have a data point for estimated power for every 30 minutes of the day. The HDF presents this as a time series of power estimates every 30 minutes.

TheChizler

https://www.boards.ie/discussion/comment/122329785#Comment_122329785

The average of the power for two half hour periods is the same as the average power for an hour, which is the definition of a kWh. I think the confusion is some people are thinking in half hour energy blocks and some are thinking in hours. If you want to break your energy usage into half hour chunks then of course divide by 2 works.

prosaic

I use half hour energy block since that's the interval in the HDF file. Glad we're all square .(⌐⊙_⊙).

coylemj

I don't understand why ESB Networks say the kWh number (if that's what you ask for) in the HDF file is 'calculated' - surely the meter principally measures your consumption in kWh? Where does the kW number comes from?

prosaic

I didn't ask for kW. Someone mentioned, maybe up thread(?), that it might depend on which provider you're with.

I don't mind kW or kWh but I don't want to chop and change.

One reason to quote average kW would be where the time interval varies, e.g. an hour or two of missed data. It could make for easier scanning for electricity consumption spikes. If the interval sizes vary the energy varies but the power might tend to be more constant.

prosaic

The download page has updated it seems. You can get units used (actual readings), average kW or kWh.

This is very good to have and takes the guessing out of the calculations. Hopefully they will combine these all into a single download file in future. No need for separate files I think.

I haven't checked over the different versions (units/kW/kWh) as yet, nor am likely to in next month.