nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Child Item

Forecasting tomorrow's sunshine (to better use the solar)

Tree

Does anyone have any reliable ways of figuring out if there'll be good solar on the following day?

I'm used to reading the forecast for rain and wind, but when it comes to sun/cloud cover, I can't match it up at all to what ends up being generated on the day. I'm aiming to use the washing machine more on sunny mornings for example, and I set it up the night before, but half the time it's cloudy and there's not much solar energy and other times it's grand. It's probably going to be a non-issue come the summer, but while it's still early spring, I'd like to be able to plan my usage better.

Find more posts tagged with

Quick Links

Comments

SD_DRACULA

I wonder if the EV tariff will still be worth it with the considerable hikes coming in one week time

+100 to the standing charge and quite a bit on the others too, even with full discount.

Peak rate is a 🤬 joke.

connesha

As its past end of month, here's an update on how the met.ie forecasts are working out following the changes we talked through a few weeks back: using 7 of the last 12-day-window with similar radiation, and adjusting the window based on the sun angle (the sun angle hasn't kicked in yet; the "12d" in the chart shows the current window size)

And using these forecasts allowed the script to charge the battery automatically. I'm finding the readings for import as seen by the inverter differs somewhat from the actual. At the end of the day its what the meter/supplier sees that matters, so am using the monthly bill to judge success.

The script uses the EV period (2am to 5am) only for charging, i.e. its purpose is to avoid grid usage at all other times. So it will use this EV period to charge for (1) a poor day of generation, or (2) if the battery has gone low by 2am and needs small charge to get through to generation time in the morning.

There is about a 1kwh/day "leakage" in my system: while the inverter is trying to match house load on the fly, it misses it by approx 1kwh imported per day. So, this 1kwh per day spread across the non-EV period seems unavoidable.

March has been a great month.

This is with a 5kwh PureDrive battery. A bigger battery (20kwh coming soon I hope) would have reduced the EV period usage significantly for months like this past month, probably to approaching 3kwh. Although that wouldn't save much this month, it should have a decent effect on winter months: i.e. only EV period usage should rise in winter, and the usage in the more expensive times of the day should remain much the same. Lets see...

Jonathan

https://www.boards.ie/discussion/comment/118801324#Comment_118801324

Merged your comments into this thread where they are better suited.

System

This discussion was created from comments split from: Daily pv production.

mp3guy

https://www.boards.ie/discussion/comment/118801324#Comment_118801324

I find solcast much more accurate than the service built into HA's energy management system. Been using it for over a year now to do load shifting like championc mentioned and also decide on the method for heating hot water.

championc

https://www.boards.ie/discussion/comment/118801324#Comment_118801324

The use of the forecast for me is really to gauge the level of night time charging I need to do, to ensure that I get to 23:00 that evening without needing to import any daytime units, while also not over-charging and ending up exporting to the grid.

From when the clocks go forward until the end of BST, I really don't see the need to be worrying about night charging, except when a really crap day is forecast.

But as for the most accurate source of data, I really haven't got to the point of really looking into things in an absolute way

Nelbert

I'm seeing familiar style graphs here. I've just purchased an energy monitor and incorporated it in to home assistant.... Background elec usage is about 5-600 watts at any given time but anytime the heat pump kicks in it jumps a huge amount.

Has anyone used the solar forecast integration in home assistant (familiar graphs so thinking there is some HA users)? If so how accurate are the forecasts Vs your production? I'm trying to gauge it's usefulness as a tool to work out potential impact.

championc

Maybe time to start incorporating some automation type intelligence into your setup ?

graememk

That's my comparison, top 2 is forecast.solar, which usually is optimistic to say the least.

My met prediction is close, within a kwh or 2.

Will be interesting how temperature will come into play.

Eg in may and June my inverter clips.

But on the hot days in Aug it doesn't clip.

connesha

Happy St Patricks Day for tomorrow all! ☘️

Todays graph below. The last 3 days are when the "7 out of 12" change was made, and I've managed to stop tinkering with the algorithm since (which was no small feat!)

I think I'm at the point of not changing it for a while now, let it run, and see how it goes for a few weeks/months... Its been a lot fun getting here (can we call our predictors Alpha stage?), definitely learned a lot, and lots of good ideas with @bullit_dodger and @graememk especially.

Would be kinda interesting to see how the predictors that come with home assistant fare compared to these over time. Not sure if anyone watching here has past data, or is gathering future data on these. I hope we haven't wasted our time ;-) .. I don't think so.

And if anything from mine is of any use to anyone (theory/logic/code/mistakes), just give me a shout.

Time to move on to the next mini-project. Seriously, since got solar just a few weeks ago, my time has vanished, and the todo list is falling off the end of the page.

Now, to try to figure out home assistant... Installed it a couple of weeks back, spent a couple of evenings playing with it, but can't for the life of me figure out how it hangs together or how to do stuff with it... maybe it'll click eventually.

As first mini-integration, may try to get a home assistant screen/button/toggle/UI to apply an "away mode" to the script. Something simple, e.g. when HA sets AwayMode, the script reduces its daily_predicted_electricity_usage to a flat BASE_LOAD x 24. So then it won't charge at all at night time, if the forecast is even giving 5 or 6kwh (this last part should work as is once the predicted usage is reduced). Anyway, that's a different one...

connesha

This'll do it.

import pytz
from utils.utils import Utils
from astropy.coordinates import get_sun, AltAz, EarthLocation
from astropy.time import Time
from suntime import Sun

class SunAngles:

  def __init__(self, longitude, latitude):
     self.longitude = longitude
     self.latitude = latitude
     return

  def get_bounded_num_days_back_within_angle(self, angle, lower_limit, upper_limit):
     num_days = 0

     location = EarthLocation.from_geodetic(self.longitude, self.latitude)
     sun = Sun(self.latitude, self.longitude)

     solar_noon_time_today = self.get_solar_noon_for_date(sun, Utils.date_today())
     zen_angle_today = self.zen_angle_at_date(solar_noon_time_today, location)

     angle_sum = 0
     last_angle = zen_angle_today
     while num_days <= upper_limit:
        date = Utils.date_from_offset(-(num_days + 1))
        solar_noon_time = self.get_solar_noon_for_date(sun, date)
        zen_angle = self.zen_angle_at_date(solar_noon_time, location)
        diff_from_last = abs(last_angle - zen_angle)
        if angle_sum + diff_from_last > angle:
           break
        angle_sum += abs(last_angle - zen_angle)
        last_angle = zen_angle
        num_days += 1

     num_days = Utils.clamp_value(num_days, lower_limit, upper_limit)
     return num_days

  def get_solar_noon_for_date(self, sun, date):
     tz_ireland = pytz.timezone('Europe/Dublin')
     sunrise_time = sun.get_sunrise_time(date)
     sunset_time = sun.get_sunset_time(date)
     sunrise_time_local_tz = sunrise_time.astimezone(tz_ireland)
     sunset_time_local_tz = sunset_time.astimezone(tz_ireland)

     # midpoint is technically not 100% accurate for SolarNoon, but it's very close, and plenty good enough for this purpose
     solar_noon = sunrise_time_local_tz + (sunset_time_local_tz - sunrise_time_local_tz) / 2
     return solar_noon


  def zen_angle_at_date(self, date_time, location):
     altaz = AltAz(obstime=date_time, location=location)
     astropy_time = Time(date_time)
     zen_ang = get_sun(astropy_time).transform_to(altaz).zen
     degrees = zen_ang.degree
     return degrees

ESTIMATE_NUM_DAYS_HISTORY_MIN = 12
ESTIMATE_NUM_DAYS_HISTORY_MAX = 21
ESTIMATE_NUM_DAYS_USE = 7
ESTIMATE_SUN_ANGLE_MAX = 4.7

  sun_angles = SunAngles(config.LONGITUDE, config.LATITUDE)
  num_days_history = sun_angles.get_bounded_num_days_back_within_angle(ESTIMATE_SUN_ANGLE_MAX, ESTIMATE_NUM_DAYS_HISTORY_MIN, ESTIMATE_NUM_DAYS_HISTORY_MAX)

import datetime

class Utils:

  def __init__(self):
     return

  @staticmethod
  def date_from_offset(offset=0):
     return datetime.date.today() + datetime.timedelta(days=offset)

  @staticmethod
  def date_today():
     return datetime.date.today()

  @staticmethod
  def date_tomorrow():
     return Utils.date_from_offset(1)

  @staticmethod
  def date_yesterday():
     return Utils.date_from_offset(-1)

  @staticmethod
  def clamp_value(value, min_value, max_value):
     if value > max_value:
        return max_value
     elif value < min_value:
        return min_value
     else:
        return value

bullit_dodger

https://www.boards.ie/discussion/comment/118787995#Comment_118787995

LOL - yeah, it's easy to get sucked into stuff mate. To be fair though, it's a lot easier to work out a date window than it is to compute the hourly sun angles from the Met Eireann forecast....twice (as I have two arrays east & west) :-)

connesha

😁 It all seems so simple before we start... and then get sucked in bit by bit...

graememk

And you were saying I'm overthinking it looking to calculate the angle of the sun and panels!

Although what I'm running with is close enough though. I think I can calculate the sun position for any given hour, just have to port it from excel.

I'll see if the mood take me there

bullit_dodger

Ahh, that's a good point about the inverter clipping and it creating a "false ceiling" on the actuals. Hadn't thought of that. For me, it's not going to be an issue. I've a 5Kw inverter and I have (currently) 5.3 Kwp in panels, but they are split in an east/west config so it's unlikely that I'll ever get to max inverter load. Now I do have some tentative plans to add 2x more panels to the east array, bringing me up to 6.1Kwp, so when that happens there's a slight chance that I might hit that 5Kw limit, but it'll be rare I reckon.

The one good thing that we have going for us in this situation is that for the actual day in question where you have clipping, your production is going to be well above any decision making that you will be doing with setting battery charging levels or whatever. If you forecast is 40Kwhr and you "only" get 35kwhr due to inverter clipping.....I think we'll still be happy :-) Maybe not with the forecast, but with the production. Your right though, if that is then built into the averages going forward, it would take a few days for it to filter out of the history , so it would negatively affect your forecasts until 10-12 days or however many you have in the look back range is passed.

I like that idea you have about the 4.7 degrees change. That's quite clever. Wish I'd thought of that! Pretty slick......so now just need to come up with a way of computing the current date window. Should be able to work it out with sine/cosine and then the date difference from March 22nd/Sept21st.

connesha

Yea, I'd be interested to to see how the closest 7 in the last 12-14 days looks across your data-set

Agree, each way we do it will probably throw up some positives and negatives, and the weather forecast data is only a forecast after all... not an actual radiation at your exact home...

I do think we're very close now, and are certainly well into the region of having "a pretty accurate estimate, most of the time",

I did make one further change last night to the way the estimate is done. While it is live now, it won't have any impact on estimates until about a month from now.

The background:

The linear ratio we assume between radiation and generation will likely be problematic for some setups, especially in summer. Taking my own setup as an example to illustrate:

9.4kwp, split evenly East-West, into a 6kw inverter, on a low pitch roof (around 20 deg). Because of the low pitch, both strings can generate a lot when the sun is high, and I'm expecting clipping on good summer days (even this week I saw a few peaks of 6kw production).

If the inverter is greater than or equal to what panels will practically produce, then no problem. But if the inverter is less, then there is a cap, and the radiation to generation ratio will no longer be linear.

So, if have a bunch of recent days that were clipped, and now want to make an estimate using todays low radiation, then todays estimate will be off. Using our original algorithm of taking the last 10 days, this will be a problem. The 7 out of 12 was trying to help, and should be a little better (considering it will exclude the more severely clipped ones), but I think we can do better again.

The change:

Conveniently, this effect will happen mostly when the sun angle is not changing so much day-to-day, so, perhaps we can use a wider window to take our previous days data from.

The algorithm now no longer hard-codes 12 as the number of days to go back (the window).

Instead, it walks backwards from today, adding up the difference in noon sun angles per day, and when the difference reaches 4.7 deg, that's how many days it goes back.

Why 4.7? Right now is the time of the year with greatest rate of change. 12 days back from today is a 4.7deg difference, so that's the max allowed.

Come April 20th, the window will rise to 13 days, and by full summer it would be over a couple of months wide if left unbounded. (I've imposed an upper cap of 21 days, as there are potentially other factors besides just angle)

By choosing 7 out of 21 previous days, it'll be able to avoid more dissimilar days, and in particular the low radiation days will be less influenced by the high radiation (clipped) days, and vice-versa.

So, in summary:

Summer (and mid-winter): pick the 7 days with the closest radiation out of the last 21 days
Spring/Autumn (more changeable sun angle): pick the 7 days with the closest radiation out of the last 12 days
And scale between those

Todays forecast 19.985 kwh

bullit_dodger

No, it's original algorithm where I sum up the radiation for the past 10 days and get the average. Then work out the similar 10 day moving average for the "actuals", work out what the Kwhr/1000 units of solar is.......and then compute the forecast based off the forecast for the day. No filtering.

Tomorrow if I get time, I'll have a crack at writing some code to do the other algorithms you mentioned above such as filtering out +/- 1000 figures along with the closest 7 in the last 12-14 days.

I do think they will yield different results. You may even see some days becoming more exact along with other days on the same chart diverging more. Should be able to chart them and then compare the deltas in the various algorithms to the actuals and see which algorithm is more exact. Course I realize that even with the data points that I have, I'm still on the low side. You'd need 100's (1000's) of points to see which algorithm historically works best.

But, I think if you get it "fairly right" 6 out of 7 days a week, then that's a win.....and it's kinds fun for a geek like me :-)

connesha

That graph is awesome.

Which exact estimation formula is the graph?

The +/- 1000 you mentioned in previous post?
Or
The 7 closest radiations out of last 12 days?

bullit_dodger

So the historic data I've stored for my latt/long going back to last August when I setup that VM in Azure to host the tool. I did it that way as I wanted it to poll every 1 hr and stay on 24x7. As a result I've not been looking at the Met Eireann history site as i have most of what I want already. (I actually didn't know it existed - so thanks)

That said, since I have all the data myself.... I extracted back towards the start of Jan and computed what the forecasts would have been with the same algorithm.

That's a decent enough correlation to be fair.

I've been giving some thought connesha to the idea that you have of the similar days (but outside of the window) verses dis-similar days (but inside the window). I think you actually need the dissimilar ones as ultimately what your doing with the 10 point moving average is to "smooth" the radiation to actuals ratio.

I'm speculating: You panels will behave differently with different angles (esp with not direct sunlight). I'm not sure that say 1000 radiation on a bright day in Jan, would be the same as 1000 radiation on a murky day in March, even taking the sun's angle to the panels into consideration. The panels generate some power from diffuse light, and or inverter inefficiencies would be different between Jan and (say) April. I've zero data to support that theory by the way, just speculating. So I think taking the last 10-14 days and computing off that (dissimilar and all) gives you a more rounded figure to base your forecast off.

connesha

Its this where you pull the historical data?

https://www.met.ie/climate/available-data/historical-data

I'm struggling to find March GlobalRadiation numbers at any stations. Local stations don't seem to have historic GlobalRadiation, and even the airport only has for Feb and earlier. What station do you use?

Todays final here BTW:

Too early to say if todays estimate calculation change is a success, or if today was just lucky... I guess will see in another week or 2...

Tomorrow is currently looking exactly the same as today. Thursday onwards look to be picking up a lot. But each forecast is locked in in the middle of each night, so will change ...

Separately, was thinking about how to make more use of older data; specifically, we rely on ratios between radiation and actual generation to make an estimate, but are only using a small set of recent days to predict. So, toying with using the suns angle compared to panel angle today vs each older_date. Maybe the Zenith Angle at Solar Noon compared to panel angle on both days, which may allow an adjustment to be made for each older day to make it usable.

Maybe this could allow for going back further in time to use a bigger data set to average? (Or maybe its just not worth it, for one, because the met data can change 10 minutes after we use it...)

Not fully formed yet, but am currently logging out the Time of SolarNoon and its Zenith Angle for a range of days. If that data, or the script that produced it sparks anything, I can send it on.

It may then allow us to really zone in on using previous days with similar radiation. For instance, get the last 20 days that are within 250 radiation of today, even if some of them are 2 months old. Adjust the each older one for sun angles, and get your estimate from that.

Currently, I'm prioritising recent days over similar days, in that its only looking back 12 days, and takes the most-similar ones from that (even though they could all be 2000 radiation out, for example)

graememk

hows tomorrows forecast looking?

Mines not great at all! although 24kwh for wed.

bullit_dodger

I like the "days like today" modification. A rough filter would be say +/- 1000 away from the Met Eireann forecast. I'll run that algorithm through my histroical data and see what it looks like to me.

E.g if Met Eireann is saying 2850, then you will accept previous days where forecast was in the 1850 -> 3850 range.

I do know that sadly in my own case that wouldn't have helped me for last Saturday estimate, as it was more a case of "when in the day" the solar radiation fell rather than how much. I think the forecast from Met Eireann was sort of ok, but the timing screwed me up.

Looking at your chart above though, you're (for the most part) getting pretty nice forecasts. The main outliers being March 7th and March 12th. Have you pulled the daily historical charts for those and seen if there's anything similar to what I noticed on Saturday?

connesha

Todays estimate in chart below.

Note, the algorithm to forecast was updated before this estimate along the lines of what I posted a couple of days ago, to "use days more similar to today". I found the previous model was over-influenced by 1 very good/bad day, and "think" its probably useful to try to compare more like-to-like days.

Since last night it is:

Get the last 12 days data.
From that 12, take the 7 days where the radiation is most similar to today
Sum the radiation for those 7 days, sum the actual_wh
estimate_factor = sum_of_actual_wh / sum_of_radiation

12 and 7 are not really based on anything more scientific than having enough data to average, and not going back too far.

I believe this is somewhat similar to yours bullit_dodger, with the addition of the "days like today"?

connesha

Yea winter will be challenging some days. But what I mean is that the sum of charging a big battery at night, plus the miserable production for the day, should get through a lot of days and evenings

I was on the same tarrif a few weeks before getting solar, so have some data on how much, and at what time, we're consuming, and how habits can sway that. During that period, I set just about everything that had a timer on it to run between 2am and 5am: Immersion, dishwasher, washing machine, (dryer sometimes, although ours is a bit too loud for nighttime, and the dehumidifier does most of our clothes drying very economically anyways). It wasnt any extra effort, and TBH actually found it more convenient for some appliances to run at night - like having the default 3h dishwasher program be done when we get up, and having the hot water boiling in the morning, besides needing to wait for a bit of sun to get from warm to hot. The appliance start times were staggered a bit, and some appliances use the heavier load earlier in the cycle

By doing this, and not trying to reduce other load, we averaged in the region of 40% of consumption coming in between 2am and 5am (out of some 16kwh or 18kwh per 24 hours total).

This is all based on the smart meter readings, 30 min granularity, downloaded from the bordgais website.

So adding that all up: having e.g. 12kwh usable available in the battery, maybe generate 2 or 3kwh, taking off X% for the conversion inefficiencies, and its right in the ballpark.

Of course need to watch that don't overload the main fuse, and how hard want to drive the battery. Charging at the max 100a would get it almost full at night in 3 hours, so that'd bring the battery to 15kwh+ usable. I guess can see how hard the panels drive it during the summer, and if they put 100a into it for a few hours, I don't see why not at night time. I know its advisable to go easier on them for prolonging their life, but I'm learning towards "I have it, so I'm going to use it as much as I need it"

Well, that's a lot of theory... the proof will have to wait...

bullit_dodger

(I'd still be doing it for fun whether it makes financial sense or not)

That's admirable. Too many people only do stuff because there's a "fiscal payback", when in truth there's value in the actual doing. The experimentation, the thinking, the joy in getting something to work etc The DIY batteries are a good example of that. I'm on the fence myself on that one, but ultimately would like to do something similar to you and get myself 20Kwhr for €4-5K and keep my existing 8.2kwhr operational somehow. Can be tricky to get two storage systems to run, but it's doable I believe. So I'm curious to see how yours works out. DrPhil did a great job on his 6 months ago.

Why so much storage? Well similar to yourself I'd like to get to 100% self-consumption. Being able to "ride through" a bad day in March is part of that. Longer term I will have an EV , as will we all ......and I have no issues in charging an EV 10-15Kwhr from a house battery, if the car was elsewhere for the daytime. I get there's losses etc. So that's the plan. Course, longer term the EV's might be the store if they standardize on a protocol, we'll see.

Winter - that's just not possible to so 100% self consumption with solar unless your making hydrogen or something to do longer term storage in the summer. Not so sure my neighbors would like the Hindenburg out my shed in the back yard. LOL

connesha

Very nice!

Yea, the money side may not always add up huge, but it is a very interesting project. Same motivation here.

My ultimate goal is to see how close I can get to using only generated electricity, or 5 cent grid electricity (2am to 5am EV plan), for the full 12 months of the year. Where as much of the generated electricity as possible is used. 20kwh DIY battery cells ordered a couple of weeks ago to support this effort... While these forecasts dont make a huge difference for me now, with current 5kwh battery (4kwh usable), the same scripts figuring out whether to charge 3wkh or 12kwh at night into the larger battery will definitely make a difference (as will automatically triggering an Eddi boost at night based on forecasts, which is also on the list...) I'm guessing all this together could mean some financial savings, but haven't calculated any of the financials out at all. (I'd still be doing it for fun whether it makes financial sense or not)

bullit_dodger

Actually that chart you have there connesha is something I should have thought of myself.....here's my own actual verses forecast. Shows it much better this way.

I'm pretty happy with that to be fair. Yeah, the 12th (yesterday) I was a bit out, as per the reasons I gave above that the "good" sunshine was in the morning on my smaller array and then the wet afternoon was on the west array. Had I a 50-50 split, I reckon I'd have been closer to the forecast. Course summing the radiation forecast into 1x figure for the day doesn't take "time of day" into consideration - but overall, I think it's good enough to use it in decision making.

Aside: I worked out the money side before for optimizing your batteries SOC off a forecast. You might (might) save about a fiver in march. Outside of this in April/May most people won't charge batteries of night rate anyway, but this isn't why I do it. I do it cause I like to see the fun in getting it to work, not to save a fiver:-)

connesha

@graememk , I can easily add another line into the graph for your calc with my data if that's any use to you?

connesha

Last 10 days, incl today, since started forecasting.

Past 2 days were less accurate.

bullit_dodger

So another one today which was a fair chunk off. I'd a figure of 2891 forecasted by Met Eireann which should have yielded 13.3kwhr. Sadly not even close. Actuals today was 8.2Kwhr.

It was real nice this morning, blue skies until 11-12am but then there was a massive rain storm that came through when the six nations game was on about 4pm. I was giving this a bit of thought, you see my East array is only 1.5kwp and my West Array is 3.8Kwp.

You can see the nice smooth(ish) line on the east array up until 11am. If my East and west arrays were reversed, I'd say I'd have been closer to the forecast as it was very much a chalk and cheese difference between morning and afternoon, and when the big arrary would be yielding it's max.....it was lashing rain.

It's a bit of a challenge that, but I might be able to code up a solution (going the hourly route and then summing it up) - have to think about it.

By the way, tomorrow (so far) looks awesome and setup for best day of the year. Which is odd as it's forecasting rain for Dublin. (3660 at the moment, which equates to 16.1Kwhr for me - I'm skeptical :-)