Forecasting tomorrow's sunshine (to better use the solar)

connesha

@graememk , I can easily add another line into the graph for your calc with my data if that's any use to you?

bullit_dodger

Actually that chart you have there connesha is something I should have thought of myself.....here's my own actual verses forecast. Shows it much better this way.

I'm pretty happy with that to be fair. Yeah, the 12th (yesterday) I was a bit out, as per the reasons I gave above that the "good" sunshine was in the morning on my smaller array and then the wet afternoon was on the west array. Had I a 50-50 split, I reckon I'd have been closer to the forecast. Course summing the radiation forecast into 1x figure for the day doesn't take "time of day" into consideration - but overall, I think it's good enough to use it in decision making.

Aside: I worked out the money side before for optimizing your batteries SOC off a forecast. You might (might) save about a fiver in march. Outside of this in April/May most people won't charge batteries of night rate anyway, but this isn't why I do it. I do it cause I like to see the fun in getting it to work, not to save a fiver:-)

connesha

Very nice!

Yea, the money side may not always add up huge, but it is a very interesting project. Same motivation here.

My ultimate goal is to see how close I can get to using only generated electricity, or 5 cent grid electricity (2am to 5am EV plan), for the full 12 months of the year. Where as much of the generated electricity as possible is used. 20kwh DIY battery cells ordered a couple of weeks ago to support this effort... While these forecasts dont make a huge difference for me now, with current 5kwh battery (4kwh usable), the same scripts figuring out whether to charge 3wkh or 12kwh at night into the larger battery will definitely make a difference (as will automatically triggering an Eddi boost at night based on forecasts, which is also on the list...) I'm guessing all this together could mean some financial savings, but haven't calculated any of the financials out at all. (I'd still be doing it for fun whether it makes financial sense or not)

bullit_dodger

(I'd still be doing it for fun whether it makes financial sense or not)

That's admirable. Too many people only do stuff because there's a "fiscal payback", when in truth there's value in the actual doing. The experimentation, the thinking, the joy in getting something to work etc The DIY batteries are a good example of that. I'm on the fence myself on that one, but ultimately would like to do something similar to you and get myself 20Kwhr for €4-5K and keep my existing 8.2kwhr operational somehow. Can be tricky to get two storage systems to run, but it's doable I believe. So I'm curious to see how yours works out. DrPhil did a great job on his 6 months ago.

Why so much storage? Well similar to yourself I'd like to get to 100% self-consumption. Being able to "ride through" a bad day in March is part of that. Longer term I will have an EV , as will we all ......and I have no issues in charging an EV 10-15Kwhr from a house battery, if the car was elsewhere for the daytime. I get there's losses etc. So that's the plan. Course, longer term the EV's might be the store if they standardize on a protocol, we'll see.

Winter - that's just not possible to so 100% self consumption with solar unless your making hydrogen or something to do longer term storage in the summer. Not so sure my neighbors would like the Hindenburg out my shed in the back yard. LOL

connesha

Yea winter will be challenging some days. But what I mean is that the sum of charging a big battery at night, plus the miserable production for the day, should get through a lot of days and evenings

I was on the same tarrif a few weeks before getting solar, so have some data on how much, and at what time, we're consuming, and how habits can sway that. During that period, I set just about everything that had a timer on it to run between 2am and 5am: Immersion, dishwasher, washing machine, (dryer sometimes, although ours is a bit too loud for nighttime, and the dehumidifier does most of our clothes drying very economically anyways). It wasnt any extra effort, and TBH actually found it more convenient for some appliances to run at night - like having the default 3h dishwasher program be done when we get up, and having the hot water boiling in the morning, besides needing to wait for a bit of sun to get from warm to hot. The appliance start times were staggered a bit, and some appliances use the heavier load earlier in the cycle

By doing this, and not trying to reduce other load, we averaged in the region of 40% of consumption coming in between 2am and 5am (out of some 16kwh or 18kwh per 24 hours total).

This is all based on the smart meter readings, 30 min granularity, downloaded from the bordgais website.

So adding that all up: having e.g. 12kwh usable available in the battery, maybe generate 2 or 3kwh, taking off X% for the conversion inefficiencies, and its right in the ballpark.

Of course need to watch that don't overload the main fuse, and how hard want to drive the battery. Charging at the max 100a would get it almost full at night in 3 hours, so that'd bring the battery to 15kwh+ usable. I guess can see how hard the panels drive it during the summer, and if they put 100a into it for a few hours, I don't see why not at night time. I know its advisable to go easier on them for prolonging their life, but I'm learning towards "I have it, so I'm going to use it as much as I need it"

Well, that's a lot of theory... the proof will have to wait...

connesha

Todays estimate in chart below.

Note, the algorithm to forecast was updated before this estimate along the lines of what I posted a couple of days ago, to "use days more similar to today". I found the previous model was over-influenced by 1 very good/bad day, and "think" its probably useful to try to compare more like-to-like days.

Since last night it is:

• Get the last 12 days data.
• From that 12, take the 7 days where the radiation is most similar to today
• Sum the radiation for those 7 days, sum the actual_wh
• estimate_factor = sum_of_actual_wh / sum_of_radiation

12 and 7 are not really based on anything more scientific than having enough data to average, and not going back too far.

I believe this is somewhat similar to yours bullit_dodger, with the addition of the "days like today"?

bullit_dodger

I like the "days like today" modification. A rough filter would be say +/- 1000 away from the Met Eireann forecast. I'll run that algorithm through my histroical data and see what it looks like to me.

E.g if Met Eireann is saying 2850, then you will accept previous days where forecast was in the 1850 -> 3850 range.

I do know that sadly in my own case that wouldn't have helped me for last Saturday estimate, as it was more a case of "when in the day" the solar radiation fell rather than how much. I think the forecast from Met Eireann was sort of ok, but the timing screwed me up.

Looking at your chart above though, you're (for the most part) getting pretty nice forecasts. The main outliers being March 7th and March 12th. Have you pulled the daily historical charts for those and seen if there's anything similar to what I noticed on Saturday?

graememk

hows tomorrows forecast looking?

Mines not great at all! although 24kwh for wed.

connesha

Its this where you pull the historical data?

https://www.met.ie/climate/available-data/historical-data

I'm struggling to find March GlobalRadiation numbers at any stations. Local stations don't seem to have historic GlobalRadiation, and even the airport only has for Feb and earlier. What station do you use?

Todays final here BTW:

Too early to say if todays estimate calculation change is a success, or if today was just lucky... I guess will see in another week or 2...

Tomorrow is currently looking exactly the same as today. Thursday onwards look to be picking up a lot. But each forecast is locked in in the middle of each night, so will change ...

Separately, was thinking about how to make more use of older data; specifically, we rely on ratios between radiation and actual generation to make an estimate, but are only using a small set of recent days to predict. So, toying with using the suns angle compared to panel angle today vs each older_date. Maybe the Zenith Angle at Solar Noon compared to panel angle on both days, which may allow an adjustment to be made for each older day to make it usable.

Maybe this could allow for going back further in time to use a bigger data set to average? (Or maybe its just not worth it, for one, because the met data can change 10 minutes after we use it...)

Not fully formed yet, but am currently logging out the Time of SolarNoon and its Zenith Angle for a range of days. If that data, or the script that produced it sparks anything, I can send it on.

It may then allow us to really zone in on using previous days with similar radiation. For instance, get the last 20 days that are within 250 radiation of today, even if some of them are 2 months old. Adjust the each older one for sun angles, and get your estimate from that.

Currently, I'm prioritising recent days over similar days, in that its only looking back 12 days, and takes the most-similar ones from that (even though they could all be 2000 radiation out, for example)

bullit_dodger

So the historic data I've stored for my latt/long going back to last August when I setup that VM in Azure to host the tool. I did it that way as I wanted it to poll every 1 hr and stay on 24x7. As a result I've not been looking at the Met Eireann history site as i have most of what I want already. (I actually didn't know it existed - so thanks)

That said, since I have all the data myself.... I extracted back towards the start of Jan and computed what the forecasts would have been with the same algorithm.

That's a decent enough correlation to be fair.

I've been giving some thought connesha to the idea that you have of the similar days (but outside of the window) verses dis-similar days (but inside the window). I think you actually need the dissimilar ones as ultimately what your doing with the 10 point moving average is to "smooth" the radiation to actuals ratio.

I'm speculating: You panels will behave differently with different angles (esp with not direct sunlight). I'm not sure that say 1000 radiation on a bright day in Jan, would be the same as 1000 radiation on a murky day in March, even taking the sun's angle to the panels into consideration. The panels generate some power from diffuse light, and or inverter inefficiencies would be different between Jan and (say) April. I've zero data to support that theory by the way, just speculating. So I think taking the last 10-14 days and computing off that (dissimilar and all) gives you a more rounded figure to base your forecast off.

connesha

That graph is awesome.

Which exact estimation formula is the graph?

• The +/- 1000 you mentioned in previous post?
• Or
• The 7 closest radiations out of last 12 days?

bullit_dodger

No, it's original algorithm where I sum up the radiation for the past 10 days and get the average. Then work out the similar 10 day moving average for the "actuals", work out what the Kwhr/1000 units of solar is.......and then compute the forecast based off the forecast for the day. No filtering.

Tomorrow if I get time, I'll have a crack at writing some code to do the other algorithms you mentioned above such as filtering out +/- 1000 figures along with the closest 7 in the last 12-14 days.

I do think they will yield different results. You may even see some days becoming more exact along with other days on the same chart diverging more. Should be able to chart them and then compare the deltas in the various algorithms to the actuals and see which algorithm is more exact. Course I realize that even with the data points that I have, I'm still on the low side. You'd need 100's (1000's) of points to see which algorithm historically works best.

But, I think if you get it "fairly right" 6 out of 7 days a week, then that's a win.....and it's kinds fun for a geek like me :-)

connesha

Yea, I'd be interested to to see how the closest 7 in the last 12-14 days looks across your data-set

Agree, each way we do it will probably throw up some positives and negatives, and the weather forecast data is only a forecast after all... not an actual radiation at your exact home...

I do think we're very close now, and are certainly well into the region of having "a pretty accurate estimate, most of the time",

I did make one further change last night to the way the estimate is done. While it is live now, it won't have any impact on estimates until about a month from now.

The background:

The linear ratio we assume between radiation and generation will likely be problematic for some setups, especially in summer. Taking my own setup as an example to illustrate:

9.4kwp, split evenly East-West, into a 6kw inverter, on a low pitch roof (around 20 deg). Because of the low pitch, both strings can generate a lot when the sun is high, and I'm expecting clipping on good summer days (even this week I saw a few peaks of 6kw production).

If the inverter is greater than or equal to what panels will practically produce, then no problem. But if the inverter is less, then there is a cap, and the radiation to generation ratio will no longer be linear.

So, if have a bunch of recent days that were clipped, and now want to make an estimate using todays low radiation, then todays estimate will be off. Using our original algorithm of taking the last 10 days, this will be a problem. The 7 out of 12 was trying to help, and should be a little better (considering it will exclude the more severely clipped ones), but I think we can do better again.

The change:

Conveniently, this effect will happen mostly when the sun angle is not changing so much day-to-day, so, perhaps we can use a wider window to take our previous days data from.

The algorithm now no longer hard-codes 12 as the number of days to go back (the window).

Instead, it walks backwards from today, adding up the difference in noon sun angles per day, and when the difference reaches 4.7 deg, that's how many days it goes back.

Why 4.7? Right now is the time of the year with greatest rate of change. 12 days back from today is a 4.7deg difference, so that's the max allowed.

Come April 20th, the window will rise to 13 days, and by full summer it would be over a couple of months wide if left unbounded. (I've imposed an upper cap of 21 days, as there are potentially other factors besides just angle)

By choosing 7 out of 21 previous days, it'll be able to avoid more dissimilar days, and in particular the low radiation days will be less influenced by the high radiation (clipped) days, and vice-versa.

So, in summary:

• Summer (and mid-winter): pick the 7 days with the closest radiation out of the last 21 days
• Spring/Autumn (more changeable sun angle): pick the 7 days with the closest radiation out of the last 12 days
• And scale between those

Todays forecast 19.985 kwh

bullit_dodger

Ahh, that's a good point about the inverter clipping and it creating a "false ceiling" on the actuals. Hadn't thought of that. For me, it's not going to be an issue. I've a 5Kw inverter and I have (currently) 5.3 Kwp in panels, but they are split in an east/west config so it's unlikely that I'll ever get to max inverter load. Now I do have some tentative plans to add 2x more panels to the east array, bringing me up to 6.1Kwp, so when that happens there's a slight chance that I might hit that 5Kw limit, but it'll be rare I reckon.

The one good thing that we have going for us in this situation is that for the actual day in question where you have clipping, your production is going to be well above any decision making that you will be doing with setting battery charging levels or whatever. If you forecast is 40Kwhr and you "only" get 35kwhr due to inverter clipping.....I think we'll still be happy :-) Maybe not with the forecast, but with the production. Your right though, if that is then built into the averages going forward, it would take a few days for it to filter out of the history , so it would negatively affect your forecasts until 10-12 days or however many you have in the look back range is passed.

I like that idea you have about the 4.7 degrees change. That's quite clever. Wish I'd thought of that! Pretty slick......so now just need to come up with a way of computing the current date window. Should be able to work it out with sine/cosine and then the date difference from March 22nd/Sept21st.

graememk

And you were saying I'm overthinking it looking to calculate the angle of the sun and panels!

Although what I'm running with is close enough though. I think I can calculate the sun position for any given hour, just have to port it from excel.

I'll see if the mood take me there

connesha

😁 It all seems so simple before we start... and then get sucked in bit by bit...

bullit_dodger

https://www.boards.ie/discussion/comment/118787995#Comment_118787995

LOL - yeah, it's easy to get sucked into stuff mate. To be fair though, it's a lot easier to work out a date window than it is to compute the hourly sun angles from the Met Eireann forecast....twice (as I have two arrays east & west) :-)

connesha

This'll do it.

import pytz
from utils.utils import Utils
from astropy.coordinates import get_sun, AltAz, EarthLocation
from astropy.time import Time
from suntime import Sun

class SunAngles:

  def __init__(self, longitude, latitude):
     self.longitude = longitude
     self.latitude = latitude
     return

  def get_bounded_num_days_back_within_angle(self, angle, lower_limit, upper_limit):
     num_days = 0

     location = EarthLocation.from_geodetic(self.longitude, self.latitude)
     sun = Sun(self.latitude, self.longitude)

     solar_noon_time_today = self.get_solar_noon_for_date(sun, Utils.date_today())
     zen_angle_today = self.zen_angle_at_date(solar_noon_time_today, location)

     angle_sum = 0
     last_angle = zen_angle_today
     while num_days <= upper_limit:
        date = Utils.date_from_offset(-(num_days + 1))
        solar_noon_time = self.get_solar_noon_for_date(sun, date)
        zen_angle = self.zen_angle_at_date(solar_noon_time, location)
        diff_from_last = abs(last_angle - zen_angle)
        if angle_sum + diff_from_last > angle:
           break
        angle_sum += abs(last_angle - zen_angle)
        last_angle = zen_angle
        num_days += 1

     num_days = Utils.clamp_value(num_days, lower_limit, upper_limit)
     return num_days

  def get_solar_noon_for_date(self, sun, date):
     tz_ireland = pytz.timezone('Europe/Dublin')
     sunrise_time = sun.get_sunrise_time(date)
     sunset_time = sun.get_sunset_time(date)
     sunrise_time_local_tz = sunrise_time.astimezone(tz_ireland)
     sunset_time_local_tz = sunset_time.astimezone(tz_ireland)

     # midpoint is technically not 100% accurate for SolarNoon, but it's very close, and plenty good enough for this purpose
     solar_noon = sunrise_time_local_tz + (sunset_time_local_tz - sunrise_time_local_tz) / 2
     return solar_noon


  def zen_angle_at_date(self, date_time, location):
     altaz = AltAz(obstime=date_time, location=location)
     astropy_time = Time(date_time)
     zen_ang = get_sun(astropy_time).transform_to(altaz).zen
     degrees = zen_ang.degree
     return degrees

ESTIMATE_NUM_DAYS_HISTORY_MIN = 12
ESTIMATE_NUM_DAYS_HISTORY_MAX = 21
ESTIMATE_NUM_DAYS_USE = 7
ESTIMATE_SUN_ANGLE_MAX = 4.7

  sun_angles = SunAngles(config.LONGITUDE, config.LATITUDE)
  num_days_history = sun_angles.get_bounded_num_days_back_within_angle(ESTIMATE_SUN_ANGLE_MAX, ESTIMATE_NUM_DAYS_HISTORY_MIN, ESTIMATE_NUM_DAYS_HISTORY_MAX)

import datetime

class Utils:

  def __init__(self):
     return

  @staticmethod
  def date_from_offset(offset=0):
     return datetime.date.today() + datetime.timedelta(days=offset)

  @staticmethod
  def date_today():
     return datetime.date.today()

  @staticmethod
  def date_tomorrow():
     return Utils.date_from_offset(1)

  @staticmethod
  def date_yesterday():
     return Utils.date_from_offset(-1)

  @staticmethod
  def clamp_value(value, min_value, max_value):
     if value > max_value:
        return max_value
     elif value < min_value:
        return min_value
     else:
        return value

connesha

Happy St Patricks Day for tomorrow all! ☘️

Todays graph below. The last 3 days are when the "7 out of 12" change was made, and I've managed to stop tinkering with the algorithm since (which was no small feat!)

I think I'm at the point of not changing it for a while now, let it run, and see how it goes for a few weeks/months... Its been a lot fun getting here (can we call our predictors Alpha stage?), definitely learned a lot, and lots of good ideas with @bullit_dodger and @graememk especially.

Would be kinda interesting to see how the predictors that come with home assistant fare compared to these over time. Not sure if anyone watching here has past data, or is gathering future data on these. I hope we haven't wasted our time ;-) .. I don't think so.

And if anything from mine is of any use to anyone (theory/logic/code/mistakes), just give me a shout.

Time to move on to the next mini-project. Seriously, since got solar just a few weeks ago, my time has vanished, and the todo list is falling off the end of the page.

Now, to try to figure out home assistant... Installed it a couple of weeks back, spent a couple of evenings playing with it, but can't for the life of me figure out how it hangs together or how to do stuff with it... maybe it'll click eventually.

As first mini-integration, may try to get a home assistant screen/button/toggle/UI to apply an "away mode" to the script. Something simple, e.g. when HA sets AwayMode, the script reduces its daily_predicted_electricity_usage to a flat BASE_LOAD x 24. So then it won't charge at all at night time, if the forecast is even giving 5 or 6kwh (this last part should work as is once the predicted usage is reduced). Anyway, that's a different one...

graememk

That's my comparison, top 2 is forecast.solar, which usually is optimistic to say the least.

My met prediction is close, within a kwh or 2.

Will be interesting how temperature will come into play.

Eg in may and June my inverter clips.

But on the hot days in Aug it doesn't clip.

championc

Maybe time to start incorporating some automation type intelligence into your setup ?

Nelbert

I'm seeing familiar style graphs here. I've just purchased an energy monitor and incorporated it in to home assistant.... Background elec usage is about 5-600 watts at any given time but anytime the heat pump kicks in it jumps a huge amount.

Has anyone used the solar forecast integration in home assistant (familiar graphs so thinking there is some HA users)? If so how accurate are the forecasts Vs your production? I'm trying to gauge it's usefulness as a tool to work out potential impact.

championc

https://www.boards.ie/discussion/comment/118801324#Comment_118801324

The use of the forecast for me is really to gauge the level of night time charging I need to do, to ensure that I get to 23:00 that evening without needing to import any daytime units, while also not over-charging and ending up exporting to the grid.

From when the clocks go forward until the end of BST, I really don't see the need to be worrying about night charging, except when a really crap day is forecast.

But as for the most accurate source of data, I really haven't got to the point of really looking into things in an absolute way

mp3guy

https://www.boards.ie/discussion/comment/118801324#Comment_118801324

I find solcast much more accurate than the service built into HA's energy management system. Been using it for over a year now to do load shifting like championc mentioned and also decide on the method for heating hot water.

System

This discussion was created from comments split from: Daily pv production.

Jonathan

https://www.boards.ie/discussion/comment/118801324#Comment_118801324

Merged your comments into this thread where they are better suited.

connesha

As its past end of month, here's an update on how the met.ie forecasts are working out following the changes we talked through a few weeks back: using 7 of the last 12-day-window with similar radiation, and adjusting the window based on the sun angle (the sun angle hasn't kicked in yet; the "12d" in the chart shows the current window size)

And using these forecasts allowed the script to charge the battery automatically. I'm finding the readings for import as seen by the inverter differs somewhat from the actual. At the end of the day its what the meter/supplier sees that matters, so am using the monthly bill to judge success.

The script uses the EV period (2am to 5am) only for charging, i.e. its purpose is to avoid grid usage at all other times. So it will use this EV period to charge for (1) a poor day of generation, or (2) if the battery has gone low by 2am and needs small charge to get through to generation time in the morning.

There is about a 1kwh/day "leakage" in my system: while the inverter is trying to match house load on the fly, it misses it by approx 1kwh imported per day. So, this 1kwh per day spread across the non-EV period seems unavoidable.

March has been a great month.

This is with a 5kwh PureDrive battery. A bigger battery (20kwh coming soon I hope) would have reduced the EV period usage significantly for months like this past month, probably to approaching 3kwh. Although that wouldn't save much this month, it should have a decent effect on winter months: i.e. only EV period usage should rise in winter, and the usage in the more expensive times of the day should remain much the same. Lets see...

SD_DRACULA

I wonder if the EV tariff will still be worth it with the considerable hikes coming in one week time

+100 to the standing charge and quite a bit on the others too, even with full discount.

Peak rate is a 🤬 joke.

connesha

Yea, wondering the same...

My guess is that if you have tight enough automation to really put almost everything into the EV period, then it probably is. But I haven't done the calcs on it yet...

Are the other non-EV smart tariff standing charges rising by 100e also?

Jonathan

https://www.boards.ie/discussion/comment/118897717#Comment_118897717

Looks like it is working out well for you @connesha. Have you any code you can share with others on Github etc? Might help someone get up and running quicker.

SD_DRACULA

About 70 odd. My plan was to use this tariff alongside a 20kwh battery for winter mainly and load shift as much as I can but coming up to €500 now for basically no usage at all if you include the gov charge.

Anyone come across any other similar tariff with better rates?

connesha

https://www.boards.ie/discussion/comment/118898290#Comment_118898290

Sure, will do.

Will need to see what the easiest way to share it will be. There are quite a few things there in total, and most won't be of interest... will probably confuse more than help. I may just write it up here later this week, and post some of the key code, that should be easier for anyone getting started. In the mean-time, if anyone wants anything, just shout.

connesha

https://www.boards.ie/discussion/comment/118902568#Comment_118902568

Code for forecasting, battery charging and a bunch of other random stuff is posted over on the Automation thread now. #275

This is the output of all my ramblings on this thread several weeks ago. Hope its useful to someone...

graememk

Ive found a nodeRed Node that will give me the sun position now and I think ive got my head around the cosine calculations.

Just need to implement them now in javascript.

That being said, my rough and ready prediction for flat panels is working well. But I would like it to be more versatile.

connesha

https://www.boards.ie/discussion/comment/118907652#Comment_118907652

If its of any use to you, I dumped the angles to Excel a month back and still have it. Was just for eye-balling a the time. Let me know if you want. Covers that angle at Solar Noon for each day from 27/08/2021 to 29/09/2022. (the lib I used only went so far into the future, so I went back a bit as well as forward to cover over a full year)

graememk

https://www.boards.ie/discussion/comment/118907742#Comment_118907742

I found this, https://flows.nodered.org/node/node-red-contrib-sunpos I just push a datetime into it and it spits out the exact position for the sun at the set location & time

SD_DRACULA

Guys is there an easy way to hook this into home assistant?

Forecast.solar is overestimating by a lot on the dark days.

Jonathan

@connesha Out of interest, what was your prediction vs actual generation for yesterday?

Yesterday morning Forecast.Solar and Solcast were predicting 10.4kWh and 9.9kWh respectively. A really dull day where I am in Fingal and ended up with only 1.9kWh for the day.

SD_DRACULA

https://www.boards.ie/discussion/comment/118920278#Comment_118920278

Same here predicted 15kwh and real was 3.5kwh

connesha

Yesterday was a real outlier for me too. Predicted 13kwh, actual 5.5kwh. Biggest deviation I since started doing this.

My met.ie weather forecast was locked in at 1:45 AM, and at that point met.ie was showing radiation in the middle of the day at around 170 to 190 per hour. (The solar generation forecast was based around those levels)

I manually looked at the met.ie weather data at around 10 am yesterday, just out of interest, and they had changed to show levels of 80 to 90 radiation for the same period.

So, I think yesterday was just one of the days where the weather forecast changed significantly over the course of the morning. Don't think there's really anything can do with days like this, given that you got to make a charge-decision in the middle of the night... I guess the up-side is that they these days are outliers...

SD_DRACULA

From what I can see most days when Forecast.Solar predicts under 20kwh it's a bad day with under 5kwh real production.

irishchris

Forecast.Solar for me is always a waste of time and never very accurate for my location. Solcast on the other hand has been nearly always fairly accurate. Today as example had 46.4 predicted with 47.1 generated. But agree the lower predicted days seem to be the less accurate ones. 15kwh predicted tomorrow but forecast not looking great on met.ie

reklamos

@connesha I have pulled your code from repo and there is a lot of going on in there. Before I start disecting the code, is there a simple way for me to just get hourly numbers of tomorrows forcast?

connesha

https://www.boards.ie/discussion/comment/118930722#Comment_118930722

The solar forecast is based on history, so requires a DB table, etc.

It was written for specific needs I had, and for figuring stuff out, so it may not currently be easily consumable into someone else's codebase. Over time I'll refactor pieces into something more manageable. I hadn't originally planned or sharing it this early.

At its simplest, it takes your generation and radiation for the past number of days, and calculates the ratio between them. Then the solar forecast for the future days is ratio * radiation. There's a bunch off stuff for selecting which of the past days to use for calculating the ratio, based on choosing days similar to the day you're forecasting, and how far to go back in time based on the time of the year (sun angle)

For pulling the radiation from met.ie see WeatherForecast

The db table that stores the daily stats where solar forecasting is done from is solar_forecasts. DbForecastDaily reads and writes to that.

ForecastAction uses WeatherForecast and DbForecastDaily to make future daily solar forecasts.

In the code version I posted, Hourly Radiation (DbRadiationHourly) is only for graphs, so you don't need to take that. I'm doing a bit more with that in the current version I'm working on

So, if you want to get the minimal running, I'd start with:

• Create the solar_forecasts DB table
• Just focus on getting ForecastAction working (you can remove the other actions to start with)
• Remove references to DbRadiationHourly from ForecastAction
• The only service you really need to get running is DbForecastDaily (refresh() it before using). You can remove the other Services.
• If you don't want to use SolarConfig as I have it (its backed by a db), then create a simple config class with the few values DbForecastDaily needs from SolarConfig (you can see their values in the README
• Use harness_forecast.py as your entry point to start with. Forecasts are written to the solar_forecasts DB table, so check that after

connesha

Following the discussion about the poor forecast last Wednesday on all the forecasting tools, and how solar forecasts are only as good as the weather forecasts they are based on, the following graph taken this afternoon shows it pretty well...

Looking at today on the bottom graph: the solid red line shows the hourly radiation from met.ie that was locked in at 1:45AM (this solid line never changes after 1:45am)

The dotted red line is the "live" forecast. It continuously retrieves the data from met.ie every 10 mins throughout the day, and "locks" them on an hour by hour basis. I.e. the "live" value at 11:00am is what met.ie returned just a few minutes before 11:00am, and so on hour by hour

The difference in the red lines shows how the weather forecast changed between 1:45am and during the day up to 17:31

At 1:45am, the total day radiation was 5076, whereas the sum of the live ones is 3925.

The solar forecast for 5076 radiation was 37kwh, whereas the live radiation of 3925 would have been 28.7kwh.

The vertical dotted red line is the "current time" when the graph was taken. The actual generation for the day up to that point in time was 27.6kwh

While this "live" data doesn't change anything for deciding whether to charge a battery, as you need to make a decision on charging in the middle of the night, it is useful to see during the day, and perhaps could make some automation decisions based on it...

bullit_dodger

Just said i'd swing back on this topic and update people with how the algorithm is getting along. Pretty good is the verdict I think.

Not really sure what happened on the 5th of April (looks like I don't have logs for that day), which in turn meant that the next couple of days were out until the algorithm "self-adjusted" to include the dodgy value. I guess I should filter out zero values when computing the averages, but I didn't think of it when I was writing it.

SD_DRACULA

Is that an excel sheet or something you can easily share for others to put our details into?

Looks interesting 😀

JayBee66

I'm amazed at how well SolCast is at forecasting solar radiance for the day. All the more, considering that I live down a hole in North KK, have an East/West configuration so I configured with the perpendicular to that axis and there are trees resulting in sunset being a little earlier due to shadowing.

Forecast Solar is way off. Seems to think that I live on planet Mercury and is always over optimistic. I have disabled it. I'd prefer to delete it.

As machine learning was my expertise before retirement, I was looking forward to a project using Met Éireann data but with SolCast doing so well, I'll settle with that and better use my time elsewhere.

silver_sky

I'm still using the met eireann forecast that was shared in one of the other threads. It's generally working out OK. Here you can see the past several days. I just put the figures into excel for the graph, it normally just does it's thing in homeassistant.

championc

https://www.boards.ie/discussion/comment/120445557#Comment_120445557

But is this taken from data at midnight, the night before ?