Solar PV Monitoring/Automation Thread

Jonathan

Starting a new thread as this was getting lost in the Hints, Tips & Troubleshooting thread.

Post details of your integration with Home Assistant etc.

Jonathan

Many inverters use the Solarman (IGEN Tech) WiFi Kit data logging stick for monitoring and data collection. Manufacturers using this approach for (at least some of) their devices include Solis (Ginlong), Omnik, Hosola, Goodwe, Solax, Samil and Sofar.

The data logging device polls the inverter using Modbus over RS-485 at fixed intervals and sends the data to the Solarman portal where it can be viewed through their app or website. This portal is branded for each Manufacturer, but examples include Solis Cloud etc.

Various methods of scraping the Solarman portal have been developed but this is not a good approach for a number of reasons;

• Fragile
• Not real-time (polling only at 6 min intervals by default)
• Read-only (Cannot make any configuration changes to the inverter)

It is possible to disconnect the data logging stick from the RS-485 port completely and interact directly with the inverter, but then you have no data being sent to the Solarman portal and the app etc becomes redundant.

After a bit of Googling I discovered that the Solarman data logging stick can be supplied with Modbus RTU requests in a specific format on a specific TCP port which will then be run against the inverter and the result returned. This has a number of advantages;

• More accessible (no hardware modifications required)
• Real-time polling
• Doesn't interfere with Solarman portal
• Read and write support (make configuration changes to your inverter remotely)

In order to simply this process, I wrote a Python module which handles the Solarman protocol side of things, and exposes an interface similar to that of a Modbus library.

The code is available on GitHub:

https://github.com/jmccrohan/pysolarmanv5

This can be used to build integrations with Home Assistant, or indeed standalone automations. Examples would be to set a particular battery SOC for a grid charge based on the weather forecast etc.

I'm only new to Home Assistant myself, so interested to hear what others have done.

ECO_Mental

Good timing as of literally 10 minutes ago I got the Solis cloud integration working on my home assistant...... I am totally new to HA as I only got RPI 4B off Santa a few days and have been messing around it since. Had to download HACS etc etc to get it working but eventually I did. What you are proposing does seem a bit more technical but I had to start somewhere, I have no fancy graphs on my dashboard or anything yet but the most important part is getting that data down and into HA.

I might look into your direct connection over the coming days.

This is what I have implemented fro github

https://github.com/hultenvp/solis-sensor

graememk

As my batteries and inverter are separate,

Im using an EmonTX (so i can have 4 cts) with wifi and a Rpi2 running emoncms (also hosts the MQTT broker)

https://shop.openenergymonitor.com/emontx-wifi-adapter-1/

For control, I only need to control my storage inverter, the ME3000sp, Communication is done via Sofar2MQTT https://github.com/cmcgerty/Sofar2mqtt

That is picked up in node Red where its handled and reposted to the emonCMS (mainly for SOC for my webapps)

A second flow does the automation/schedule for the ME3000. The main things I set is the Overnight charge %, and overnight (ie nightrate discharge %)

The nightrate discharge % is for the summer months where otherwise I have seen it rocking to nightrate at 70% battery, So i let it discharge overnight to a certain %.

Currently i have solar predictions from forecast.solar but no automation yet,

All controlled from a node red ui.

I also have an EmonEVSE to charge the car... on the rare days its actually home during the day. - that is fed a value over mqtt that can track the solar like a zappi, but I dont have to wait until the battery is full.

There is another node red flow that tweaks the excess value to limit the house battery charge depending on the SOC of the battery. eg if the battery is at 90 % it will up the amount of power available to the emonevse, thus slowing the charge to the batteries.

I'm planning on replacing my immersion switch with a wifi controlled one, putting a CT on the immersion and just having a dumb "diverter" when the panels are belting out the power in the summer (and batteries are either at a high SOC or charging at greater than 2kw)

I have dabbled with HA but its just HA core in a Truenas jail. (Might just load up a VM for it for the whole manged one).

reklamos

Here is my monitoring/automation

Inverter:

v1 wrote the code to scrape Solis Cloud. This was providing data to Prometheus/Grafana and also pushing to Mosquitto MQTT. In HASS configured sensors just to pull metrics that I needed. As the data was aggregated and delayed 15-30mins I moved onto v2

v2 wrote code to pull data directly from rs485 port on Inverter but also had relay that was switching between my modbus device and wifi stick, as there can be only one master on the bus at a given time. This also enabled me to modify settings on Inverter remotely through modbus. I was using nodeRed for that. Again the data was provided to Prometheus/Grafana and also pushing to Mosquitto MQTT. This worked well for more than half a year but then @Jonathan posted his code so moved to v3

v3 used code from @Jonathan and merged it with my v3. That meant I can pull data directly from wifi stick and there is also possibility to remotely control the Inverter.

Battery:

For batteries I've built serial monitor on top of esp8266 and pull data from master Pylon battery. This gives me detailed information each battery/cell etc. All of this is sent to Prometheus/Grafana. I have no real use for this data in HASS

All above give me a lot of metrics in Prometheus use for longterm storage and dashboard in Grafana. Data is beautiful :)

In August HASS has added Energy monitor. Which is ok but leaves much more to be desired. The data that I collect allows me to use it but I am only testing it and will see if I will keep it.

What I have found is a custom card that allows to visualize power and this is the one that I started using but still needs tweaking. Here is the screenshot of my wall mounted tabled and on the right it has live data what Solar is doing.

At the moment I do not do any automation on Solar side as I have 24hour electricity plan but that is about to change next month.

Apart from solar my whole house is being monitored and automated. Here is a list of things that I remember :)

Lights in some areas are powered on/off based on time of the date and triggered by motion or door sensors

Lights have different modes and light temperature settings in room.

FishTank lights and brightness.

Each room has temperature&humidity sensor and automatic TRV. Gas boiler is turned on based on set room temperature, which differs depending on house occupancy, time of the day etc. Also boiler is configured to keep certain water temperature again based on house occupancy, time of the day etc.

Bathroom/shower fans are controlled based on humidity levels

Voc&Co2 monitors in main rooms.

There are automatic switches with power monitors to ensure that no devices are left in standby for long time.

TV/HiFi systems are also controlled via HASS. Your phone is your remote.

Brightness level of 2 lights and also HiFi volume level can be controlled with Aqara cube

Have 2 LAN cameras integrated to HASS and also started testing frigate for object detection with Google coral TPU

Xmas lights controlled by HASS :)

In summer I monitor temperature&humidity and activates airpumps for plants depending on solar level and battery voltage on my shed.

For controls there are phones(everyone has one) and also old tablets mounted on the walls.

I'm sure there is something else but cannot remember now. The whole brain of this is running in my home lab in containers. As mentioned already I use HASS, nodered(automations), prometheus, grafana, mosquito and there are some other containers that provide metrics or do some specific tasks.

What I do not have is Alexa or Google home and have no intention of getting this. I prefer to have everything self hosted and keep control :)

ECO_Mental

I have a lot to learn.....😥😥here is me happy and proud I got to pull my data off the Solis cloud

Thats first dashboard look great well done

graememk

https://www.boards.ie/discussion/comment/118382281#Comment_118382281

The first step is the hardest.. these weren't built in a day.

Just an indication of the possibilities!

reklamos

It is a rabbit hole once you get hooked on it. As @graememk said there multiple possibilities.

For anyone starting with HASS and raspberry pi, lease do yourself a favor and do not use sd card. HASS is pretty big and once you start adding sensor it starts writing/reading a lot. SD card are not design for such application and sooner or later will fail. Use SSD drive, they are cheap now.

I have published early draft version of Solis monitoring code that is based on @Jonathan code. I have been running it for a week or so and it is stable enough but I am sure there will be issues. :) It can be used to present metrics to Prometheus and publish to MQTT or just publish to MQTT which I think most will use for HASS. Let me know if you have trouble running it and I will try to help.

graememk

Other home automation I have is tado for the heating+ a rpi controlling the pump on my solid fuel stove.

On the lookout for an outside temperature monitor that I can pull into mqtt.

Started poking the API for my car too.

THE ALM

Happy festive season to all.

Running a combination of Loxone, Hassio and Loxberry.

All lighting (hue), Sonos, heating runs on Loxone with various humidity and temp sensors in all the rooms for heating control which is done through various shelly relays (standard oil boiler with pumps for heating and hot water) and 1-wire sensors.

Most of the lighting in controlled by motion sensors or sunset timers. Retractive switches or wireless loxone air are used to control most things in the house from light, heating, music, sockets etc.

Like above I pull the data from solis into hassio using the custom integration in hacs and then use mqtt to push this data to loxone which I can then use for various automations if it sees too much is being exported back to the grid, although with a zappi and eddi most excess is used anyway.

Hassio and loxberry and mainly used as a bridge for anything that can't be directly controlled though loxone.

If I can get the data directly from the inverter, through modbus/RS485, to loxone this would be my next task so reklamos and Jonathan's work might be something to look at although above my knowledge for now.

tnegun

https://www.boards.ie/discussion/comment/118383550#Comment_118383550

Hi @reklamos thanks for your work on this integration. I'm trying to get your code running and have installed the dependencies. I can hack python code together but wouldn't be an expert by any means! Anyway I've got your script running and it's executing the while loop but returning an error every time I've extracted what I think are the important bits

Message: 'Cannot read registers 33022 length 19'

Arguments: ('IllegalDataAddressError()',)

umodbus.exceptions.IllegalDataAddressError: The data address received in the request is not an allowable address for the server

I've tried excluding the 33022 register from register.py but the error just moves to the next one. A tcpdump shows it is successfully connecting to the Inverter/logger on port 8899 but it's not getting the answer it expects?

solis.txt

Jonathan

https://www.boards.ie/discussion/comment/118395883#Comment_118395883

Have you got your data logger serial number set correctly? The pysolarmanv5 library could do with more error checking. I'll add some as time allows.

Try using the example program to confirm that all is working. The example registers are actually Solis hybrid registers, so it should work out of the box with just changing your IP and S/N.

tnegun

I think so I changed the serial no to test that and it returned a CRC error, my inverter is a non hybrid Solis 3.6kW 5G so might not return all those registers? The example failed with a similar error. The data address received in the request is not an allowable address for the server.

example error.txt

reklamos

https://www.boards.ie/discussion/comment/118395989#Comment_118395989

6 registers starting from 33022 should read fine as this where inverter date/time is stored. As you get response and it is not CRC, it looks like your serial is valid.

Try to read a single register like:

print(modbus.read_input_registers(register_addr=33022, quantity=1))

or

print(modbus.read_input_registers(register_addr=33049, quantity=1))

Also use other numbers from 30000-39999, use single first. See if you can get something out.

tnegun

Will do and thanks for the help. It will have to be tomorrow now as the inverter has shutdown!

harderthanf

The amount of dashboards I have setup, over the years, to monitor my solar setup is ridiculous. At first I was using modbus direct to my SMA SunnyBoy and populating a local sql database. I got the Myenergi setup this summer and I have added that integration to my home assistant. However, I still grab the data myself, using the myenergi API and populate a local db.

I used to have all sorts on the dashboard but as I have wallpenal tables around the house the rest of the non-nerds found the screen too busy. So I have settled on this. The € value of the electricity and EV costs are being pulled via a rest sensor I have setup on another box. The db has the unit cost, discount, pso, standing charge etc so it's all calculated (and editable) to give the correct total. It's easy to see how much things are and how much the EV is costing to juice.

No batteries - yet!

I have loads of lights and smart sockets. All the wallpanels are on smart sockets, and the tablets use mqtt to send update to home assistant. One automation I have is when the battery level of the wallpanels get below 20% I switch the plugin on, above 85% and it switches off.

Loads of contact sensors and motions sensors. e.g when the attic door is open the lights in attic comes on - off when closed.

Motion sensors in hall and landing - lights come on depending on sun elevation then switch off when no motion after 2 mins etc.

Some location awareness - if any comes home and it's dark the outside lights will come on (if off)

Integration with Nest for home assistant control.

Some nice Android TV integrations - e.g when we're watching a show or movie my downloader application pauses (as the disk i/o isn't amazing). When watching movie and play is pressed and it's night time, the sitting light dim - they come back on when pause or stop is selected.

Heating on in morning (if workday) and temp below certain value

5 Reolink cameras all to a dashboard in HA

All the TV and Amps are controlled via HA too.

Loads of other stuff - bloody hell I've spent a stupid amount of time on HA!

Below is my energy / weather screen.

Jonathan

https://www.boards.ie/discussion/comment/118396897#Comment_118396897

Back at PC now so had a look at those verbose logs in more detail. You have your profile on private so I can't PM, but can you confirm if your data logger S/N begins with 406 and ends (last three digits) in a combination of two of those three digits?

Can you also confirm what firmware version your data logger is running? Mine is running MW_08_512_0501_1.82.

It is possible that the Modbus pass-through functionality is not implemented in older logger firmware versions. If you are on an older firmware version, you could try to contact Solis support to look for an upgrade. Alternatively, you could try setting a Remote Server B (point it at a local machine as UDP and listen on that port). The newer firmware versions removed that Remote Server B (still there, but no longer works).

https://www.boards.ie/discussion/comment/118396702#Comment_118396702

Actually, that CRC check is within the uModbus library. Pysolarmanv5 also does a CRC check on the V5 frame reponse and will throw a V5FrameError if it fails, but as of right now, I need to do more reverse engineering to figure out how I can confirm a valid modbus reply before offloading to uModbus to decode. Can you PM me some verbose logs reading the two sets of timestamp registers (33022 and 43000) if you can please. Hard to spot a pattern from just my own data. Thanks.

graememk

I also have a non hybrid 4g inverter. I'll have a poke at it.

(I'm wonder is it because it's a standard inverter not a hybrid)

Will be tomorrow or Sunday as I have to do it during the day as the inverter shuts off at night

reklamos

https://www.boards.ie/discussion/comment/118398236#Comment_118398236

As the inverter is 5G, the stick should be the newer one. Mine is actually older and starts with 401. I am attaching the output of 33022 and 43000. The error he is getting 'The data address received in the request is not an allowable address for the server' is coming fro umodbus.

registers.txt

tnegun

https://www.boards.ie/discussion/comment/118398236#Comment_118398236

Happy New Year all :) The logger serial no is in that format and the logger firmware is MW3_15_0501_1.18. I'll generate and send you those logs now too and I've changed my profile so I think should be able to receive PMs now too.

Jonathan

https://www.boards.ie/discussion/comment/118399377#Comment_118399377

Thanks for this. It appears that the Modbus frame in the V5 reply is always preceded by 0x61. I'll add that into the validation later.

tnegun

I've opened a ticket and emailed Solis support to request firmware updates for the logger and inverter. I tried the remote server option, adding my laptop then running a tcpdump shows nothing is received from the logger over 10 minutes. I tried the different registers too all with the same result. My logger could be an old one as it took 3 attempts to get it going here and the solution was to replace the first logger so I've no idea where it came from and could have been in the bottom of a drawer somewhere!

registers.txt

Jonathan

https://www.boards.ie/discussion/comment/118399591#Comment_118399591

Digging a little deeper, it seems that your data logger is working fine, and is passing the Modbus through correctly. It seems the Modbus error code is actually coming from the Inverter itself. See decoded Modbus below.

I wonder is there a different set of registers used for the non-hybrid Solis inverters?

01 Slave address 0x01 (1)

84 Error code0x80 + 0x04 (4) - Read Input Registers

02 Exception code[02] ILLEGAL DATA ADDRESS

C2 C1 CRC0xC2C1 (49857)

reklamos

https://www.boards.ie/discussion/comment/118399591#Comment_118399591

It looks to me a protocol version problem. Here are wifi the firmware versions:

MW3_15_0501_1.18 @tnegun

MW_08_512_0501_1.82 @Jonathan

MW_08_512_0501_1.89 @reklamos

1.18 looks like V4 or maybe even V3

Jonathan

@tnegun Found a different set of Soils registers. Try reading register 1078. Does it contain kWh generated today? Similarly 1075 should contain AC power in W. Both of those are Input Registers (Modbus Function Code 04).

reklamos

Maybe try to loop through all registers? To cut down the noise set verbose=0

Here is for all input registers,

print('Started')
for x in range(30000, 39999):
    try:
        print('!!!!!!!!!!!FOUND!!!!!!!!!!!', x, modbus.read_input_registers(register_addr=x, quantity=1))
    except:
        continue
print('Finished')

Here is for holding registers

print('Started')
for x in range(40000, 49999):
    try:
        print('!!!!!!!!!!!FOUND!!!!!!!!!!!', x, modbus.read_holding_registers(register_addr=x, quantity=1))
    except:
        continue
print('Finished')

It will only print output when it starts/finishes or when it finds a register. Otherwise it will be empty screen. Start it and leave it running, it will take a while but maybe it will find something. You can also break into smaller batches as at the moment the range is set to 10k

Jonathan

https://www.boards.ie/discussion/comment/118400513#Comment_118400513

I think the previous generation (or maybe just non-hybrid) inverters use a completely different range for their registers. See link: https://www.aggsoft.com/serial-data-logger/tutorials/modbus-data-logging/ginlong-solis-pv.htm

tnegun

Same error back for registers 1075 and 1078, no registers found iterating through them all I've attached the script I ran in case I did something wrong but it completes successfully very quickly and just outputs Started and Finished one line after another.

discover_reg.txt

tnegun

https://www.boards.ie/discussion/comment/118400598#Comment_118400598

I just tried from register 1000-4000 but nothing came back :(

Jonathan

https://www.boards.ie/discussion/comment/118400607#Comment_118400607

Your for loop is outside main(), so it never gets run. 🙂

reklamos

https://www.boards.ie/discussion/comment/118400629#Comment_118400629

Also trying to read holding registers with function 3

@tnegun replace all client_example.py with below, add your IP and Serial. Run it and leave for good while. It will do input and holding registers.

""" A basic client demonstrating how to use pysolarmanv5."""
from pysolarmanv5.pysolarmanv5 import PySolarmanV5


def main():
    """Create new PySolarman instance, using IP address and S/N of data logger

    Only IP address and S/N of data logger are mandatory parameters. If port,
    mb_slave_id, and verbose are omitted, they will default to 8899, 1 and 0
    respectively.
    """
    modbus = PySolarmanV5(
        "192.168.1.55", 123456789, port=8899, mb_slave_id=1, verbose=0
    )

    """Query six input registers, results as a list"""
    print('Started Input Registers')
    for x in range(30000, 39999):
        try:
            print('!!!!!!!!!!!FOUND!!!!!!!!!!!', x, modbus.read_input_registers(register_addr=x, quantity=1))
        except:
            continue
    print('Finished Input')


    print('Started Holding Registers')
    for x in range(40000, 49999):
        try:
            print('!!!!!!!!!!!FOUND!!!!!!!!!!!', x, modbus.read_holding_registers(register_addr=x, quantity=1))
        except:
            continue
    print('Finished Holding Registers')

if __name__ == "__main__":
    main()