December adventure 2024

2024-12-01

For a couple of years I've done Advent of Code but that's no longer my thing. This year I came across December Adventure and that resonates more with what I'd like to do: a little progress every day on a project rather than having to solve a couple puzzles every day.

Lately I've been working on my own tool to optimise the decision on if/when/how to charge our home battery overnight to take advantage of cheap electricity prices. This doesn't matter much during the summer when we have plenty of energy from the sun but it can make a (small but noticeable) difference in winter.

The stack I have consists of a few different components:

• a tool to interface with the inverter. Written in Rust and pretty much feature complete but could always use a bit more polish;
• a web app, written in Elixir, that fetches data from various API (Solcast, Octopus Energy), stores historical data, and periodically makes decision on the optimal policy. This is where I reckon I could do most of the work;
• a small Python script, for reporting;
• Nix configuration to manage deployments, dashboards and in general the infrastructure side of the project.

The surface of the project is not too small so finding new things to work on should be easy. Before starting I already know I'd like to improve our home load estimation and add an "holiday mode", which will be really useful at the end of the month.

This post is a living document. I'll update it through the month to log my progress. I fully expect to have some "rest days", especially at the end of the month, but that's okay.

Day 1

Just a minor tweak to the notification message I get (via Ntfy) whenever my program decides what to do for the next day. I like to receive these notifications so I can sanity check that my algorithm decided something sensible.

Day 2

I improved the load estimation algorithm. Before I was taking the average from the historical values for each of the 30-minute slots. That's okay-ish but what we end up comparing to isn't a day that actually existed and peaks are smoothed out.

To have a more realistic estimate, and to be able to be more conservative with my optimisation, I changed the load estimate to compute the total load for every day in the last month and pick one of them. This also makes it possible to tune how conservative we want the estimate to be. For now I'm going to use the day that sits at the 70th percentile. It feels like it could work.

Day 3

Deployed the changes I made in the previous days, found a bug and fixed it. While I was there I also added an assertion to a test I already had to check that the code now behaves correctly and that it won't regress in the future.

Day 4

Between seeing chatter online about Phoenix LiveView 1.0.0 and Dependabot notifying me about new version of a few packages today I only updated dependencies. Luckily it was easy and painless.

Day 5

Started working on holiday mode by adding a table to the database to hold first and last day of an holiday and the pages to interact with it. All of this has not required any coding. All I had to do was to invoke one of the Phoenix generators and let it do all the work.

Day 6

More work on supporting holiday mode. The function that estimates the load profile now knows about holidays and:

• it returns a constant, pre-configured value during holidays;
• it excludes holidays from the dataset used to compute the load profile to avoid biasing it with days with pretty much no load at all.

Day 7, 8, 9

Rest. I've been busy and had no time to work on the project. At least everything I had deployed works fine and produced sensible policies.

Day 10

Initial work to compare forecasts and the decisions made with that information. This analysis has two goals:

• to evaluate the quality of the forecasts;
• to check that my model of the system is close to reality.

Day 11

Today and tomorrow Octopus Agile prices are the highest I've ever seen. This led me into looking at future price predictor Agile predict has API and looks to be directionally correct so I'm thinking of integrating it in my project.

Today was also the first "saving session" of the season and it highlighted that my app should support discharging the battery to export to the grid. This is an easier win than integrating prediction so I started making some progress with it.

Day 12 and 13

Completed support for grid-first mode.

Day 14

I started by trying to record additional information in the database but I noticed that my app wasn't receiving updates via MQTT. So I ended up updating the UI to make it obvious if I'm looking at stale data. I should also figure out why I wasn't receiving updates in the first place, but that's a task for another day.

Day 15

I finished adding the attitional information to the database I wanted to add yesterday before getting sidetracked by the missing updates.

Day 16

A few queries on the database trying to understand how often I am missing updates. I haven't spotted any missing data, which is good. I guess I'll leave this on the side for now and keep investigating if I the problem re-occurs. I put some alerting in place to get notified of missed MQTT messages so I can have a look when the issue is happening.

Day 17

Easy dependencies update. I wasn't sure this should actually count as it was just executing a command, but I also started the application to test everything looked fine and created the PR, so it is okay according to my rules.

Day 18

Made a few options configurable via environment variables to make changing them easier. The biggest win is decoupling logic changes, which are in the application repo, and configuration changes, in my Nix config.

Day 19 and 20

Rest due to travelling. From now I'm not going to work much on my adventure.

Day 21

Added some validation of an holiday date range. This fix brought you by the realisation I had fat-fingered the return date from our holiday and as a consequence the feature wasn't working as intended.

Day 22, 23, 24, 25, 26

Rest due to the Christmas break

Day 27

Update Elixir version to 1.18. Uneventful. I was expecting the type system improvements to flag issues with my code but nothing showed up. mix format --migrate is pretty cool.

Day 28, 29, 30, 31

Rest. Again busy with "holiday stuff" with very close to zero computer time. Not the best way to conclude the December adventure but a great way to conclude the year. 10/10 I would do it again.