SP Energy Networks – Energy Platform

Changes in the way we consume electricity (demand patterns and new technologies such as electric vehicles or heat pumps), as well as how we generate it (an increase of renewable, smaller and larger numbers), make the electricity system a lot more volatile than it used to be. 
 
For each element of the network, understanding the flow of electricity, what it is made of and what is driving the demand and generation has become critical to ensure security of supply at optimal costs. Several ML models have been developed to capture the dependence of the networks on weather variables for both demand and generation 
 
The team worked directly with Control Room engineers to develop the solution, which is now live on a desktop as well as on a control room wall. We worked with Data engineers to identify, structure, ingest and clean historical load data from their SCADA systems. 
 
The solution provides operational forecasts for the control room engineers up to 5 days ahead and now supports other business processes (mid-term forecast, emergency plans simulation...). Depending on the level of the point of the forecast, the quality of the reconstructed load forecast can vary. MAPE, MAPE*, RMSE, and MAE are indicators used throughout to determine the quality of the models and of the forecasts. 

Our workflow centres around tasks in our Heka infrastructure. The main task is our ETL pipeline for the weather forecast data, which is scheduled to run every 30 minutes and will first check to see if a new forecast is available. If there is (usually every six hours) the ETL pipeline will trigger. Once this pipeline has been completed successfully new load and generation forecast tasks will be initiated. Each weather scraping task has a maximum of 250 mCPU time and 200 Mb of memory. The forecast tasks are much larger, requiring 1 CPU and 2 GB of memory. The forecasts typically take one hour to run through all 911 assets. Weather data is also retrieved directly from our Weather & Climate solution.