We write often about the tidal wave of smart meter data that’s sweeping across the globe — how big it is, who’s benefitting from it, and all the good it can do for utilities and their customers. What we don’t often cover is the sheer technical challenge of transforming hundreds of billions of smart meter reads into valuable insights. It’s a topic we intend to explore in the months ahead, beginning today with a deep dive into a platform tool that Opower developed to process and analyze AMI data when it arrives at one of our data centers. Let’s start by looking at a highly populated region in the American Midwest. Smart meter installations here have grown at a furious pace — increasing five-fold in little more a year.
As more and more smart meter reads roll in from our utility partners in the area, we feed them into our data integrator — validating the information, then adding it to the more than 300 billion meter reads already stored in Opower’s customer data warehouse. (After that, we run the meter reads through our analytics engine to generate personalized, same-day insights for homes and businesses, like high bill alerts and peak day savings opportunities.)
But to make the process work, our incoming AMI data need to be spotless. That’s usually not a problem. Smart meters produce reliable, accurate reads nearly 100 percent of the time. But collectively, they generate billions of data points every year, which means small gaps and errors in the AMI data Opower receives are inevitable. It’s our QA engineers’ job to expose them. They’ve got dozens of tools for the task — many of which are completely automated. But on top of those scans and checks, our engineers also built a visualizer to get a quick, big-picture view of smart meter data as it comes in the door and look for anomalies. Here’s what a season of good, clean AMI data looks like for our region in the Midwest. Each row represents 24 hours, midnight to midnight; days progress as you move down the chart. Individual squares quantify one hour’s worth of energy use, aggregated and averaged across tens of thousands of homes. Redder squares correspond to higher residential consumption across the service territory.
What you see is essentially what you’d expect: households using less energy when they’re asleep, consuming a little more during the morning and early afternoon, and peaking when the workday wraps up and everybody comes home. Energy use is more evenly distributed on weekend days. Those patterns are punctuated by big blobs of red — peak events driven by summer heat waves. This is normal. Jumping ahead in the calendar, you can also see abnormal events, like Thanksgiving.
Look closely at November 28, 2013, and you’ll see that average residential energy use peaked much earlier in the day. That's tens of thousands of Thanksgiving dinners, turkeys and potatoes and pies, roasting quietly in tens of thousands of ovens. No issues here, either. What could a data anomaly look like?
Here, we see a white square on March 9, 2014, from 3:00 to 4:00 am. This is a red flag that the AMI data we received has a gap in it — and that we would need to work our regional utility partners to patch it up. Ultimately, that’s where Opower derives operational value from this AMI visualizer. It gives us another tool to quickly spot and fix data integration issues on the front end — and even alert utilities to smart meter outages they might have missed. And with our AMI data rock solid and locked in, our software platform can do what it does best: run high-speed analysis on the world's largest energy dataset, and deliver personalized, timely insights to more than 50 million utility customers across the globe. But still, it’s icing on the cake that these graphs are cool to look at.
In the wide-ranging discussion about the wants and needs of the utility customers, we often forget the small and medium...