[ItsJohannaWren]

I’m using the PyEmVue module someone wrote to pull the data from Emporia’s system using the same API’s as the mobile applications. That script pulls data for the last 60 seconds every 15 seconds. Why the overlap? I’ve noticed zero watt entries or missing recent entries that are later filled in (makes me think they’re using an eventually-consistent database to store data or their pipeline is a little too long). Once my application has the data it puts it into ElasticSearch… ES is a form of database that is usually described as a document store, where you’re not required to have a set schema and documents don’t need to have the same fields, ie. the Tesla Wall Connector has hertz, volts, watthours, uptime, 3 temperature values, and fields for being connected and actually charging… but no other device does.

I’m thinking about getting another Vue Gen 2 to experiment with. Specifically, replacing the firmware to spit out data locally. I’m familiar with the Espressif microcontrollers that are used in the Vue products.

This chart covers two hours of samples for a Vue Connect and a Vue Gen 2. The drop-off near the end is me unplugging the Vue Connect to have a looksy inside. One document per sensor per second. Could I combine all of the sensors for a device into a single document? Yes. But that means I’d need more indexes to efficiently find data. More indexes generally mean more storage.

• This reply was modified 3 years, 11 months ago by ItsJohannaWren. Reason: Realized a long processing pipeline could also explain the missing recent entries

[ItsJohannaWren]

So this is the simple version of what I’ve been working on. The backend will support multiple accounts and multiple devices per account. Grafana allows you to build in alerts.

ElasticSearch -> Grafana using Python” />

[ItsJohannaWren]

I’ve been working on a local data viewing system for about a week. Right now it pulls data into ElasticSearch for charting with Grafana. It can handle multiple accounts if people are interested in interacting with data in a more customizable way. The downside is that it does need your Emporia account email address and password– and I haven’t even touch setting up a secure system for that (though I’m think something web-based that instantly converts the info to tokens by talking with Amazon Cognito so raw credentials aren’t ever stored). Anyways, let me know if you’re interested.

@Emporia – I’m being nice to the servers… 15s gather interval, caching what I can for a while (devices and channels), and only grabbing the last minute of data with 1s resolution. If you’d like me to adjust this, I can. As I add more accounts to test with I’ll be spreading them over the 15s interval to avoid a thundering herd.

• This reply was modified 3 years, 11 months ago by ItsJohannaWren. Reason: Adding notifications

[Author]

Posts