Download Cached Data

I am wanting to use Tempest WeatherFlow stations in an area where wifi is not an option. My plan was to visit them every few days and use my mobile hot spot to connect them to the internet. I’ve managed to set up the station to link to my hot spot data and I’ve linked the station to the IFTTT that exports data in 1 minute increments.

The issue I’m having is that the IFTTT only uploads data when station is connected to network. I need a way to download data that is cashed on the station or in WU after data has refreshed. I’ve played around with work arounds, but am not having any luck. Any suggestions?

Do you have the ability to have a small PC that can store the UDP data? Such as a raspberry pi.

One way to go would be to use WeatherFlow2MQTT, import to Home Assistant, then export to InfluxDB for all historical data. There are more streamlined ways, but since I have all of the running at the moment, that is the route I would go and then grab the data when I needed it.

I don’t have anything, but I could invest in something if that would make this all easier.

In case it helps, here are more details:

I’m going to install seven stations at seven houses in rural Mexico. I’m giving each household a cell phone and will be adding buying cell data (in Mexico people buy MB of data as needed rather than having a monthly plan) at least once a week for them to use the phone’s hotspot to upload data to weather underground. Best case scenario would be if I could log on to weather underground after they have synced data and download the data myself.

I did my best to understand your second response. Is raspberry pi needed to use WeatherFlow2MQTT? Looking at forums I pieced together that I needed to create a PWS. I did this, though the station is offline on the PWS but online on the weatherflow app, and thus I’m waiting to see if time will sort this out.

Thanks,

Sara

Hi @sara.dykman
There is not a simple solution that does exactly what you want, unless you can just leave the phones hot spots active full time would work.
There are several ways that it can be done though.
Basically in my experience if the hub does not upload the data to Weatherflow because there was no wifi for a while it only uploads an hour or two of the most recent data with some sensors loading more than others. Somewhere there is a specific amount mentioned but I just recall about an hour or twos worth. That means you will need the hotspot available every hour for several minutes (more than 10 minutes is my guess) to have enough time to upload its history for the past hour.
To guess how much data will be used through your mobile hotspot see this topic:

So if you simply left a mobile hotspot running at each location would be the most reliable method but perhaps too expensive depending on your purpose.
Now if you do that I am suggesting the data may not be at Weather Underground because that only works if it is live and it does not work as well as getting the historical data directly from Weatherflow servers.
There are several topics describing why Weather Underground is sometimes unreliable if loaded directly from the Tempest and will not be the one minute values.
So then all the history is available from Weatherflow servers online using several different options.
If you want all one minute data history then it can be accessed using the API which is described here:
https://weatherflow.github.io/Tempest/
For one minute data you would need to download one days worth at a time.
Several people have created different methods of using the API to display their data. Some are written in python using a raspberry pi computer or other languages or other types of computers.
Or you can use a link in your browser to pull the data straight into the browser window. The developers API area has a description where you can test doing that.
Or you can view the history simply using a web page and look at graphs or history data similar to if you were using the App.
If you actually decide to progress with your plans then I can describe more detail about how to access the history using a raspberry pi computer.
I keep having more ideas that may complicate your understanding and I just give you the links to consider but they are not actual solutions without some more work.

cheers Ian

Suggestions from Ian and Glenn look good to me.

If you can enable the hotspots often enough that the hub can play catch up and get the data to Weatherflow’s servers, do that. (Can the Hub really just cache data for an hour or two?) BTW, I highly recommend USB batteries with passthrough charging like the official Hub Power Bank to ensure the Hub will keep receiving data as long as the Tempest is sending it.

If you cannot do that, then I’m afraid you’d have to do something more labor intensive and fragile like configure a Raspberry Pi to listen for the Hub’s realtime UDP broadcasts, store them, and somehow send that data somewhere when the hotspot is available. I’d hope the Pi could act as both a WiFi hotspot and see the UDP broadcasts. Part of the fragility here is that during any Pi downtime you’d lose data. It seems likely that you could use a “USB cellular modem” with the Pi to have it connect to the Internet occasionally and upload its data.

How much data? The daily “device” data from the Weatherflow APIs Ian mentioned, including minute-by-minute observations, is usually about 125 kilobytes per day, and compresses via gzip to about 25 kilobytes. Full UDP captures might take more space since they’re more frequent than once a minute when the Tempest battery is in good shape.

In addition to being a lot less work, providing Internet to the Hub periodically and having it send to Weatherflow also would allow you to take advantage of Nearcast/Rain Check for what should be more accurate precipitation data. For daily “station” info I usually find the Nearcast final daily precipitation values are available by about 7am the next day.

Let me bring some extra details (some can be found on this page)

  • The Hub can store about a week of data from one Tempest
  • The amount of data uploaded by the Hub from a single Tempest device is about 1 GB per month or about 30-35 MB per day (understand with the 3 second wind interval)
  • Next to the hub the Tempest itself can cache also a few hours of data if it can’t reach the hub but it is rather limited (if memory isn’t to bad, about 2 hours)

in both caching systems once the memory is full it deletes oldest and continues to cache recent data …

Hope this helps better dimension the solution.

2 Likes

I was Incorrect. Sorry I forgot that in my experience that my hub also lost power in that situation. You should trust what Eric said.
cheers Ian :slight_smile:

Dumb question - Does the Hub cache the three second rapid_wind data for later transmission, or only the slower obs_* information? That would surely save a lot of bandwidth in a location without a 24/7 Internet connection…

1 Like

Why then, does my sister’s station (45970) only upload 6-6.5 hours of data after her station gets Wi-Fi access again? I’ve seen something similar on my stations but I have more than just a Tempest attached. Her station looses Wi-Fi at 11:00pm and gets it back at about 6:00am and the whole gap does not get back filled.

2 Likes

Isn’t this a hub from beta testing ? Those had smaller buffers. It is only when production models came out the full buffer memory was used.

I should test this again but I do think yes.

No, it is a production hub.

What about a pi pulling in udp packets → weewx → script to copy and restart the weedb each time. The back end could zip them together as needed.

If the pi could be replaced with a better machine with a built in hotspot you could simplify and the weewx db could store lots more than a week. Even with a pi.

I’m used to weewx so when u have a hammer…

Oh ya. You can get weewx to sync direct to wu also.