TEMPEST Raw data vs. curated data... UDP messages vs REST and Websocket APIs

Developers
1

i) I am able to capture UDP messages. I assume that these values are the actual (raw) values reported by the sensors.

ii) If I use the REST APIs, to request observations, have any of the values been “massaged”? I understand that the values we get on the app have been “curated” as stated in my conversations with someone at .Weatherflow.

iii) What about the Websocket API - do I get raw data or is it massaged/curated before sending it out to the client that is listening? In the documentation it states:

“Start listening for all new observations and events for a device_id. Each new observation sent by the Device will be pushed to the client as soon as possible. All Device events will be immediatly pushed.”

Would be nice if I could set my own MQTT broker in the hub config so that the hub would send out the messages to where I want them to go.

2

Yes and yes, both serve data that has been through further QC analysis on the servers. This will affect rainfall accumulations (further suppression of false rain readings occur server-side) and lightning strike counts (which are augmented server side with data from the entire Tempest network). No other observations should be altered compared to the UDP data.

3

Thanks @peter… is this documented anywhere? I.E. know for sure what variables have gone through QC and which are definitely not touched? In some cases I need raw values off the sensors.

4

I don’t know of any official documentation that states this. It is just my understanding based on past threads on this forum.

Out of interest, why would you be interested in raw values rather than values that have undergone additional quality control? (I accept this is different for lightning strike counts as there is also additional data added, not just QC).

5

You can’t capture raw sensor data of what the sensors send to the Hub. I don’t know if anybody has tried to sniff the RF and reverse engineer that info, but there’s so much altering of those bits in the Hub that I’d doubt you could do it accurately and consistently anyway. One CL update to your Hub or one firmware update and any mapping you might do would be obsolete.

UDP messages are what the Hub sends to WeatherFlow after the Hub does whatever the firmware and tunings etc. do to convert raw sensor data from the Tempest/Air/Sky to the usual readings in the UDP API.

Rain is altered the next day if you have RainCheck enabled. Lightning is always totally crowd-sourced by WF so that will not match the UDP data ever. WF has never said what they include in such calculations, how much your info has been ‘massaged’, nor have they given any way for the user to disable them ‘helping’.

Yes it would be nice to be able to configure MQTT locally. It’s been asked before.

The usual way around the MQTT configurability question is to add a third-party utility on a pi or other computer on your LAN that listens for the UDP broadcasts and emits MQTT locally to the broker of your choice. My UDP listener is one utility that can do this. There are several others if memory serves.

6

Thanks @vinceskahan. I didn’t realize that the Hub did more stuff beyond formatting the values into JSON. I can imagine that the firmware might calibrate the values, but that is ok with me.

I’ve implement my own UPD capture and MQTT publisher on an ESP8266… sending it to my own endpoint (MQTT Broker)… cost me $15 for the chip and a few hours of programming…

@peter I’m conducting a few experiments with the TEMPEST and some other sensors, where the devices may be quite close to each other but at different heights, for example… really close to the ground, some a bit higher etc. and I wanted to be able to see the real values…

I won’t go into all the details, but knowing the actual values before the magic would really help me, if nothing else for sanity’s sake… I’ll stick to my solution for now.

When putting together experiments with sensors from various vendors, the ability to configure the hub to “just send unaltered values to MQTT Broker XYZ” would facilitate the combination. Thanks!

7

What you want to do is accept the values the hub emits in the UDP as truth as your station measures the various parameters.

With the exception of lightning they should match the WF server values if you turn rain check off.

1 Like
8

"What you want to do is accept the values the hub emits in the UDP as truth as your station measures the various parameters.

W it the exception of lightning they should match the WF server values if you turn rain check off."

That would work for me! I was actually going to compare my UDP captures with REST API values, but I just haven’t had the time and was hoping that WF would just put out some docs. Thanks for the help.

9

There are dozens of links and threads here and a long-published set of API docs, so it is available if you look around.

10

I have those and have used them successfully… I was meaning docs on what was “massaged” and where and how.

11

It’s in the forums in dozens of threads.

Lightning is done server-side on the WF servers. RainCheck is done the next day server-side on the WF servers. Everything else is done in the Hub as part of calculating what values to emit in the UDP and messages to the WF servers.

12

Thanks for your help @vinceskahan

13

I am not seeing that anymore. I have not compared with the REST values, but I now have a huge difference between the rain total reported by weewx and what the Tempest app reports during the day, before raincheck Like 2x. This just started with the replacement (yet another) unit I received a few weeks back.
Prior to the swap, the amounts reported by the UDP and app were similar until raincheck, and once the unit was dialed in, it was acceptable for my use. Now, the rain udp data seems mostly unusable.

14

Dunno, but I’d ignore the WF App and just concentrate on the server ‘without’ rain check vs. the UDP messages. Keep the comparison down to what’s on their server vs. what your station emitted.