Lightning reports (tempest)

I got a user report from somebody using my UDP listener about lightning strikes not matching the info shown in the WF mobile app. Looking at my influxdb here I see the same issue with my field-test Tempest.

I see the mobile app reporting strikes on the 17th and 21st and multiple other days in the last month, but my influxdb from the UDP broadcasts hasn’t seen values for strikes since april-23rd or so.

A few questions come to mind:

  • does the mobile app lightning info come from evt_strike or obs_st in the UDP broadcasts ?
  • does WF adjust lightning readings based on any other data ?
  • was there anything that changed in the most recent field test firmware update related to lightning ?

Here’s some influxdb data with strike info from evt_strike and then obs_st in case that helps. They look like they basically line up ok to me. But nothing for a month recorded. Odd.

Again, this is a field test Tempest, but the sensor status reports clean. Ideas ?

# influx --precision rfc3339
Connected to http://localhost:8086 version 1.7.3
InfluxDB shell version: 1.7.3
Enter an InfluxQL query
> use testdb
Using database testdb
> select * from "sensors/ST-00006021/wf/evt/strike" order by desc limit 10;
name: sensors/ST-00006021/wf/evt/strike
time                 distance energy   timestamp
----                 -------- ------   ---------
2020-04-23T05:35:53Z 20       16777216 1587620153
2020-04-23T05:35:26Z 20       16791778 1587620126
2020-04-23T00:07:55Z 14       16800892 1587600475
2020-04-22T22:50:59Z 12       16850124 1587595859
2020-04-22T21:15:03Z 12       16846680 1587590103
2020-04-22T17:25:01Z 5        17022663 1587576301
2020-04-22T11:08:05Z 6        16935260 1587553685
2020-04-22T08:19:12Z 8        16885132 1587543552
2020-04-22T05:49:19Z 14       16810806 1587534559
2020-04-22T05:09:20Z 17       16798103 1587532160
>
>
> select timestamp,lightning_strike_count from "sensors/ST-00006021/wf/obs_st" where lightning_strike_count > 0  order by desc limit 10;
name: sensors/ST-00006021/wf/obs_st
time                 timestamp  lightning_strike_count
----                 ---------  ----------------------
2020-04-23T05:36:52Z 1587620212 1
2020-04-23T05:35:52Z 1587620152 1
2020-04-23T00:08:55Z 1587600535 1
2020-04-22T22:51:58Z 1587595918 1
2020-04-22T21:16:02Z 1587590162 1
2020-04-22T17:26:01Z 1587576361 1
2020-04-22T11:09:04Z 1587553744 1
2020-04-22T08:20:11Z 1587543611 1
2020-04-22T05:50:18Z 1587534618 1
2020-04-22T05:10:20Z 1587532220 1

1 Like

update - we’re getting a very active thunderstorm currently (very rare) with dozens of rumbles overhead and an occasional flash. The WF app is reporting the strikes. I see ‘nothing’ in the UDP data related to lightning activity.

Nothing in the obs_st messages, no evt_strike events. Everything zeroes.

Where does WF get its lightning data from ?

1 Like

tagging @WFsupport @dsj @WFstaff to see if anybody has any ideas…

1 Like

I’m seeing the same problem. Active thunderstorms in the area. My Air is reporting lightning via UDP. My collocated Tempest is not reporting lightning in the UDP messages.

2 Likes

Good questions, @vinceskahan

Not all lightning data provided through the remote API (and seen in the apps) is available locally (via UDP or BLE). The reason for that has to do with the way the Tempest back-end system processes and qualifies lightning strike events, similar to rain accumulation and Rain Check data.

For more info on Tempest Lightning detection see here: https://help.weatherflow.com/hc/en-us/articles/360046877513

1 Like

Boy I wish there was a ‘NOT’ like button in the forums. As more and more things become calculated on the WF servers, that’s making the local UDP interface significantly less valuable, which is one of the reasons a lot of us purchased the gear to begin with.

Appreciate the reply, but really not happy in the least with the answer.

5 Likes

We have discussed feeding the “extra” data produced on our servers (including the forecast info) back to the hardware (and out over UDP & BLE), but we don’t have plans to do that any time soon.

Or an API to the calculated/processed data so we can pull it from the WF servers? Not ideal, but at least a workaround.

For sure. That already exists (see API docs), and is the preferred method for third-party applications to access Smart Weather / Tempest data. The UDP broadcast (over the local network) mentioned by @vinceskahan is a convenient method when your internet connection is down, but the API should be the primary source for all data.

Thanks, wasn’t sure if the lightning data was included!

1 Like

Really ? I don’t recall that. My recollection is that UDP was supposed to be functionally equivalent (assuming you did the derived calculations) with the exception of RainCheck.

@GaryFunk what’s your recollection ?

3 Likes

I’m not pleased with this information but I understand that UDP is not part of the consumer model.

My opinion is, Nothing should leave my network that is not available on UDP.

I guess it’s time to intercept all data sent by Hubs and decide what will be forwarded. I can only say that I am greatly disappointed by this information.

5 Likes

That’s a bummer. The promise of local data availability was the reason I backed the Kickstarter. I assumed we’d get usable data over UDP from all of the sensors.

3 Likes

But it kind of still is available. all the sensor data is available, only the result of the processing on the servers afterwards isn’t through udp. That result is available from the servers through the api. That at least is how I understand the situation and it seems logical to me to do so.

1 Like

Having read the link that @dsj shared, I am going to stick my neck out here and say that I don’t think the way that UDP works for lightning has changed that much between Sky and Tempest. The major difference, however, is that lightning is now treated like a derived variable in the app. We already know that derived variables (such as sea level pressure) are not available in the UDP data as they have to be derived on the WF servers.

Based on my reading of the link (https://help.weatherflow.com/hc/en-us/articles/360046877513), there are three possible scenarios:

  1. normal Strike Confirmation: Your Sky/Tempest records a lightning strike. It is shown in the UDP data and is also sent to the WF servers. The WF servers “derive” that this is a normal lightning strike and it is reported to the app. In this scenario the “local” device data reported by UDP matches the “derived” data reported in the app

  2. False Strike Suppression: You Sky/Tempest records a lightning strike. It is shown in the UDP data and is also sent to the WF servers. The WF servers “derive” that this is a false lightning strike and therefore it is not reported to the app. In this scenario the “local” data reported by UDP does not match the “derived” data reported in the app. However, the UDP data is still showing what your local device reported and is still locally available. You could decide to implement your own false lightning suppression algorithm at a local level if you choose.

  3. Missed Strike Correction: Your Sky/Tempest does not report a lightning strike, and therefore nothing is shown in the UDP data. However, the WF servers “derive” that a lighting strike should have been detected so a lightning strike is sent to the app. Again, although the app and the UDP data are diverging, the UDP is still showing what your local hardware reported: i.e. nothing.

Being able to use your hardware in a “local”/UDP mode is definitely one of the strengths of the WeatherFlow hardware. However, WeatherFlow also offer the “remote” or perhaps “smart” mode of operation where your local data is sent to the WF servers and is enhanced by being passed through their algorithms. This mode allows simple parameters to be derived (like sea level pressure), but also allows more enhanced processing/variables to be derived such as RainCheck and the lightning variables.

You are free to choose between the two modes, but it is not reasonable to expect them to remain functionally equivalent. Otherwise, in extremis, the app shouldn’t even report sea level pressure because this is not available at a local/UDP level. I’m spitballing here but perhaps in the future WF could sell a local processing unit containing all their magic algorithms (like a suped-up hub) that would allow the local data to be processed in an equivalent manner to the processing done on their servers. That way you could then expect much closer functional equivalence.

5 Likes

I understand, but sea level pressure is something you can derive locally on your own, filling in missing lighting data isn’t. :slight_smile:

I’ve read the specs on the as3935 chip and I think I understand the limitations. I guess I’m just a little surprised that with a storm that was fairly energetic as it came at us, moved overhead, and departed over the weekend, only 3 strike events were picked up on the UDP broadcasts.

I guess the question I still have that isn’t clear from the FAQ is can we expect the characteristics of the lightning data that is sent over UDP to change as any remote tuning of the as3935 chip-level parameters are set on our individual station? Is it possible the volume of lightning events might change as our individual sensor is better tuned?

It’s clear my interest in tinkering with this is different than WeatherFlow’s intent and I can deal with that. It might motivate me to pickup an as3935 board and do some tinkering with that.

1 Like

but you are free to use your own processing of the udp data. I assume that’s the idea behind it. You can do your own false lightning suppression, or you could add some strikes based on some other source. All you get is the raw data, it always have been that way.

One of the differences between tempest and Air is that they can tune the parameters of lightning in the tempest (at least that is what I remember reading somewhere). So if yours isn’t sensitive enough they might change some of the parameters. You might ask them if they could do so for you (but I’m not sure if that requires a special firmware or not)

1 Like

These maps might be helpful to Tempest and AIR/SKY owners (personal hobby use) trying to better understand their last detected lightning data. You can look at the public blitzortung lightning maps without any hardware or membership (pan and zoom, also see menu settings upper right): https://map.blitzortung.org/#3.75/27.48/-64.81 (there are other maps, including dedicated local areas and regions http://en.blitzortung.org/live_lightning_maps.php ).

For those interested in a machine with tunable parameters, you might consider getting on the list for a blitzortung receiver. There is some assembly both of the box and antenna. Usually a long waiting list too (can be years), but if you like to tune parameters, this may be the perfect project: http://en.blitzortung.org/cover_your_area.php

For those who want more detailed lightning data including polarity, type, and strike current estimates (hosts get some access for personal hobby use, not public, except for some low res pics) with a pre-assembled box (no tuning), the TOA network might need a host in your area: http://toasystems.com/contact-us/host-a-sensor/ (I have no idea if they are still adding stations, but the web page is still there)

I host both, but otherwise have no affiliation with either. You buy the blitzortung hardware (reasonable for what it is), TOA provides all hardware at no cost. They both work as networks of many receivers.

[I will not reply below so as not to derail the thread. The main point was (and is): There are ways to look at lightning data to have something compare to Tempest. --No comparison of system performance or purpose was intended. – I too have had no word with TOA in some time (also years), however the live data is still there. I had no problems building or installing the blitzortung receiver (my initial location works fine) settings take some work, some luck may be needed as to location, others may have a bad experience.]

1 Like

Disagree. A derived observation has a published algorithm that we can get to ourselves based on our local observation from the UDP messages plus a bit of math.

Lightning is ‘not’ derived. It is totally made up even more than RainCheck. If you want to be buzzwordy, lets be polite and call it crowd-sourced.

But it is ‘not’ derived or derivable for folks who want ‘local-only’ control of their data.

What I expect in the UDP is reality as seen in my location. Simple reality.

So I would expect (1) and (2) in your list above, and would not expect (3).

Short answer is that Tempest measurement ‘local-only’ in ‘realtime’ is very questionable for rain readings, and just should be totally ignored for lighning. Important to know for third-party developers and their users.