Lightning reports (tempest)

What I expect in the UDP is reality as seen in my location. Simple reality.

So I would expect (1) and (2) in your list above, and would not expect (3).

Short answer is that Tempest measurement ‘local-only’ in ‘realtime’ is very questionable for rain readings, and just should be totally ignored for lighning. Important to know for third-party developers and their users.

Remember that the lightning in the original Air and to some point the Tempest is just an early warning system and not much more. As such I think it is doing a rather good job. Indeed the Tempest version can be tuned remotely (my nose says it will be like the calibrations as we can get all a personal tuning and that isn’t backed in a firmware we all get identical)

As for the systems mentioned above TOA and Blitzortnung (I have both) they can’t run alone in local mode. All is server driven mostly. For TOA I have not much information since they never ever communicate (haven’t heard from them in years). For BO I have more access. Before a signal is used it has to go true several strict conditions. Already you need a minimum of 4 stations reporting the strike within a very very very small timelapse (GPS time stamped… count the numbers after the decimal …) Won’t go in all details but it is complex and the server behind is … rather a monster … And even with the BLUE versions the accuracy is +/- 200 meters on average … so don’t expect a single Tempest to be better in any way. And from all stations installed many are pretty bad since there is so much noise locally … and people quickly abandon … it can take weeks to find the sweet spot for such a station. Just as example, I have a very heavy military emitter 75 km east off me (for submarines) at 62kHz … it is spamming heavily my blue station and I need to use cutoff filters to get some reasonable signals.

Weatherflow has partnered to improve the warning system (and I guess also have some basis to compare for calibration) As most of you now know, lightning is very tricky and over sensitive to many interference (a switch, a bulb, TV … even several miles away can trigger false positives) … or if noise level is so high, the little Franklin sensor is deaf since it has to turn down it’s sensitivity to avoid false positives.

I think Peter’s description won’t be far from what we have and again don’t expect the Tempest to become more than an early warning system. It wasn’t build for that and doesn’t have the elements for it. Even if one day all signals could be processes like BO does … just the fact the timestamp isn’t accurate enough … (calculate the distance per second … BO uses 9 decimals after the second … and we can’t get better then 200 m average … image with just second as timestamp … )

Hope this brings things a bit into perspective. It’s an early warning system that will get probably some fine tuning via calibration. Server side Weatherflow, with extra information, will eventually filter out or in some signals based on third party input to improve the warning but according me it will end there.

5 Likes

Understood. I just wanted it equally understood that you can ‘not’ get all the data from UDP broadcasts, although everybody has been under that impression for multiple years (excluding RainCheck).

Third party developers shouldn’t bother to support lightning metrics if they use UDP to get the raw data.

Agreed. Without the crowd-sourced information (which I think is actually a good description of what WF are doing), you cannot recreate locally what WF do on their servers.

And also agreed. The UDP data will miss out on the more complex secondary processing WF do on the raw data.

1 Like

That’s not the issue being reported…The extra data from your servers is one thing but the issue from the poster is:

You have supplied conflicting answers. Please clarify.

Sorry for any confusion I caused here, folks. It is all a bit complicated and I often confuse myself. Here are some responses that hopefully clarify things:

This is still completely true. The local (UDP & BLE) data streams are functionally equivalent to the remote (WS & REST) data streams in the sense that the message format is identical. We will strive to keep that the same, for the sake of simplicity, though it’s possible they will diverge eventually.

Either way, the remote data stream has more and better data than the local stream because. And this will only increase over time. Simple “derived parameters” like dew point temperature are examples of this, and most can be derived from data the hardware provides. Sea Level Pressure is probably the simplest parameter that can’t be derived purely from the local data (you also need to know something about the meta-data: elevation and height above ground). RainCheck is a more complex example of a “derived parameter” that requires lots more data, meta-data and processing to compute - there’s no simple formula we can give you. The lightning data described above is yet another example of something that depends on more than just the data produced by your hardware. And here’s another: the forecast data available in the app (and soon via the API) is not something the hardware alone can give you - and therefore not something you’ll see on the UDP stream (yet? maybe one day?).

Our primary goal is to provide users with the best weather data period, even if it isn’t purely what your station measures. And this why we say: Tempest is more than just hardware - it’s a complete system. So, while all of the data you see in the Tempest app (and from the API) is informed by and driven by the sensor data your station observes, your station is not capable or producing all of this data itself.

This is still the case, @GaryFunk. No observation data leaves your network that is not available locally (via UDP and BLE).

That is still a valid assumption (see above). The local data is still useful, just perhaps not quite as useful as the remote data.

Well put (much more succint than my answer)!!

Correct. In fact, nothing has changed! All of the strikes detected by your Tempest hardware are put out over UDP.

Yes, that is one way of looking at it, if you also consider RainCheck and forecast data to be “derived variables.” Another way to look at it is that the lightning data visible in the Tempest apps (and available via the remote API) is “qualified” by our back-end system. The raw, unqualified lightning data is still available over UDP & BLE.

Yes!

Yes!!

Yes!!!

YESSSSSSSSSSSSSSSSSSSSSSSSSS!

The performance of the AS3935 is heavily dependent on local conditions. And even when it can be optimally sited (away from sources of positive & negative interference), it’s not perfect.

Yes to both questions. That’s what the last section of the FAQ refers to: the process of qualifying the raw lightning data and the ability to tune individual sensors will lead to improved reporting over time.

8 Likes

I’d reword Gary’s opinion (for me) to be “given the published equations, I can derive locally the same values that WF derives on their servers”.

Obviously impossible for RainCheck, but the addition of 'impossible for Lightning" is the thing that surprised us.

Hope you have a whole lot of compute to handle all the websockets or WebAPI requests we might need to work around that. Or maybe we’re just a vocal few. I dunno.

ok I know at least the “we’re vocal” part.

(data point - there are certainly several dozen weewx users running @vreihen’s driver, and several have asked about lightning recently, which is where this thread kinda started originally)

1 Like

Thank you, David. That clears up all the confusion.

@vinceskahan I think your client has a Tempest that us defective or in need of calibration. And you may very well have a hardware issue.

1 Like

Possibly, but we have ‘multiple’ people reporting the same thing. Basically zero to no lightning readings detected via UDP and zillions on the WF app. Both FT and Prod units I believe.

It’s not “impossible for lightning” - it’s “impossible for anything not produced by the hardware”. To be clear: all lightning data detected by the AS3935 sensor in your Tempest device is reported over UDP. It’s only the “extra” or “qualified” data (whether suppression or correction) that is not available over UDP.

Got it. I wonder if @vreihen’s weewx driver (and other UDP-listening apps) could use the web-socket interface as primary, with UDP as a back-up?

No worries there! Think of all the 3-sec rapid_wind messages coming in and going out now. Generally, that stream swamps the stream of lightning data events by a couple orders of magnitude.

That is certainly possible, but more than likely it’s a siting issue.

1 Like

I get warnings from both Air units and both Tempest devices.

I’m usually sitting on my spot when lightning strikes. Sometimes I have to remind the family, Your on my spot.

2 Likes

Guess we’ll see when my production Tempest someday arrives.
Thanks for the explanation !

1 Like

Technically of course yes, but everybody (almost) based their solutions around the concept of local UDP data having enough to calculate everything derived, so (theoretically) we could run without any connectivity off our LAN once the tunings from CL were stable for our sensors.

2 Likes

The thought has crossed my mind a few times to use the web copy as a pseudo Davis archive logger for back-filling missing records. The only WeeWX station driver that implements this is the Davis one, which not-so-coincidentally was the inspiration for WeeWX and its LOOP versus ARCHIVE packets.

The whole point of WeeWX is to capture, display, archive, and report on the station’s output locally. There is even a whole lecture in the developer’s guide about only reporting items that the hardware emits. If it isn’t supported by the hardware, WeeWX itself will try to derive the missing values based on what it has at a level above the driver in the software stack.

The sample sensor-map distributed with the driver pulls all wind from rapid_wind packets, and calculates the summaries from that and not obs_sky (or obs_st). All of the other fields like temp/hum/pressure/solar come from obs_air/obs_sky (or obs_st). This works well for everything but rain and lightning, as @vinceskahan has pointed out.

As I had written in the WeeWX thread last weekend, they added official support for storing lightning data from AS3935-type devices as of a few months ago. A few users lucky to be near t-storms tested sample sensor_map entries to capture the evt_strike UDP data, and the counts were so far off (low) in many cases where people were even questioning whether evt_strike packets were being sent by UDP. Long story short, I was going to provide updated sample sensor_map setups that captured evt_strike and push it out as the next point release. I now feel obligated to also include a multi-paragraph explanation about why the UDP lightning data will probably never match the Tempest app’s data (to save having to answer this “bug report” every day), my usual lecture about the AS3935 being intended for consumer electronics and not scientific instruments, and of course a legal disclaimer that lightning data is provided for entertainment purposes only,and not to be used as a sole source of information for life-or-death decisions…

5 Likes

I’m going to mark David’s explanation as ‘solution’ but leave this one open for futher commentary.

2 Likes

Gotcha - that definitely keeps it simple.

Good idea! Feel free to point out how we (WeatherFlow) see the Tempest hardware as just one piece in a much larger system of weather data, a concept that may be unfamiliar to owners of other personal weather station hardware. It’s really a hybrid software-hardware platform (though marketing does not love that mouthful, so they are not using it!).

A good point as well - the AS3935 is great for it’s size and cost, but it’s definitely not a professional lightning sensor. The weaknesses of it are part of what drove our development of the “enhanced lightning” system described above.

Speaking of the “consumer vs. professional” distinction reminds me to mention that we do intend to develop and support a line of more commercial-grade (“professional” if not “scientific”) weather sensor hardware, including a better stand-alone lightning sensor. That’s still a ways off, but it’s on our roadmap. For now, we’re staying focused on the consumer market.

3 Likes

400+ strikes yesterday as a cluster of strong thunderstorms moved through our area. I have every reason to believe the number was real and not an embellishment. Since my Tempest now appears to be fully calibrated, I’m convinced this is a nice upgrade from the Sky/Air combo.

I haven’t gotten a good handle on rainfall accuracy, but I’m sure that will remain the weak point.

At any rate, color me impressed overall. :slight_smile:

2 Likes

Hello – I’ve read through all of this thread last night and again this morning. I’m still a bit confused. Lots of lightning strikes last night and I did not pick up a single strike via “obs_st” – the fields stay at “0”. Also “evt_strike” never fires off. This is for my new Tempest I got a few days ago. Just trying to understand how I capture that data. Someone in this thread pointed out that all the UDP data we got via Air and Sky we still get with Tempest. My question is how do we capture it (lightning) via UDP? Or can we? Thanks – JP

Hi JP. Any lightning the as3935 sensor in your Tempest detects will be seen in the UDP broadcast. But you will, in general, see more & better lightning information via WS & REST thanks to the “Tempest is more than hardware” concept described above. You can expect the hardware to improve over time because of the same concept (as we collect more data we can tune your lightning sensor). If you continue to see minimal-to-no lightning when you think you should be seeing it, you either have a siting issue (more likely) or a hardware issue (less likely). Feel free to open a support ticket and someone from our kick-ass customer service team will help you figure out what the issue is and how to address it.

1 Like

Thank you for taking the time to reply. I was seeing plenty of lightning via REST last night so I do not think it is a hardware or sighting issue.

Back to UDP – how do I see the lightning data? If it’s “obs_st” it did not register as the lightning fields were “0” – all other data came in. If the reporting period is 1 minute I presume it will have to have a strike at the reporting time to register via “obs_st” (not sure about that - just guessing). If it’s via “evt_strike” that never transmitted last night for the Tempest (works for the old Air and Sky).

Is there another string to catch via UDP for lightning?

Thanks
JP