Is binning the lightning distances really the best way to display that info?

I have noticed that when I receive an alert for lightning on my iOS Device, it is either a 22-mile or 12-mile range. Are these the only range “steps” provided in the alerts?

I think the unit, for some reason, bins the distances in 5 classes.
I would assume it can give alerts for all the classes. Note that “To prevent users from being overwhelmed by lightning detection alerts, we set up a rule that our software follows: for each lightning strike, an alert is sent if it’s been more than 30 minutes since the last alert or if it’s closer than the previous strike.”
(of course I would prefer the exact guesstimate of the distance instead of the binned value. There will be inaccuracy involved, but binning is NOT the right way to display it. The same inaccuracy is involved with binned data, a data point might be on the edge of the bin, and end up in the wrong bin because of the inaccuracy. Note that I think the sensor itself gives distances in 14 steps. Show them to us!)

The lightning strike distance estimate is presented as a range and there are actually five “steps” or “bins” of ranges: 0-5km, 5-10km, 10-20km, 20-35km, and >35km. In miles that works out to 0-3, 3-6, 6-12, 12-22, 22-32, and >32.

Correct - you won’t necessarily get an alert for every lightning strike.

Actually, it’s 15 (including “0”). Yes, the sensor itself provides a “statistical distance estimatation” to the “head of the storm”. If you’re curious, give the sensor’s tech sheet a read. It manages to be both incredibly detailed and incredibly vague about what those definitions actually mean!

Although this value can be one of 15 precise value (0, 5, 6, 8, 10, 12, 14, 17, 20, 24, 27, 31, 34, 37 & 40 km), we have found this value to be misleading in its precision. That’s why the presentation of the value is “binned” or “smoothed out” in our apps. We present the value as a range in an attempt to convey the uncertainty in the device.

Note, in addition to uncertainty in the distance estimation, the lightning sensor also has a variable “efficiency” in detecting strikes at all. This ranges from ~20% at the 40km range (missing roughly every 8 our of 10 strikes) to ~40% (missing roughly 6 out of 10) at the closer range.

3 Likes

That is not the way to represent uncertainty. and binning the data this way actually increases the uncertainty. For example if the precision of the device is +/- 6 km and the value it gives is 12km, one could expect the real distance to be between 6 and 18. After you binned it in the range 10-20km, the user can expect the value to be between 4 and 26, making the device appear way less accurate then it is.
You know of course that the correct way to represent this data, is as a dot in the graph with a vertical bar that gives the expected accuracy. If the accuracy is symmetrical about the measured value you could perhaps leave the dot out and only show the bar, but that bar is then still centered around the measured value.
Possibly, the height of the bar might vary with each measurement to indicate that a measurement of 8km is more precise than one of 37km (if that is the case with this sensor. I can’t find anything in the specsheet about accuracy)

1 Like

You’re right. I totally agree that an error bar would be a better way to represent the data on the graph than by fixed range bins. Unfortunately we don’t actually know the uncertainty in the distance estimation - the manufacturer of the sensor is vague at best on that point. But we can come up with our own estimation with a little further testing. I’m going to move this topic to the #owners:feature-requests section, and add it to our backlog. Thanks, for bringing it up, @FP761. And thanks for pushing us on it, @sunny :slight_smile:

2 Likes

Great! When doing further testing, keep in mind that the sensor is supposed to give distance to the nearest part of the storm, not to all lightning events, only the nearest one. So you cannot use all lighting reported by other sensor networks, to determine the accuracy. Also keep in mind that some other networks report only cloud to ground lightning, whereas this one also reports lightning that doesn’t hit the ground.
The only reference to accuracy of this sensor I could find claims 1km, but that can’t be true for all values and must be marketing at work.
Of course the quickest and probably most clear way, is just to display the data without binning and without an error bar. If you ever find out what the real accuracy is, you could mention that in the specs of the unit.

Error bars would be lovely. Too bad the sensor mfg can’t be more precise lol!

Today’s release of Smart Weather Apps (v3.2) changes the way we present the lightning distances. Previously, each of the 14 possible values the lightning sensor returned were shown like this on the graph:
image
These same 14 values were displayed in the text parts of the app with five different values “<5km, <10km, <20km, <35km and >35km”.

As of today, we’re not doing that anymore. Instead, each value is presented (both in text and on the graph) as a range, centered on the value returned by the sensor, with roughly +/- 3km on either side, like this:

image

6 Likes

Would you be willing to share the exact values and ranges that will be reported so that we can incorporate similar behaviour into 3rd party applications? Cheers!

Yes, happy to share: lightning spec july 2019.pdf (65.9 KB)

3 Likes

Great! Thanks very much :smiley: .

1 Like

great improvement. Thank you!

2 Likes

@dsj Sorry to dig up an old post, but have any changes been made to lightning strike distance bins? I noticed today that I was getting a raw value of 1 km, which isn’t listed as a raw value in the document you shared from July 2009.

Hi Peter. Yes, that looks like a typo in the spec. The lightning sensor should never report “0”. Rather, “1” is the lowest value you’ll see. Good catch!

2 Likes

note that when the sensor returns ‘1’ it is supposed to be interpreted as “the storm is overhead”. It is the lowest value it can return. If you would tell people the lightning is 1 km away, that might be confusing. The current graph seems to be implemented as displaying a bar form zero to 4(?).

That’s right. A “1” translates to “overhead” according to the technical data sheet provided by the Franklin sensor manufacturer. In our experience, we’ve found the values reported by the Franklin to contain quite a bit of uncertainty. That’s why we report the “1” value as “0-5 km” in the apps. Reporting anything else would be inconsistent with the data we’ve seen. We try to err on the side of caution.

Note that we have improved the ability to calibrate and fine-tune the Franklin sensor in the Tempest device and we will likely need to re-evaluate how we report lightning (for the better) in the forthcoming Tempest apps

4 Likes

why only in the tempest?

Because the Air/Sky do not have a usable remote firmware update process for starters…

1 Like

data handling is normally done on the hub, so whatever data sky or tempest is sending to the hub, it can be massaged on the hub. The hub does fine firmware with upgrades

Their work is at the chip level in the Tempest, not post-processing at the Hub. Unfortunately, there is no over-the-air capability to back-port this to the Air/Sky AFAIK…

1 Like