API/Websocket/App rain discrepancy

@corrineb, I am trying to reconcile a discrepancy in rainfall amounts from yesterday given by an API call today, what is shown in the app history for yesterday, and what was collected live over the websocket yesterday.

Both the history in the App and my Weewx database that was listening to the websocket show a total rainfall of 28.3 mm yesterday, however an API call this morning gives a total of 27.4 mm. I cannot identify what is causing the difference.

The API call I am using is: https://swd.weatherflow.com/swd/rest/observations/device/16392?time_start=1597532400&time_end=1597618800&api_key=20c70eae-e62f-4d3b-b3a4-8586e90f3ac8. We are currently on British Summer Time, so the start and end times are 23:00 UTC 08/15/2020 until 23:00 UTC 08/16/2020.

if you shift your api call by exactly one hour, does it match the app values?

Good question! Let me check…

Edit: Nope, that doesn’t help. Shifting forward by 1 hour gives 27.3 mm. Shifting back (just for completeness) gives 27.8 mm

1 Like

you do have one measurement too much in your query (deduct one second on the end time), but that only would make the difference worse if you had rain in the last minute. So that doesn’t help.

what happens if you plot the difference between the two? Is it one value that is way off?

something went wrong at epoch 1597591020 ( Sun, 16 Aug 2020 15:17:00 UTC). the measured amount of rain in that minute is 0.6177 (equiv of 37.06 mm/hour) but the accumulated value increased by 1.503 an extra increase of 0.885. All the other values are spot on. So, it looks like the accumulated value changed unexpectedly.
Here is the data from the api:
1597590900 3589 0.07 0.933425 0 0.98 3250 3.51 1 29 24.528226 3 3 null null 0
1597590960 4175 0.09 0.841153 0 0.42 1.83 214 3.51 1 34 25.369379 3 3 null null 0
1597591020 5013 0.13 0.617782 0 1.07 3.08 234 3.51 1 41 26.872376 3 3 null null 0
1597591080 5666 0.15 0.275281 0 0.92 2.46 225 3.51 1 47 27.147657 1 3 null null 0
The difference between the values in the column for accumulated rain (the 11th column, starting with 24.528226) is:
0.933 (from the row above, but not shown)
0.841
1.503
0.275
Which match the reported values for rain in the last minute (rain rate) in the 4th column except for the one at epoch 1597591029 where the rain rate reports 0.618 instead of 1.503.
Pretty strange. There isn’t anything more I can find so it is now up to @WFstaff to find the reason why.

2 Likes

Thanks for looking into all the detail @sunny! I am supposed to be working from home, but this difference caught my eye and I wasn’t really going to look into it further until this evening.

Given that everything was matching yesterday (app, weewx, PiConsole; all using the websocket) it seems instinctively (although I could be wrong) that it is the rain rate that has been stored in the API that is wrong rather than the accumulation. The rain rate in the original websocket messages from yesterday must have been correct to match the accumulation given in the API today. Let’s see what the WF team can find out.

That’s also a good point. I should get that fixed in the PiConsole.

@peter I will take a closer look at the data. What index are you using in the obs array for your calculations? I want to make sure to rule out any RainCheck issues.

Thanks @corrineb. I am using index 3 from the SKY API calls. I am just doing a simple sum over all values that are not equal to None.

Thanks for the extra info @peter. I just wanted to make sure that RainCheck was not a factor. I did the same thing that you did and I am noticing the same discrepancy. I did a quick quick check of the 15th and those numbers look good to me. We are not sure of the exact cause of the issue of the 16th, but we will continue to look into it.

1 Like

Thanks @corrineb. It’ll be interesting to hear what you find

@peter We re-ran the bucket generation script, which also recalculates the rain accumulation, for 8/16. We are not sure of the root cause of the issue, but it should be resolved now.

Looks good to me. The accumulated value (index 11) and the sum of all the individual 1-minute rain accumulation values (index 3) now match.

It is interesting though that whatever caused this glitch also caused the 1-minute rain accumulation value in the websocket to be incorrect. I think the error happened at epoch 1597591020 (based on @sunny’s analysis), but I can’t be sure