Change in graphing rain rate

Ah, ok. That’s a different issue, then. related to the way our graphing system works. Currently, the graphs let you zoom and in and out and the data gets thinned as you zoom out. At the “deepest” level, you see raw 1-min observations. As you zoom out, the 1-min records get thinned to one record every 5-min, then one ever 30-min, then 3-hour, then 1-day. These “data buckets” are described in more detail here. The reason for that is to prevent the graphs from choking on too much data. If you tried to look at a year of one-minute data, for example, that would be over 500,000 points which is too much for many phones to handle.

A post was merged into an existing topic: Data archive buckets explained

My phone only shows 32 points in your graphs on any screen. Surely any phone can handle more than that.
500.000 data points in a year is not what we expect on a phone. The highest number would be the max number of pixels along a horizontal axis, but even half of that or a quarter would be great. And the phone might cache a lot extra to allow smooth scrolling backward and forward in time.
So at the current time, the resolution could be a lot higher without problem and even allow scrolling back in time, requesting more data as the user goes back in time.

But if you would like to reduce disk storage on the server side, it would be understandable to average observations together that are more then a few days old.

Most of the measurements don’t change very rapidly so for those a higher resolution isn’t needed. For rain rate this is different, and the higher resolution graph probably also prevents some of the confusion that was present with the previous bar graph version. That one showed the rain rate during the time of the width of the bar.

A post was merged into an existing topic: Data archive buckets explained

sunny,

If I understand correctly, the graphing software is in the application and the data is sent to the application. The phone applications can’t handle being sent the mass amount of data.

The data on the servers is stored in a database so server storage is not an issue.

yeah it kind of seems to send more data then can be seen at once on the screen. however, at the most detailed zoom level, it sends 1 day of data. And that currently is already a lot of scrolling for 1440 points of data. But nowhere near 500.000.

The phone could store easily a week of high res data (15 observations per minute, is around 150.000 points. Phones have enough permanent storage to do so, but is not something you want to receive over the air every time you start the app).

I wouldn’t expect high res data for very old observations anyway.

I am worried the phone application will become so bloated I won’t use it and then how will I add and delete devices.

Yes. The graphs have some issues and this is one that we intend to correct!

2 Likes

I want to see accumulated rainfall from my system: daily, weekly, monthly, and especially water year. This is hugely important for an understanding of hydrologic changes. It may not be THE most important parameter that describes climate change impacts and adaptation, but I just can’t think of one that’s more important.
Dave
Hydrologist

1 Like

Hi Dave. The Smart Weather app itself is purposely simple, but the data is easy to access to encourage working with it. Check out this list of Smart Weather Station Third-Party Applications over in #owners:integrations, especially Weather Display which has all sorts of bells and whistles!

1 Like

With respect to your Point 2 of Nov '18 to use the descriptive intensity rates then the categories seem to have a bias towards temperate conditions and suggest a worldview of uniformity and linearity that is rarely found in nature.

Rainfall intensities in the humid tropics are significantly higher than in temperate areas and require different groupings. Likewise, rainfall rates in semi-arid regions are considerably lower.

Applying the temperate standard globally means that a significant proportion of “normal” rainfall events in, say, the humid tropics will fall into the very heavy and extreme categories. I don’t see how this will help “the average user”…

Since the categories defined aren’t universally applicable, it would be better to use the actual rain rates, as you suggest in your Point (3), and show the values on a second y-axis.

To guide “the average user” in their interpretation, categories suitable for other regions (e.g. the humid tropics and semi-arid conditions) should be added to the existing temperate groupings.

Users’ data on actual rainfall rates, distributed to these three regions of temperate, tropical and semi-arid, could be used to define more relevant thresholds for rainfall intensities.

Edit: clarification

2 Likes

Hi Micael. It is an excellent point that one person’s “heavy” is another person’s “light” when it comes to rain. To be sure, there is no universal standard for descriptive rain rates. We tried to go with a “best fit” by looking at how others do it (you can find a bit more detail in this topic: Rain Intensity values - #3 by WFmarketing )

Others have suggested a customizable scale (buckets/cats/dogs), but that would add a good deal of complexity to the app that most would not use. Your suggestion of three different regions is interesting… Do you know of any source that has attempted to do that?

I love the idea of letting the data decide - over time that would be a really interesting project (or masters thesis, perhaps?)

That’s the plan!

2 Likes

Hi David,

Sounds great.

I have also looked in vain for such comparative categorization and, though there is consensus of the significant differences between rainfall characteristics (e.g. frontal, convection, orographic) and where on the globe events take place, comparative assessments are far between. There is one here, though (let me know should you want the complete article).

A study on Rainfall Erosivity in Europe (2015) uses a revised USLE (i.e. RUSLE) to estimate erosivity based on rainfall amount and intensity (multiplying kinetic energy by the maximum rainfall intensity during a 30 min period for each rainstorm). Thus, rainfall intensity rates are a crucial element but are not specifically shown in the report. However, the report clearly states that there is large amount of data available regarding rainfall intensity. Given its large geographical scope and availability of data this study may provide interesting ideas for how to go about organizing and presenting WF data.

Other sources are using categories for the intensity levels but often fail to include proper references to their origin – for instance, your 2nd link from AMS contains no reference, Wikipedia (2nd and 3rd categories refer to AMS), and Central Weather Bureau in Taiwan (also no reference).

A simple way of doing categories may be to establish these as intensity levels where, for instance, rainfall rates representing 90 – 100% of highest values correspond to Extreme events, and so forth. However, I wonder what researches and meteorologist would respond to such a system and what they would suggest as appropriate percentage levels.

One impact of climate change often reported (e.g. this search) is the general expectation that rainfall rates will increase over time (i.e. warmer air contains more moisture). The above categories would not capture this dynamic, if only using the percentage system mentioned. Actual rainfall rates remain important (mm/inches per time unit) together with their processed averages (min, high, max) which would change over time.

Climate change hazards are not only increased rainfall rates causing (flash) floods and landslides, but also strong winds of increased strength. It’s interesting that Weather Underground (WU) group our data for wind in three groups (average, high and gusts), which I find very useful.

A couple of years ago, and thanks to a PWS available through WU, I looked at nearly 7 years of wind data, which is clearly too short for any major conclusions to be made. Nevertheless, in that period we did experience a marked increase in both average, high and gusty winds. Significantly, since 2015 many events of wind gusts were above 60 km/h which is when tree crowns start breaking up and trees will fall, which is - indeed - what happened when Nate was still a storm moving north in the Caribbean.

Obviously, both rainfall and wind data are important indicators documenting the unfolding of climate change around the planet. As a citizen science project, the WF network, and – as you rightly say, students, may contribute to documenting such changes. In that context, the rainfall erosivity study in Europe may give some inspirations.

A (global) citizen science project may also turn out to be an important motivator for the community of WF station holders if synthesized data on emerging results is provided as feedback. Indeed, visualizing such data would be a marvelous addition to the GaryFunk map on the location of WF stations.

3 Likes

Maybe it’s just me, but I’d love to see actual #s for rain rate. The data is there, why not use it? I think some underestimate who the users are. IMO, they’re enthusiasts who appreciate the data, the more the better. I always thought the Davis VP base station showing “It’s raining cats & dogs” was cute, but a bit silly for a device like this.

Showing actual numerical rates would obviate the need for relative terms.

That group makes up a very number of the total stations installed. By my count it’s less than 100.

So Gary, you’re saying that most who do have the WF aren’t in to the data end of things? You’re here a lot longer than I am, so I’ll defer to you.

Don’t take my opinion; count for yourself.

On the other hand, I don’t have a pony in this dog show. I don’t use the graphs.

My guess would be that most people like to see accurate numbers. A device that tells you it’s cold or it’s very hot is not what most people would prefer. They would see a temperature of -10 and would translate that mentally to what they think it is (probably very cold, unless you live in the artic). Especially people interested in the weather enough to buy a weatherstation, most of them would expect the correct temperature. Same is valid for other data like rain rate, accumulated rain, humidity, well basically all the data. The poll above currently tells us at least 75% would prefer it.

1 Like

That’s 75% of the people that voted, 26 out of 800 registered members. And 26 out of 2200 stations.

We are not the target audience. We are the 1%.

And by we, I don’t mean me. I just need a general idea of what is happening. I don’t need 99% accuracy.

Sofar the voting was done by 35 people and the 75% is probably even on the low side, as among the people who voted against it, are the people who build it. If we would discard those votes, the number goes up to 85%.
35 votes isn’t a lot but gives at least an indication of what.
My very strong gut feeling is that even people outside this group and the future customers would like the numbers. Most weather forecast in the news paper or on TV will show you the numbers. Next to it there often is some text helping users to make sense of it.
For temperature that is certainly true, for things like rain it varies a bit, some forecasts will predict 15mm of rain tomorrow, others just show a dark rain cloud. Uv index always a number (at least I haven’t seen anything else) wind speed, mostly numbers, barometric pressure, mostly numbers etc.
So you even though we are only 1%,there are some strong indicators besides the tiny poll, that people like to see the numbers.

1 Like