Downloading all the back data

Developers
1

Hi, like many folks here I’ve developed my own application for capturing the data from my weather station, saving it and displaying it how I like. What I’m curious about is finding out if there’s a way to download all the data that my station has logged here since it went online 9 months ago and putting that in my database as well. I thought I’d seen something in the API documentation that showed how to pull down single observation data from past days but I can’t find that anymore and it seems like an awkward and inefficient way to get the data anyway.

Any thoughts?

1 Like
2

The API has a method to pull an entire days worth of data per device. Just start with day 1 and increment until you get three consecutive days with no data.

5 Likes
3

Hi

Back-data are available from the API: Tempest API using the “day_offset” parameter in the query-string.

Using the development API key and the demo device this URL will give data from yesterday:
https://swd.weatherflow.com/swd/rest/observations/device/1110?day_offset=1&api_key=20c70eae-e62f-4d3b-b3a4-8586e90f3ac8

If you are familiar with simple coding it should be easy making a small script that exports data from the API to MySQL. I made a simple PHP-script and one year worth of data (676089 rows) took about 22minutes (local MySQL on my laptop). I used the json_decode() function and a loop for looping through day_offset=1 to day_offset=365. If you want all days you can loop through day_offset=1 to day_offset=1000 (or greater) and test if length on the json_decode() is greater than 0, if 0 then empty and no processing required.

Kind Regards

  • Waiting for my own WF to arrive so cant test with own data :slight_smile:
4

Yes, I’ve done that. I just figured that it was an inefficient way of getting the data. I needed to make 300+ web requests to download about 45 MB of data and then code up & debug the routines for the derived metrics. I have it all now and won’t need to do that again (hopefully) but it just seemed like being able to make a request to download an entire station’s data all at once wouldn’t be too complicated, especially since it could be zipped up before sending which would significantly reduce the bandwidth.

just a thought

5

So what would your recommendation to WF be? Be specific.

6

Once you have captured the old data it’s easy to stay current.

Also, the API allows you to use a start time and end time. So you can get a set time period of data.

1 Like
7

Okay, in an ideal world, I"d like to see something like this:

https://swd.weatherflow.com/swd/rest/observations/station/2049?data=all

That would return a zipped up version of all the data in a csv (or other common) format…

1 Like
8

I thought the start time and the end time in the api were within a specific day?

9

No, not (only) within a specific day.

10

Not necessarily within a day, but the timespan apparently can’t exceed a single day.

11

https://weatherflow.github.io/SmartWeather/api/swagger/#!/observations/getObservationsByDeviceId

Observation data at a one minute time resolution is available for a time range that is five days or less.

12

Hmm, I tried a minute over a day and got an "Unauthorized’ response. I’ll have to re-check my code/math.

13

If you prove it does not work then you found a bug to report.

14

Just to sum op, this is what I have found:

  • Five days or less using time_start and time_end: One minute time resolution
  • Using day_offset: One minute time resolution
  • More than five days AND no more than 5 months using time_start and time_end: 30 minutes time resolution

More than 5 month using time_start and time_end gives me something i don’t know what is:

        [0] => 2018-01-31
        [1] => 1021.6
        [2] => 11.6
        [3] => 85
        [4] => 7
        [5] => 0
        [6] => 20.8
        [7] => 4.7
        [8] => 1023.4
        [9] => 1019.8
        [10] => 100
        [11] => 61
        [12] => 1427
        [13] => 3.43

Maybe min/max data?

1 Like
15

Interesting. This is not documented so I will guess there is more that is not documented, such as below.

That looks like 24 hour data

16

Lets hope WF will update the API information :slight_smile:

17

There are a lot of changes coming in the API. As you know, the staff at WeatherFlow is extremely busy getting hardware produced, firmware updated and preparing for shipments to customers.

I’m as anxious as you to get the updated documentation.

5 Likes
18

Exciting days :slight_smile:

Also looking forward for information about how to get “Z4” data for a specific day:

19

I’m interested how one makes a request for any one of the “Z” data.

2 Likes
20

One of my objectives of getting the station was to have access to historical data in an easy way. Unfortunately, since I have very little coding experience, I guess I won’t be able to do that right?

1 Like