I just got my Tempest and after searching around I couldn’t find a udp listener script that wasn’t overly complex or contained other functionality (redis, mqtt, etc) so I figured I would create one.
Contains maps for the main event types and outputs a dictionary of the observation values. Works in Python 2.7/3.x with no dependencies. Tested on Win10 and a Raspberry Pi.
Hopefully this is useful for the beginners to play around with.
Really ? Mine is pretty simple in its most minimal form…
python listen.py --raw
or
python listen.py --decoded
I like your sensor map thing though. Cool construct.
Yes I did see yours(which is pretty cool) but, while the cli commands are easy to use, the listen.py script is pretty complex and would be hard to extend/modify for beginners who may just want to write the observations to a file or trigger some action when it starts raining, for example. Didn’t mean to step on your toes, just wanted to create an alternative for anyone that might be starting one of their first adventures into Python by playing around with the Tempest.
Oh geez no. No offense taken. More choice is always good and it’s fun to see other people’s approaches…
Hello! I just picked up a new Weatherflow Tempest and I’m really interested in this python UDP listener. It’s the only script that I was able to use (so far) that I can get to listen in on the UDP broadcast packets. I’m looking to build some dashboards from this raw data and this gets me really close. The raw JSON is actually a bit more useful but I love the dictionary of observability maps. The problem in my case is the output isn’t formatted in consumable JSON. Do you think I could work with you to get a simple line output of flattened JSON for each of your maps? Strictly name-pairs? Like an NGINX log for Weatherflow? 
Thanks again!!
_DS
From the line data, addr = s.recvfrom(4096) the data var is the raw json string, is that what you are looking for?
So yeah - I used:
# convert data to json
data_json = json.loads(data)
print (data)
To just get all of the raw json into my logging solution - problem is - I really like the name pair break out - it’s just that I can’t use the raw JSON as is (in my tools):
[[1612551064,0,0.49,1.3,123,3,986.52,8.89,38.61,70754,4.31,590,0,0,0,0,2.442,1]]
To create name pairs. What I’m trying to get close to is how the API returns data (just live from UDP):
{"timestamp":1612551124,"air_temperature":8.9,"barometric_pressure":986.4,"station_pressure":986.4,"sea_level_pressure":1009.6,"relative_humidity":38,"precip":0,"precip_accum_last_1hr":0,"precip_accum_local_day":0,"precip_accum_local_yesterday":0.072504,"precip_accum_local_yesterday_final":2.654446,"precip_minutes_local_day":0,"precip_minutes_local_yesterday":1,"precip_minutes_local_yesterday_final":1,"precip_analysis_type_yesterday":1,"wind_avg":0.6,"wind_direction":265,"wind_gust":1.9,"wind_lull":0,"solar_radiation":433,"uv":3.38,"brightness":51937,"lightning_strike_count":0,"lightning_strike_count_last_1hr":0,"lightning_strike_count_last_3hr":0,"feels_like":8.9,"heat_index":8.9,"wind_chill":8.9,"dew_point":-4.7,"wet_bulb_temperature":3.5,"delta_t":5.4,"air_density":1.21831,"pressure_trend":"steady"}
Which is totally usable!
This kind of output:
{‘Datetime’: datetime.datetime(2021, 2, 5, 14, 14, 3),
(‘Time Epoch’, ‘Seconds’): 1612556043,
(‘Wind Direction’, ‘Degrees’): 339,
(‘Wind Speed’, ‘m/s’): 0.89}
would be okay - but it’s not JSON. (I’d also need it all on one line…)
I’m not sure I am following the problem here. Assuming you are using Python, it is relatively straight forward to recreate how the API returns there data from the raw UDP output:
udp_data = [[1612551064,0,0.49,1.3,123,3,986.52,8.89,38.61,70754,4.31,590,0,0,0,0,2.442,1]]
json_data = {"timestamp": udp_data[0][0], "air_temperature": udp_data[0][7]}
All the different values and their indexes are listed here: WeatherFlow Tempest UDP Reference - v143
My script (HERE) could easily be extended to do what you want. I already have a --raw option to print the JSON it hears, and a --decoded to decode that into human readable non-JSON format.
Just extend my script to add a new option that does the decoding+printing into whatever format you want. Should be very straightforward to add.
I’m not using Python I’m using Grafana Loki to ingest straight JSON log files but it doesn’t support arrays. So I’m was hoping to find a way to flatten/expand or map the arrays to JSON name pairs straight from this UDP listener.
@vinceskahan - I really liked what you were doing with your scripts but for some reason - it never showed any UDP messages with either any of the permutations (docker, command line, etc.). And it looked like you didn’t have any Weatherflow devices anymore.
Goal would be to have a single line of JSON name pairs for each UDP message without any arrays.
Will take these suggestions and see what I can put together. Will report back!!
Oh sorry, its been a while since I worked with this so forgot its received just as a csv… If you want the final dict as a string you can do json.dumps(observations).
Or as @vinceskahan said his script has a command line option that will give you what you want more easily.
Hmmm - I’ve never heard it not show the messages unless there was a firewall issue or perhaps you were running something else on the same computer that didn’t share the socket correctly.
(example - if you run the weewx driver you have to set it to share the socket as the driver’s default is to grab the socket and not share it among multiple listeners)
It is correct that I don’t have any WF gear any more, but the UDP API hasn’t changed and I do try to answer questions, FWIW.
I would need to know what platform, python version, os and version, etc. you are running in order to help much. If you got @pdoyle003’s listener to hear the UDP there should be no reason mine wouldn’t also. We’re basically doing the same thing under the hood.
A quick thing to try would be to add the --address option to bind to a particular network interface.
# use your network address below
python3 listen.py --raw --address 1.2.3.4
(apologies if I kinda hijacked the thread there…please feel free to PM or email me if you need more help on my listener specifically, or ask question about my script in its own thread to not go offtopic here…)
Maybe you should look at JaveScript code to give you what you want.
Thanks for all of your help and quick feedback. I’m still going to work on getting raw logs into Grafana Loki - but for now, I was actually able to get @vinceskahan UDP InfluxDB exporter working. Not sure why the earlier attempts didn’t discover any UDP traffic but I tried it again while working through this review and was able to get data pumped in. (Thank you for sure!!) I’ll post my final results of my dashboards once they’re a little more cleaned up but so far it’s working well. Thanks again!!
Glad to hear it. I guess in hindsight I could have added an option to output complete decode JSON but nobody ever asked for it.
In double why-didn’t-I-think-of-it hindsight I should have written a WF Simulator to pump out the same UDP that the gear does so folks (like me) could do development of apps and stuff like this without even needing real gear on their network. I might actually take a crack at that one of these weeks…
Now I’m curious why you don’t have a Weatherflow Tempest any more
If you write it as a WeeWX report gizmo that sends LOOP packets via UDP, you could use the simulator station driver and also make a potentially useful utility.
FWIW, I slapped together a quick Python program to broadcast fake Atmocom UDP packets when I off-target developed that WeeWX station driver using nothing more than a few packet captures from someone who owned one…
It looks as simple as said.
With a Tempest under test, having succeeded with the basic functions, and now aiming at reading the UDP-messages for generation of JSON-files and XML-files for application ‘elsewhere’.
Applying RPI with Raspian_Buster, for that purpose installed & added dicttoxml for the imports, and tried following construction in PeterDoyle’s script, first for the ‘obs’-list.
if data_json['type'] == 'obs_st':
# takes the map list and the list of observations and creates a dictionary with the first value in
# the map as the key and the first value in the list of observations as the value, second value
# to second value, etc
observations = dict(zip(OBS_ST_MAP, data_json['obs'][0]))
observations['Datetime'] = datetime.datetime.fromtimestamp(observations[('Time Epoch', 'Seconds')])
print ('List of values')
# Make&fill JSON-file
with open('/home/pi/Tempest_List.json', 'w') as outfile:
json.dump(observations, outfile)
# Convert dictionary to XML-file & print
TempestObs_xml = dicttoxml.dicttoxml(observations, attr_type=False)
print(TempestObs_xml)
xml_output = open("/home/pi/Tempest_List.xml",'w')
xml_output.write(TempestObs_xml)
xml_output.close()
pprint.pprint(observations)
However, with Python2.7 from Putty’s CLI stuck for the JSON-generation with an error report
List of values
Traceback (most recent call last):
File "/home/pi/weatherflow_listener1.py", line 98, in <module>
json.dump(observations, outfile)
File "/usr/lib/python2.7/json/__init__.py", line 189, in dump
for chunk in iterable:
File "/usr/lib/python2.7/json/encoder.py", line 434, in _iterencode
for chunk in _iterencode_dict(o, _current_indent_level):
File "/usr/lib/python2.7/json/encoder.py", line 382, in _iterencode_dict
raise TypeError("key " + repr(key) + " is not a string")
TypeError: key ('Illuminance', 'Lux') is not a string
Variations with json.dumps or with python3 just give different error reports related to json-generation.
For xml-generation similar effects.
Apparently somewhere overlooked a ‘subtlety’:
any idea what & where?
Impossible to help without seeing the JSON for python2 and the JSON for python3.
It is not unexpected that python2 code will not work perfectly in python3, depending on what data it’s written and how perfectly the original python2 code is written. It’s not unusual to find places you need to tweak to make the code run equally well in both python versions.
I think we need to see one observation of your ‘Tempest_List.json’ for python2 and python3 to be able to help any here.
Vince,
Attached are the requested files.
To allow upload to this forum changed the extension to .txt.
Tempest2 is from python2, and Tempest3 is from running python3.
IMHO it proves that the files are being opened, but no filling occurs (because of other errors).
Tempest_List.txt (281 Bytes) Tempest2_List.txt (1 Byte) Tempest3_List.txt (1 Byte)
As indicated, not same fault indications for python2 and for python3 (as expected).
Below is the faultreport when running the modified Simple_UDP_reader_script under python3 from Putty’s CLI.
Traceback (most recent call last):
File "/home/pi/weatherflow_listener1.py", line 98, in <module>
json.dump(observations, outfile)
File "/usr/lib/python3.7/json/__init__.py", line 179, in dump
for chunk in iterable:
File "/usr/lib/python3.7/json/encoder.py", line 431, in _iterencode
yield from _iterencode_dict(o, _current_indent_level)
File "/usr/lib/python3.7/json/encoder.py", line 376, in _iterencode_dict
raise TypeError(f'keys must be str, int, float, bool or None, '
TypeError: keys must be str, int, float, bool or None, not tuple
To check the scriptlines, also done another test, by insertion of a comparable jsonfile-generating script earlier in Peter’s script.
# convert data to json
data_json = json.loads(data)
# Make&fill JSON-file
with open('/home/pi/Tempest_List.json', 'w') as outfile:
json.dump(data, outfile)
Result is in file Tempest_List when running python2, which proves that construction seems OK (although contents perhaps not very useful)…
Errorreport when running under python3:
Traceback (most recent call last):
File "/home/pi/weatherflow_listener1.py", line 90, in <module>
json.dump(data, outfile)
File "/usr/lib/python3.7/json/__init__.py", line 179, in dump
for chunk in iterable:
File "/usr/lib/python3.7/json/encoder.py", line 438, in _iterencode
o = _default(o)
File "/usr/lib/python3.7/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type bytes is not JSON serializable
As indicated earlier ‘probably some subtlety’, in this case some differing aspect of python2 and/or python3.
Anton