Your one file with actual content looks totally invalid to me.
"{\"serial_number\":\"HB-00033545\",\"type\":\"hub_status\",\"firmware_revision\":\"147\",\"uptime\":2439773,\"rssi\":-28,\"timestamp\":1617998643,\"reset_flags\":\"BOR,PIN,POR\",\"seq\":242138,\"fs\":[1,0,15675411,524288],\"radio_stats\":[25,1,0,3,65212],\"mqtt_stats\":[4486,7]}"
None of those backslashes should be there.
== Today several iterations on this message due to ongoing experiments ==
The contents of the ‘strange’ JSON-string are (running with python2.x) direct derivative of the ‘data’ read from messages by means of
Checked against Peter’s original script, which produces the following pprint-output (also using python2.x)
{u'serial_number': u'ST-00021804', u'type': u'light_debug', u'ob': [1618043269, 2522, 197, 0, 0], u'hub_sn': u'HB-00033545'}
{'Datetime': datetime.datetime(2021, 4, 10, 10, 27, 49),
('Time Epoch', 'Seconds'): 1618043269,
('Wind Direction', 'Degrees'): 0,
('Wind Speed', 'm/s'): 0.0}
{u'hub_sn': u'HB-00033545', u'uptime': 1629366, u'hub_rssi': -34, u'firmware_revision': 134, u'timestamp': 1618043269, u'sensor_status': 0, u'voltage': 2.66, u'rssi': -30, u'debug': 0, u'serial_number': u'ST-00021804', u'type': u'device_status'}
{'Datetime': datetime.datetime(2021, 4, 10, 10, 27, 49),
('Air Temperature', 'C'): 6.24,
('Battery', 'Volts'): 2.656,
('Illuminance', 'Lux'): 14191,
('Lightning Strike Avg Distance', 'km'): 0,
('Lightning Strike Count', ''): 0,
('Precip Accumulated', 'mm'): 0.0,
('Precipitation Type', '0 = none, 1 = rain, 2 = hail'): 0,
('Relative Humidity', '%'): 78.9,
('Report Interval', 'Minutes'): 1,
('Solar Radiation', 'W/m^2'): 118,
('Station Pressure', 'MB'): 1015.73,
('Time Epoch', 'Seconds'): 1618043269,
('UV', 'Index'): 0.86,
('Wind Avg (average over report interval)', 'm/s'): 0.77,
('Wind Direction', 'Degrees'): 64,
('Wind Gust (maximum 3 second sample)', 'm/s'): 2.37,
('Wind Lull (minimum 3 second sample)', 'm/s'): 0.0,
('Wind Sample Interval', 'seconds'): 3}
{u'uptime': 2484400, u'fs': [1, 0, 15675411, 524288], u'seq': 246599, u'firmware_revision': u'147', u'timestamp': 1618043270, u'reset_flags': u'BOR,PIN,POR', u'mqtt_stats': [4495, 7], u'rssi': -29, u'serial_number': u'HB-00033545', u'type': u'hub_status', u'radio_stats': [25, 1, 0, 3, 65212]}
Conclusion:
the \ \ in the ‘strange’ JSON-string seems linked to the u in the pprinted ‘data’ for Status or QC.
Indicator in the Tempest data to distinguish ‘Status&QC’-info from ‘real’ data?
Using python3.x these u’s are absent, and all output from Peter’s script per message has same type of layout.
Further down in Peter’s script these data are input for the dictionaries.
Have the nagging feeling that the json-module imposes requirements to earlier stage:
if ‘data’ is not a ‘clean&legal’ content for generation of dictionary ‘observations’, then the dictionary is not properly filled
=> not compatible with the added, ‘standard’ json generation-function
=> the script strikes due to deviation.
With module dicttoxml a similar experience:
also not digesting the dictionary ‘observations’ (probably for same reason).
Differnt error reports for python2.x and python3.x
Comes the question:
how to simply ‘clean/shrink’ the data (and hence the resulting dictionary) to produce dictionaries compatible with json.dump?
Some aspect of the reference dictionary-mapping?
just a wild guess, isn’t the backslash an escape character for the double quote?
eric,
Have looked in the ASCII-tables, but cannot find any conjunction.
For the second error it looks like you are trying to dump the original data object, not the data json. It should be: json.dump(data_json, outfile)
As for the original issue, the reason you can’t dump the observations dict is because the keys are tuples, as it indicates in the error TypeError: key ('Illuminance', 'Lux') is not a string. Depending on what you are wanting to do with the json there are a few ways you could fix this. The easiest would probably be to edit the OBS_ST_MAP variable to just be strings and get rid of the units:
string_map = [key[0] for key in OBS_ST_MAP]
observations = dict(zip(string_map , data_json[‘obs’][0]))
Then you can dump that out to a file.
If you wanted to keep the units you could concatenate them together with
string_map = [key + " - " + units for key, units in OBS_ST_MAP]
Let me know if that works.
Peter,
Thanks for the hints.
You confirm that I was near, but not yet in touch with the ‘culprits’ causing the error reports.
Aim for the contents of an output json-file and xml-file is a set of labels in combination with values.
Similar to json-file/xml-files coming from e.g. solar-inverters or from ESP8266s with ESPEasy.
Planned application setup:
one nodal point for generation and then distribution of the json/xml-files as input for various applications, both meteo and domotics.
Have tweaked the original script and put the result at Tempest Weather System - Domoticz for further use for anyone interested.
The tweaking description should explain the changes made and why.
I somehow missed all these scripts and ended up reinventing the wheel as well…
I needed JSON as well: GitHub - N2ZLC/tempest_weather_helper: An offline (LAN-only) singleton service that simplifies data-gathering directly from a WeatherFlow Tempest Weather System hub.
Lightning-functionality hard to test, because thunder&flashes only occur unplanned.
Presently I hear&see it happening rather fiercefully, info reported through Weatherflow & WU, no errors in script-running, but also no registration & reporting by the ‘Simple’ UDPListener of the Lightning-events.
Somebody a hint what correction required to the scriptsegment below?**
# Original scriptlines:
# observations = dict(zip(EVT_STRIKE_MAP, data_json['evt']))
# observations['Datetime'] = datetime.datetime.fromtimestamp(observations[('Time Epoch', 'Seconds')])
# For edit of the EVT_STRIKE_MAP variable to just be strings and get rid of the units:
string_map = [key[0] for key in EVT_STRIKE_MAP]
observations = dict(zip(string_map , data_json['evt'][0]))
# Insert dictionary into JSON-file
with open('/home/pi/Tempest_SInfo.json', 'w') as outfile:
json.dump(observations, outfile)
print ('StrikeInfo')
pprint.pprint(observations)
At the end of the following scriptline tried several variations for [‘evt’][0], but without any effect
observations = dict(zip(string_map , data_json['evt'][0]))I’ve been logging the raw UDP data for months now and have never seen an ‘evt’ event, despite have lightning strike what seemed to be fairly close. I am not sure where the data is being pulled from for the app.
Anton, your Tempest will be updated in a few minutes (will be down for a good 10 minutes while doing so, don’t reboot, just let it finish)
Once done I’ll ping you again and from there test your script again.
This should already make sure the data is available, before you go bonkers 
The JSON-file from Weatherflow-server includes Lightning-info having resemblance to actual events.
Therefore ‘in some way’ the sensor-hub must generate a message for that purpose as listed in WeatherFlow Tempest UDP Reference - v143
Have tried to read with Tempest API , but difficulty with ID-insertion.
Any known ID for my Tempest is resulting in either 401-error or 404-error.
[
More, deeper trouble needed for me going bonkers …]
Anton, it was not the hub’s firmware that needed update but the Tempest. It has now been done and your script should now be able to retrieve the lightning data from UDP. And as I’m not a coder I let others help you out with that part 
Eric, Thanks!
;-( But to check have to wait for the following thunderstorm passing (and me being in the vicinity to monitor events).
Added 26May2021:
Puzzle remains:
why no filling of ‘observations’ by these 2 scriptlines?
string_map = [key[0] for key in EVT_STRIKE_MAP]
observations = dict(zip(string_map , data_json['evt'][0]))
Same action for you, your Tempest is updating as we speak, don’t reboot, let it do and in about 10 minutes it should be ok.
Hi @eric,
I believe I experience the same problems with lightnings. Tempest Android app shows them, the Tempest API REST API does too:
{
“status”: {
“status_code”: 0,
“status_message”: “SUCCESS”
},
“device_id”: XXX,
“type”: “obs_st”,
“source”: “cache”,
“summary”: {
(…)
“strike_count_1h”: 21,
“strike_count_3h”: 217,
“precip_total_1h”: 0,
“strike_last_dist”: 36,
“strike_last_epoch”: 1621715802,
(…)
but the mqtt is silent:
{
“battery”: 2.698,
“firmware_revision”: 134,
“illuminance”: 0,
“lightning_strike_avg_distance”: 0,
“lightning_strike_count”: 0,
(…)
}
Could you please have a look? Probably also the firmware update would do the trick.
Yep @tjach
Your Tempest firmware is not good. Not sure I’ll find someone on watch today. Best option is to open a support ticket and ask to update your Tempest do enable the lightning detection via UDP. This way your request isn’t lost in the flow of daily things. They’ll come back to you tomorrow.
Thank you, I’ve just submitted a ticket. Now waiting for the support.