Can anyone guess what is wrong here?
Testing a Heltec board : mosquitto_sub delivers the raw JSON that is delivered by TTN, but in the data with counter=4, TTN does not deliver the payload fields, hence the JQ processing throws an error.
Thank you,
(yes I know the 2 minutes dutycycle is not TTN fair policy)
I’ve seen this reported before, but cannot find it. (I probably saw it in Slack, which doesn’t provide much history in the free account with that many messages. I can only find references to “Internal error: Interrupted javascript execution for Decoder after 1.326067817s”, which was due to limited resources in Meshed’s Australia servers.)
If this happened more than once: please subscribe to the MQTT error events to see if that tells you something?
Thank you, will try that and see if it happens again.
Just for future reference, while importing some old data as received through MQTT in the EU region, I ran into this too:
For February 2020, when you encountered this, I did not run into this for 6,028 messages.
In my data, the message closest to your erroneous one at 2020-02-10T20:12:48, was received at 2020-02-10T20:11:10.204625736Z (and was handled just fine).
For March 2020, this happened for 3 out of 6,714 messages (about 0.04%, or 0.0004).
Pasting some of the payloads into the decoder for testing worked just fine, so the actual data was not the culprit for those.
For April 2020, so far all were fine.
For a set of 127,569 messages since 2018, 66 (about 0.05%) did not did not get their payload_fields populated.
Though it’s surely nice to see the fields in TTN Console, in most cases I’d simply recommend doing all parsing/decoding in one’s own application code if possible. (And always keep the raw data as well, to allow for fixing decoding errors after the fact.)
Last week I happened to be using a Decoder for some test and noticed a timeout for 0.5% of the messages (6 out of 1,238 = 0.005), during the 15 hours of that test:
Unable to decode payload fields: Internal error: Interrupted javascript execution for Decoder after 120.862081ms
Unable to decode payload fields: Internal error: Interrupted javascript execution for Decoder after 153.496916ms
Note that this also delays delivery of the message through MQTT or HTTP with the timeout shown in the error.
I’m currently not using it, so I don’t know if that was a temporary glitch or not. But when I reported in the #ops channel in Slack it seemed this is quite common, every now and then:
I had a similar issue some time ago, not getting decoded payload through the http integration.
I then programmed my own payload decoder but I never activated it…
Haven’t had missing decoded payload since a while…
@jpmeijers 2020-08-23 02:38 PM
I’ve seen this happening on some of my data too a while back. Not critical for most cases when one decode your payload yourself, but ttnmapper.org depends on this to work.
I’ve seen it recently in an MQTT feed but didn’t record numbers as I was testing things.
To reduce load on the TTN servers and reduce any non-delivery issues, I’ve removed the decoders from live systems that were built when it seemed a good idea. Now-a-days I decode everything at my end.