Data missing from TTN data storage but displaying in live data

Hi,

The image is the live data of TTN. I have highlighted 04:06:05 because that is the data message missing in the data storage. The second image below the first is the data storage using the API which does not feature everything from the live data. Can you please advise if this is expected or we are doing something wrong? Your quick response would be appreciated. Thank you.
Screenshot 2022-06-16 094901
Screenshot 2022-06-16 at 14.52.03

Regards

Angad

These things happen, in the context of a highly complicated free to use service with no SLA and the data storage being a thing of last resort, not a primary data feed.

With lots of data being piped in all sorts of directions it does seem likely that occasionally a pipe will get backlogged and perhaps overflow.

For research / curiosity I have a couple of devices with both a web hook & data storage and some logging at this end so I can see what may not get through. The most typical issue I see with data storage is a delay (a minute or two) in the records making it in to the database as they are not committed individually but in batches.

If you need to be sure of getting your data in the context of using TTN, I’d have a web hook and data storage and if there are apparent gaps in the web hook, pull data from data storage. If both are missing and you can’t live with that, as the expected overall loss is 10% of uplinks (mostly due to radio issues), then either have some sort of rolling uplink window or replay facility or use a paid for instance or another communications method.