Hi all!
Since I started my application, I have had missing packets on the Application data interface, so I thought that the packets were really missed in the way node-gateway. The thing is that I noticed that in the gateway traffic interface, the message really reaches the Gateway. So, in summary, all packets are seen on the gateway traffic, but some of them doesn’t appear on the application data interface. What could be the reason? In the image below, you can see that packets 2278, 2280, 2282 and 2283 are not in the app data (right) but they appear on the gateway traffic (left).
Thanks!.
1 Like
Someone having same issues? The problem keeps there. Thanks!!
I have the exact same issue with my application, also when using the command-line interface to monitor the incoming data. Although all data appears to make its way to the http integration that I’ve configured. What could be going on?
Hello everybody, we have the same issue on more then 150 devices.
I can tell that this problem affects different gateways manufacturer and multiple device manufacturer.
The device are all using OTA.
We have tried to connect to the TTN MQTT broker (via command line) and we actually saw that data missing in the application pannel is not forwarded.
We noticed that this issue started about 10 days ago.
Hope someone can help us to understand what is going on.
Regards.
Same issue here! Lot of lost packets in web UI backend despite the packets are correctly sent/received between my nodes and the NS. It seems only an annoing problem related to the web UI.
Alex
I have the same issue with that topic. But I have activated the storage. And yes after refresh the page i see all the data. Even in “life” mode they are not seen. Also not forwarded to my node-red. Only in the storage of TTN backbend.
I have the TTN indoor Gateway and I see the packages is received right. I use the ABP auth.
Well… So, is it a thing that TTN should fix or what? I’ll try the storage and see if I have all data there. Thank you all for the replies, I will keep the topic updated.
I think it is. It looks like one node in an internal cluster could be eating messages…
FYI, as indicated on Dataloss in the backend?, what we did to “solve” the issue is to add a Data Storage integration on the TTN Application and use the REST API on that storage instead of the MQTT interface.
Cool. Just tried to integrate the Data Storage and I have all packets there. Let’s see if they solve this.
Thank you!!
You are right, I enabled the Data Storage Integration and missing data get correctly stored.
But how could you fetch data in an application server? Using the MQTT API makes this operation very easy by subscribing to the correct topic. How can be this acchived using Data Storage? Should I poll the server using the web API once in a while? To me this do not seems a very nice solution. I hope this can be solved soon.
Sounds nice! Just did a quick test with postman, but got only authentification error? Any hint?
Yes, we use influxdb / telegraf and have replaced the MQTT client with a scheduled REST request that queries for the history (with some overlap…) every few minutes. It’s no coincidence I put the word “solved” in quotes 
You can find an authentication key in the application overview tab
Then you’ll have to add it to your request as an Authorization header. Something like the following, IIRC:
“Authorization” = “key ttn-account-v2.blablabla-blabla”
@DeltaQ yes I think also that’s not complete nice to schedule REST requests. But I will do this every 200ms because I want to catch the motion of an PIR Sensor. Here it is important to get each event nearly real time.
LoRaWAN and near real time? You know there is no guarantee any transmission makes it to the network? Let alone all packets (events) sent by a node.
@kersing yes I know that issue of transmitting between node and Gateway. But I think the backend should forward each package that the gateway receives via node red or mqtt and not lost it!
Keep in mind the Semtech based forwarders use UDP which will result in some loss between gateway and back-end.
For packets received by the back-end we all expect them to be delivered to the application, however experience learns this will not always be the case and because the community network does not have any service levels we can only accept the service we are getting.
I hope the V3 back-end when finally deployed to the community network will result in less issues for all of us.
In the mean time you could check the TTI offerings if you need reliability…
1 Like
We have monitored packet losses in TTN very thorowly during the last time (see dataloss-in-the-backend) and the backend currently eats about 25% of our packets.
Regarding the original SPF things are really a bit funny. Semtech has implemented an uplink acknowledge, so the gateway really knows, if a packet was transmitted or not. It just does not use it. Kerlink has implemented a new forwarder they called common packet forwarder (CPF), that can retransfer unacknowledged packets. That helps, if a single packet was lost over UDP.
We found that loss rates are still high. If a packet is lost between gateway and backend, it ist not possible to distinguish if loss was caused by UDP or if the backend was just not listening for some times. So it may be an UDP issue, but also maybe caused by some other part of the system.
This seems to be a different case because the data is available in the database integration. So the back-end has received and processed the data, just not forwarded it to MQTT.
Precisely what we had: packets seen in the gateway traffic, but not in the device traffic.