TTN Console stopped showing downlinks for TTIG

Gateways TTIG - The Things Indoor Gateway
1

Hello everyone,

I am asking for help as I am encountering a strange behaviour with my things indoor gateway.
I am working on a project where I would like my sensors to send information once a day to my gateway, and to get confirmation that the message has been received. My testing where doing fine, but suddenly yesterday confirmations for my uplinks stopped appearing in my gateway traffic console. I was correctly seeing them before (no capture, sorry… Didn’t see it coming) but now all I see is the following :
image
(my device is the one with devaddr = 26 01 1E A9, sending confirmed uplinks)

here is my application data for the sensor: (if it helps…)
image

And here is one of the uplink (decoded) received by the gateway :

Assuming base64-encoded packet
gKkeASaAAQACCYrwCqQ33alH

Message Type = Data
  PHYPayload = 80A91E012680010002098AF00AA437DDA947

( PHYPayload = MHDR[1] | MACPayload[..] | MIC[4] )
        MHDR = 80
  MACPayload = A91E012680010002098AF00AA4
         MIC = 37DDA947 (from packet)
             = 37DDA947 (expected, assuming 32 bits frame counter with MSB 0000)

( MACPayload = FHDR | FPort | FRMPayload )
        FHDR = A91E0126800100
       FPort = 02
  FRMPayload = 098AF00AA4 (from packet, encrypted)
             = 4178696D61 (decrypted)

      ( FHDR = DevAddr[4] | FCtrl[1] | FCnt[2] | FOpts[0..15] )
     DevAddr = 26011EA9 (Big Endian)
       FCtrl = 80
        FCnt = 0001 (Big Endian)
       FOpts = 

Message Type = Confirmed Data Up
   Direction = up
        FCnt = 1 (from packet, 16 bits) 
             = 1 (32 bits, assuming MSB 0x0000)
   FCtrl.ACK = false
   FCtrl.ADR = true

No other gateway is in range. My gateway is the only one receiving the uplink message.
I have tried to disable ADR, resetting my sensor, my gateway… But nothing helped. Any help is welcomed! I am a beginner with this kind of technology, I’m sorry if the solution seems obvious to you.

Thanks in advance,
Paul

3

Did you confirm by clicking an uplink in the application/device’s Data page, to see what is in the metadata?

Aside, assuming you’re quite close to your own gateway: using SF12BW125 seems like a lot of overkill. That is the worst available data rate, using about 50 times as much airtime as SF7BW125.

4

Yes, I did. Here is the metadata, in case i misanderstood it :

image
image1008×378 8.21 KB

5

Are all uplinks repeated (so: does the device indeed not receive an ACK for all its confirmed uplinks)? And are all repeated the same number of times? Does clicking an uplink in the gateway’s Traffic page tell you anything?

Seeing only a single retry in the screenshot above could indicate that the retry actually worked, so: that the downlink was actually transmitted and received? The gateway’s Traffic data in TTN Console is just very unreliable. I’ve not seen it before, but maybe it sometimes shows the uplinks but not the downlinks, while they were actually delegated to the gateway just fine. If only some uplinks need a retry, then that could indicate much different issues.

(Aside: don’t ignore my remark about SF12. As shown in your screenshot, that needs almost a full second to transmit the data!)

6

Thanks for your time.

All uplinks are repeated twice, according to my settings, so it seems that no ack are received. I must say that even when my GW was sending downlinks (at least, the console was showing it) my node were repeating all uplinks twice, and I was working on fixing that on my project.

Clicking on an uplink doesn’t help a lot from my opinion, but here is it :

image

I tried many time in the last 24h, like 100 times or so, so it is not something occuring “sometimes” but at least nearly always.

Aside : I am not ignoring your remark, and will configure my node accordingly as soon as I am done eating :wink:

7

You’re right. Most details are in the downlink, in the gateway Traffic, but that’s the one you do not even see…

What if you schedule a dummy application downlink? (That’s in the device’s Overview page in TTN Console.) If that shows as expected, then ACKs for confirmed uplinks might be broken.

Also, subscribing to the MQTT Data API might show more. Use the -v flag to include the topic in the output, and use double-quotes on Windows, like:

mosquitto_sub -v -h eu.thethings.network -t '#' -u my-application-id -P ttn-account-v2.kFd8<redacted>_il4

my-application-id/devices/my-device-id/up {"app_id": ...}
my-application-id/devices/my-device-id/events/down/scheduled {"message": ...}
my-application-id/devices/my-device-id/events/down/sent {"payload": ...}

If things are suddenly fixed when no longer using SF12, then I guess it’s related to your device not adhering to ADR commands that TTN has sent earlier (which it very likely has tried, as the network does not like SF12/DR0 either), which will make TTN stop sending ADR commands. That, of course, should not stop all downlinks, but who knows…

Am I right to assume this is ABP? If yes: did you set the correct EU868 RX2 parameters?

8

I tried it just now, but nothing appears on the gateway traffic.

My node is now emitting using SF7, and is showing accordingly in GW traffic, but still no downlink.

This is indeed ABP, and i set RX2 parameters with 869.525 MHz and SF 9, as requested by the documentation I found.

For the MQTT Data API thing, I have to install things on my computer if I understood correctly, and I cannot do that, as I am working on a locked companie computer…

Thanks again for your help!

9

Is it possible that my GW is actually physically broken and so cannot send LoRa messages anymore?

10

I doubt that the TTN network would be aware of such gateway failure. It seems TTN is not even delegating the downlinks to your gateway.

Do you only have a single device? Maybe delete that in TTN Console and add it again using the same DevAddr, AppSKey and NwkSKey? Oh, bummer, that needs the ttnctl command line interface to force using the existing DevAddr… (Or needs you to reprogram the device as well.)

At least something good comes out of this then: this should convince someone that you should be able to use more than just Windows Notepad and Clock.

11

I already tried to “delete and recreate” a device, using new keys (and devaddr), and indeed I had to reprogram the device. Changed nothing as well…

12

So, to summarize:

  • ABP node with correct settings for RX2.

  • The metadata shows that the uplinks are only received by the TTIG.

  • The TTIG’s Traffic page in TTN Console shows the confirmed uplinks, but since yesterday is no longer showing the network ACK/confirmation downlinks, nor application downlinks.

  • The application’s Data page shows the uplinks too, so TTN clearly accepts the uplinks. (So: no frame counter security problems.)

  • The Data page also shows downlinks, which I assume are the ACKs. However, even though on port 0 (so: even though those are not application downlinks) I wonder if the application/device should even know about those, as an ACK is really a network thing? I assume this part works as expected though.

  • We cannot be sure if the downlinks are really not delegated to the TTIG, as it seems the node did not receive/handle the ACKs before this problem started either. (It already kept repeating the confirmed uplink when the ACK downlinks were still shown in TTN Console before yesterday.)

In the gateway’s Overview page in TTN Console, does the counter for “Transmitted Messages” change?

Though I doubt the problem is with the TTIG, and just in case it helps others debug: what’s the TTIG’s firmware version? What did you select for the gateway’s router and the application’s handler? Likely unrelated: am I right to assume you disabled the device’s frame counter security?

@KrishnaIyerEaswaran2 can you think of anything else one can test?

13

If at all possible, I would try to handle regular application downlinks in your device. Maybe your LoRaWAN stack is just not handling ACKs correctly…? What LoRaWAN stack are you using? Anything that can validate that the TTIG is indeed not being told to transmit the downlink…

14

You are right on the problem description.

No, it doesn’t change.

I disabled the device’s frame counter security, my router is ttn-router-eu.

I am building my own module, something close (hardware wise and software wise) to the RN 2483 module.

I will look for the GW firmware version (but i have the auto update on, so it should be up to date) and add it here tomorrow.

15

image

My Application Data now shows the downlinks I simulated way earlier replacing the “normal” downlinks with no payload I usually see after an uplink. Is this supposed to happen?

Nothing changed in the GW traffic though, still no downlink.

16

Yes, that is normal. Well, except for the delay. Maybe you scheduled multiple downlinks? Then those will have been queued, and for each uplink one downlink would/should have been sent off to the gateway. TTN will then still add the ACK bit to confirm the last uplink to such application downlink.

(Aside: it’s not simulation but actual scheduling/queueing.)

17

I don’t have much to add @arjanvanb’s comprehensive answer.

The Data page also shows downlinks, which I assume are the ACKs. However, even though on port 0 (so: even though those are not application downlinks) I wonder if the application/device should even know about those, as an ACK is really a network thing? I assume this part works as expected though.

Yes indeed. This is Network layer only.

@Maxima: I think indeed MQTT is the best way to debug this. You don’t need to install anything locally. You can use this online MQTT client for testing MQTT Websocket Client.

18

The TTN MQTT broker does not support WebSockets… :slight_smile:

(The above website does not work for me when using the default non-WebSocket MQTT port 1883.)

19

Thanks for all your answers.

I can try any thing you think could help, but I have to confess I don’t know how to use a MQTT client. I am still trying to figure out why i do not get my downlinks, and if I must I will do what Arjan adviced me, but from my personal computer at home. This way I would not be blocked by my companie internet policy.
Anyway, I can try using the online MQTT client, but I don’t know how to use it…

20

Is there any way I could “reset” my gateway on TTN? I mean, I reset it manually, but TTN still show me the total of messages received, transmitted etc, even the ones before the reset, so I didn’t clean TTN’s memory about my gateway, and I have the feeling that it may help. Is that stupid?

21

I don’t know, but: DO NOT DELETE IT.

Aside: don’t focus too much on what you see the gateway Traffic in TTN Console. It’s just very unreliable. Like I wrote earlier: try to ensure your device is indeed not receiving the downlinks.