MultiTech Gateway Sending Data, but not showing up in TTN Console

My MultiTech conduit was added to TTN per the instructions, but I’m not seeing anything in TTN console. I even enabled port forwarding in both directions for UDP port 1700 on my router.
Anybody have any ideas what I can do to troubleshoot this?

I ran sh installer.sh many times and followed the prompts. When I go to ‘gateways’ in my TTN console it says “Status: not connected” and “last seen” is blank. The output says that it’ starting ttn-packet-forwarder, but when I run ps aux | grep lora the only lora process that appears to be running is “/opt/lora/mp_pkt_fwd”

When I run tail -f /var/log/lora-pkt-fwd.log it looks like I’m getting successful logs (see below)

I can even see packets coming in when I press the button on my sensor.

I’ve updated my MultiTech conduit to AEP version 1.7.2, which updated mLinux to version 4.1.3

example logs:
> ##### 2019-03-23 23:13:21 GMT #####

    ### [UPSTREAM] ###
    # RF packets received by concentrator: 0
    # CRC_OK: 0.00%, CRC_FAIL: 0.00%, NO_CRC: 0.00%
    # RF packets forwarded: 0 (0 bytes)
    # PUSH_DATA datagrams sent: 0 (0 bytes)
    # PUSH_DATA acknowledged: 0.00%
    ### [DOWNSTREAM] ###
    # PULL_DATA sent: 0 (0.00% acknowledged)
    # PULL_RESP(onse) datagrams received: 0 (0 bytes)
    # RF packets sent to concentrator: 0 (0 bytes)
    # TX errors: 0
    ### BEACON IS DISABLED!
    ### [JIT] ###
    # INFO: JIT queue contains 0 packets.
    # INFO: JIT queue contains 0 beacons.
    ### GPS IS DISABLED!
    ### [PERFORMANCE] ###
    # Upstream radio packet quality: 0.00%.
    ### [ CONNECTIONS ] ###
    # bridge.us-west.thethings.network: Connected
    # Semtech status report send.
    ##### END #####
    18:13:21  INFO: [TTN] bridge.us-west.thethings.network RTT 80
    18:13:22  INFO: [TTN] send status success for bridge.us-west.thethings.network

Logs showing sensor data:

18:20:42 INFO: [TTN] bridge.us-west.thethings.network RTT 81
18:20:42 INFO: [TTN] send status success for bridge.us-west.thethings.network
18:20:42 INFO: [TTN] send status success for bridge.us-west.thethings.network
18:20:42 INFO: [TTN] send status success for bridge.us-west.thethings.network
lgw_receive:1165: FIFO content: 1 89 1 5 17
lgw_receive:1184: [8 16]
Note: LoRa packet
18:20:56 INFO: [up] TTN lora packet send to server “bridge.us-west.thethings.network”
18:20:57 INFO: Disabling GPS mode for concentrator’s counter…
18:20:57 INFO: host/sx1301 time offset=(1553383048s:473649µs) - drift=39µs
18:20:57 INFO: Enabling GPS mode for concentrator’s counter.

18:21:00 INFO: Disabling GPS mode for concentrator’s counter…

Looks like the gateway is connecting to TTN alright. The triple send status success line is a worry, are there multiple instances of mp_pkt_fwd running or did you accidentally copy-and-paste the same line multiple times?

With regards to the status in the console, check if the sensor data is being received by your application. There have been issues with the console not showing actual gateway status information several times over the last few weeks.

BTW, port forwarding for UDP port 1700 is not required, your gateway is connecting to TTN using TCP.

1 Like

I did not copy and paste it multiple times. If I run sh installer.sh will that spawn new processes? Or should I make sure to stop other processes before running that script again?

My application did receive some data this morning, so that bug in the UI is probably still happening. I was going to post that at one point the gateway was online per the UI, but I didn’t take a screenshot or anything and thought I must have been making it up. Do you know if there is a bug tracker somewhere for this issue?

Makes sense, I thought they would have needed to put that in the setup guide if needed, but when it was showing offline I tried opening those ports to see if it was working.

If you run installer.sh multiple times without reboot you might end up with multiple processes running. Just reboot the gateway to make sure that isn’t the case. (Or run ps to check for multiple mp_pkt_fwd instances)

There is no bug tracker for the issue as TTN isn’t actively working on solving it. The available time is spent building V3 of the stack and only very urgent V2 issues are being fixed.

Thank you for the help! I think that fixed most of my issues!
It looks like there was 4 of them running:

admin@mtcdt:~# ps aux | grep mp
admin 23535 3.0 0.5 46636 1480 ? Sl Mar23 111:06 /opt/lora/mp_pkt_fwd -c /var/config/lora -l /var/log/lora-pkt-fwd.log
admin 24480 3.0 0.5 46636 1492 ? Sl Mar23 111:19 /opt/lora/mp_pkt_fwd -c /var/config/lora -l /var/log/lora-pkt-fwd.log
admin 27127 2.9 0.5 46636 1492 ? Sl Mar22 146:34 /opt/lora/mp_pkt_fwd -c /var/config/lora -l /var/log/lora-pkt-fwd.log
admin 27166 0.0 0.2 3092 560 pts/0 S+ 06:33 0:00 grep mp
admin 29347 3.0 0.5 46636 1492 ? Sl Mar23 110:53 /opt/lora/mp_pkt_fwd -c /var/config/lora -l /var/log/lora-pkt-fwd.log

I tried to just sudo kill the processes, but apparently admin doesn’t have sudo rights on the conduit? Restarting the gateway seemed to work. Now there is only one running.

admin@mtcdt:~# ps aux | grep mp
admin 2522 2.4 0.5 46636 1436 ? Sl 06:37 0:02 /opt/lora/mp_pkt_fwd -c /var/config/lora -l /var/log/lora-pkt-fwd.log
admin 2915 0.0 0.2 3092 560 pts/0 S+ 06:39 0:00 grep mp

Now I see packets coming through to my application, and to the 3rd party server that I have set up with an http integration!

However, I’m also seeing downlink failure logs, which make it look like the downlink is too late to be sent down to the end device. I see my device trying to join, and the device gets the join accept. But then the device sends up it’s data and does not get the confirmation downlink. Therefore the device sends the same data payload back up again, resulting in duplicate data.

06:38:44 INFO: [up] TTN lora packet send to server “bridge.us-west.thethings.network”
06:38:44 INFO: [down] TTN received downlink with payload
06:38:44 INFO: [TTN] downlink 22 bytes
src/jitqueue.c:305:jit_enqueue(): ERROR: Packet REJECTED, already too late to send it (current=76923431, packet=76698556, type=0)
06:38:44 ERROR: [down] Packet REJECTED (jit error=1 Too late to send this packet)
lgw_receive:1165: FIFO content: 1 a8 0 5 15
lgw_receive:1184: [2 17]
Note: LoRa packet
06:38:49 INFO: [up] TTN lora packet send to server “bridge.us-west.thethings.network”
06:38:50 INFO: [down] TTN received downlink with payload
06:38:50 INFO: [TTN] downlink 22 bytes
src/jitqueue.c:305:jit_enqueue(): ERROR: Packet REJECTED, already too late to send it (current=82370192, packet=81925340, type=0)
06:38:50 ERROR: [down] Packet REJECTED (jit error=1 Too late to send this packet)
lgw_receive:1165: FIFO content: 2 cd 0 7 15
lgw_receive:1184: [3 17]

This results in duplicate data coming up:

First of all, don’t use confirmed uplink if not absolutely required. A sending gateway can not listen for any transmissions.

Looking at the time stamps in the rejection message the gateway is right to refuse them. The packet time stamps demand transmission in the past which clearly is not possible, so queuing them is not useful.

The # in the commandline prompt suggests the user is has root uid, sudo is not needed.

admin@mtcdt:~#

1 Like

Is there anything I can do to help the downlinks come back down faster? Are there any configuration settings I can change in TTN or in the gateway?

I understand that it is overkill to have confirmed uplinks all the time but there is only one sensor right now within range of my gsyew so I don’t think I’m in risk of saturating my bandwidth or gateway because I don’t have a full duplex gateway.