TTN Mapper: Data present but heatmap is not shown

[sniper4uonline (Sniper)]

SNR close to 10. 8,9 or 10.

RSSI from -70 to -25

Experiment is done close to gateway. So lower RSSI is simulated with an open antenna port. I want the experiment to be realistic RSSI so ok to lose an node in the process.

But RSSI with antenna can go upto -25. Without usually -70. Both same SNR in the between 8-10

If it was ADR NS should have asked me to go to lowest air time.

[Jeff-UK]

Just move node say 10m away with an absorber like a wall in the way and you should quickly drop below -50…without risk of killing a node via no ant…

[sniper4uonline (Sniper)]

Yes already doing it. But no antenna only to check if anything is wrong because of over strength.

[Johan_Scheepers]

@sniper4uonline

But no antenna only to check if anything is wrong

With no antenna you could push the reflected power thorough the roof for the node, so you are still causing the same issues with high RSSI.

[LoRaTracker]

@sniper4uonline

But no antenna only to check if anything is wrong because of over strength.

And a good way to destroy a transmittter, so it goes very low to no power output.

[sniper4uonline (Sniper)]

@LoRaTracker

And a good way to destroy a transmittter, so it goes very low to no power output.

My bad. Ok I will never do that. Even if so I won’t say it

@Johan_Scheepers

With no antenna you could push the reflected power thorough the roof for the node, so you are still causing the same issues with high RSSI.

Every tranceiver should have TR switch or a duplexer or diplexer. So high RSSI at RF front receive front with open antenna port is a bad design.

Sorry to put in a unnecessary deviation input. Let’s focus on this. Either ther is something I am doing wrong or Network Server is not happy with my device ID.

Does the network server has some code which says, if there is only one node joined or like few nodes joined well below capacity, go with SF12?

@sniper4uonline

I have one more input.

1. I send a payload when node is switched on. Frame count is 0. SF7
2. Sever starts request for SF12.
3. I send back Mac command saying SF12 is not acceptable.
4. Server now ask ok change to SF7.
5. I send back. Ok SF7 is accepted. 0x03 0x07. If I remember right
6. But NS again ask me to get to SF7 for all next uplinks. (Last uplink already had SF7 confirmation and the last uplink was in SF7)

NB: The same server stops the request if I accepted SF12 and started uplinks in SF12

[kersing (Jac Kersing)]

@sniper4uonline

Does the network server has some code which says, if there is only one node joined or like few nodes joined well below capacity, go with SF12?

No! There is never one node joined to TTN. And nodes don’t join a gateway (that is WiFi, not LoRaWAN)

[sniper4uonline (Sniper)]

Ok… so that is clear.

[sniper4uonline (Sniper)]

@descartes

… the LoRaWAN Alliance requires network operators to limit devices that run on SF11 or 12 but I’ve not figured out if or how that may be implemented

Click to see the JSON

[
  {
    "name": "gs.up.receive",
    "time": "2022-02-25T17:50:38.095048454Z",
    "identifiers": [
      {
        "gateway_ids": {
          "gateway_id": "openwaytvm"
        }
      },
      {
        "gateway_ids": {
          "gateway_id": "openwaytvm",
          "eui": "B827EBFFFEC085CA"
        }
      }
    ],
    "data": {
      "@type": "type.googleapis.com/ttn.lorawan.v3.UplinkMessage",
      "raw_payload": "QOC2CyaCAQADBzuALvQ=",
      "payload": {
        "m_hdr": {
          "m_type": "UNCONFIRMED_UP"
        },
        "mic": "O4Au9A==",
        "mac_payload": {
          "f_hdr": {
            "dev_addr": "260BB6E0",
            "f_ctrl": {
              "adr": true
            },
            "f_cnt": 1,
            "f_opts": "Awc="
          }
        }
      },
      "settings": {
        "data_rate": {
          "lora": {
            "bandwidth": 125000,
            "spreading_factor": 12
          }
        },
        "coding_rate": "4/5",
        "frequency": "866185000",
        "timestamp": 2678192836,
        "time": "2022-02-25T17:50:38.081756Z"
      },
      "rx_metadata": [
        {
          "gateway_ids": {
            "gateway_id": "openwaytvm",
            "eui": "B827EBFFFEC085CA"
          },
          "time": "2022-02-25T17:50:38.081756Z",
          "timestamp": 2678192836,
          "rssi": -57,
          "channel_rssi": -57,
          "snr": 8,
          "location": {
            "latitude": 8.527403715667655,
            "longitude": 76.94841653108598,
            "altitude": 30,
            "source": "SOURCE_REGISTRY"
          },
          "uplink_token": "ChgKFgoKb3BlbndheXR2bRIIuCfr//7AhcoQxPWH/QkaCwjureSQBhC5+54tIKCb7oX5jhUqCwjureSQBhDg/v0m",
          "channel_index": 4
        }
      ],
      "received_at": "2022-02-25T17:50:38.094879161Z",
      "correlation_ids": [
        "gs:conn:01FWP7GZ0T4G6QHPSW3QZC9J3Z",
        "gs:uplink:01FWS034GFZJVSVECF9A09ADXS"
      ]
    },
    "correlation_ids": [
      "gs:conn:01FWP7GZ0T4G6QHPSW3QZC9J3Z",
      "gs:uplink:01FWS034GFZJVSVECF9A09ADXS"
    ],
    "origin": "ip-10-100-5-46.eu-west-1.compute.internal",
    "context": {
      "tenant-id": "CgN0dG4="
    },
    "visibility": {
      "rights": [
        "RIGHT_GATEWAY_TRAFFIC_READ",
        "RIGHT_GATEWAY_TRAFFIC_READ"
      ]
    },
    "unique_id": "01FWS034GFJ884SKY5TQYE45X9"
  },
  {
    "name": "gs.down.send",
    "time": "2022-02-25T17:50:31.684781610Z",
    "identifiers": [
      {
        "gateway_ids": {
          "gateway_id": "openwaytvm"
        }
      },
      {
        "gateway_ids": {
          "gateway_id": "openwaytvm",
          "eui": "B827EBFFFEC085CA"
        }
      }
    ],
    "data": {
      "@type": "type.googleapis.com/ttn.lorawan.v3.DownlinkMessage",
      "raw_payload": "YOC2CyYFAAADAP8AARIIJ9s=",
      "request": {
        "downlink_paths": [
          {
            "uplink_token": "ChgKFgoKb3BlbndheXR2bRIIuCfr//7AhcoQpPzo+QkaDAjnreSQBhD3kMeDASCg+Yrc344VKgsI563kkAYQwNP5fA=="
          }
        ],
        "rx1_delay": 5,
        "rx1_data_rate": {
          "lora": {
            "bandwidth": 125000,
            "spreading_factor": 7
          }
        },
        "rx1_frequency": "866785000",
        "rx2_data_rate": {
          "lora": {
            "bandwidth": 125000,
            "spreading_factor": 10
          }
        },
        "rx2_frequency": "866550000",
        "priority": "HIGHEST",
        "frequency_plan_id": "IN_865_867"
      },
      "correlation_ids": [
        "gs:conn:01FWP7GZ0T4G6QHPSW3QZC9J3Z",
        "gs:up:host:01FWP7GZ10WKNBN914165D09SF",
        "gs:uplink:01FWS02XVCRWBYYW9BHWCX8MP7",
        "ns:downlink:01FWS02Y84N87D0SN132D7TW98",
        "ns:uplink:01FWS02XVDQP2V1NZBZEW2FG5B",
        "rpc:/ttn.lorawan.v3.GsNs/HandleUplink:01FWS02XVD6981687T4Z9FPTXD",
        "gs:conn:01FWP7GZ0T4G6QHPSW3QZC9J3Z",
        "rpc:/ttn.lorawan.v3.NsGs/ScheduleDownlink:01FWS02Y84E5NRGR0XRCQCPP7P"
      ]
    },
    "correlation_ids": [
      "gs:conn:01FWP7GZ0T4G6QHPSW3QZC9J3Z",
      "rpc:/ttn.lorawan.v3.NsGs/ScheduleDownlink:01FWS02Y84E5NRGR0XRCQCPP7P"
    ],
    "origin": "ip-10-100-5-46.eu-west-1.compute.internal",
    "context": {
      "tenant-id": "CgN0dG4="
    },
    "visibility": {
      "rights": [
        "RIGHT_GATEWAY_TRAFFIC_READ",
        "RIGHT_GATEWAY_TRAFFIC_READ"
      ]
    },
    "unique_id": "01FWS02Y84Z50X2A2RPA8GSDHK"
  },
  {
    "name": "gs.up.receive",
    "time": "2022-02-25T17:50:31.276069Z",
    "identifiers": [
      {
        "gateway_ids": {
          "gateway_id": "openwaytvm"
        }
      },
      {
        "gateway_ids": {
          "gateway_id": "openwaytvm",
          "eui": "B827EBFFFEC085CA"
        }
      }
    ],
    "data": {
      "@type": "type.googleapis.com/ttn.lorawan.v3.UplinkMessage",
      "raw_payload": "QOC2CyaAAAABnG0SJix1SA7Nw1mf2mca",
      "payload": {
        "m_hdr": {
          "m_type": "UNCONFIRMED_UP"
        },
        "mic": "n9pnGg==",
        "mac_payload": {
          "f_hdr": {
            "dev_addr": "260BB6E0",
            "f_ctrl": {
              "adr": true
            }
          },
          "f_port": 1,
          "frm_payload": "nG0SJix1SA7Nw1k="
        }
      },
      "settings": {
        "data_rate": {
          "lora": {
            "bandwidth": 125000,
            "spreading_factor": 7
          }
        },
        "coding_rate": "4/5",
        "frequency": "866785000",
        "timestamp": 2671394340,
        "time": "2022-02-25T17:50:31.262040Z"
      },
      "rx_metadata": [
        {
          "gateway_ids": {
            "gateway_id": "openwaytvm",
            "eui": "B827EBFFFEC085CA"
          },
          "time": "2022-02-25T17:50:31.262040Z",
          "timestamp": 2671394340,
          "rssi": -61,
          "channel_rssi": -61,
          "snr": 9.2,
          "location": {
            "latitude": 8.527403715667655,
            "longitude": 76.94841653108598,
            "altitude": 30,
            "source": "SOURCE_REGISTRY"
          },
          "uplink_token": "ChgKFgoKb3BlbndheXR2bRIIuCfr//7AhcoQpPzo+QkaDAjnreSQBhD3kMeDASCg+Yrc344VKgsI563kkAYQwNP5fA==",
          "channel_index": 7
        }
      ],
      "received_at": "2022-02-25T17:50:31.275892343Z",
      "correlation_ids": [
        "gs:conn:01FWP7GZ0T4G6QHPSW3QZC9J3Z",
        "gs:uplink:01FWS02XVCRWBYYW9BHWCX8MP7"
      ]
    },
    "correlation_ids": [
      "gs:conn:01FWP7GZ0T4G6QHPSW3QZC9J3Z",
      "gs:uplink:01FWS02XVCRWBYYW9BHWCX8MP7"
    ],
    "origin": "ip-10-100-5-46.eu-west-1.compute.internal",
    "context": {
      "tenant-id": "CgN0dG4="
    },
    "visibility": {
      "rights": [
        "RIGHT_GATEWAY_TRAFFIC_READ",
        "RIGHT_GATEWAY_TRAFFIC_READ"
      ]
    },
    "unique_id": "01FWS02XVCPAJFS3JD59FCJFPH"
  }
]

The above JSON shows the first three communication between node and server. First at bottom: ABP node first payload, then above that: Server Reply and the top most: the ABP node reply.

The server clearly asks to go to ADR 0 (SF12) and power level 0 and the node does.

Click to see first payload decoded

Assuming base64-encoded packet
QOC2CyaAAAABnG0SJix1SA7Nw1mf2mca

Message Type = Data
  PHYPayload = 40E0B60B26800000019C6D12262C75480ECDC3599FDA671A

( PHYPayload = MHDR[1] | MACPayload[..] | MIC[4] )
        MHDR = 40
  MACPayload = E0B60B26800000019C6D12262C75480ECDC359
         MIC = 9FDA671A (from packet)
             = 9FDA671A (expected, assuming 32 bits frame counter with MSB 0000)

( MACPayload = FHDR | FPort | FRMPayload )
        FHDR = E0B60B26800000
       FPort = 01
  FRMPayload = 9C6D12262C75480ECDC359 (from packet, encrypted)
             = 8B327B50BAB421328B37E2 (decrypted)

      ( FHDR = DevAddr[4] | FCtrl[1] | FCnt[2] | FOpts[0..15] )
     DevAddr = 260BB6E0 (Big Endian)
       FCtrl = 80
        FCnt = 0000 (Big Endian)
       FOpts = 

Message Type = Unconfirmed Data Up
   Direction = up
        FCnt = 0 (from packet, 16 bits) 
             = 0 (32 bits, assuming MSB 0x0000)
   FCtrl.ACK = false
   FCtrl.ADR = true

Click to see Server Request decoded

Assuming base64-encoded packet
YOC2CyYFAAADAP8AARIIJ9s=

Message Type = Data
  PHYPayload = 60E0B60B260500000300FF0001120827DB

( PHYPayload = MHDR[1] | MACPayload[..] | MIC[4] )
        MHDR = 60
  MACPayload = E0B60B260500000300FF0001
         MIC = 120827DB

( MACPayload = FHDR | FPort | FRMPayload )
        FHDR = E0B60B260500000300FF0001
       FPort = 
  FRMPayload = 

      ( FHDR = DevAddr[4] | FCtrl[1] | FCnt[2] | FOpts[0..15] )
     DevAddr = 260BB6E0 (Big Endian)
       FCtrl = 05
        FCnt = 0000 (Big Endian)
       FOpts = 0300FF0001

Message Type = Unconfirmed Data Down
   Direction = down
        FCnt = 0
   FCtrl.ACK = false
   FCtrl.ADR = false

Click to see Node reply to server decoded

Assuming base64-encoded packet
QOC2CyaCAQADBzuALvQ=

Message Type = Data
  PHYPayload = 40E0B60B2682010003073B802EF4

( PHYPayload = MHDR[1] | MACPayload[..] | MIC[4] )
        MHDR = 40
  MACPayload = E0B60B268201000307
         MIC = 3B802EF4 (from packet)
             = 3B802EF4 (expected, assuming 32 bits frame counter with MSB 0000)

( MACPayload = FHDR | FPort | FRMPayload )
        FHDR = E0B60B268201000307
       FPort = 
  FRMPayload = 

      ( FHDR = DevAddr[4] | FCtrl[1] | FCnt[2] | FOpts[0..15] )
     DevAddr = 260BB6E0 (Big Endian)
       FCtrl = 82
        FCnt = 0001 (Big Endian)
       FOpts = 0307

Message Type = Unconfirmed Data Up
   Direction = up
        FCnt = 1 (from packet, 16 bits) 
             = 1 (32 bits, assuming MSB 0x0000)
   FCtrl.ACK = false
   FCtrl.ADR = true

Clearly, the server asks for ADR 0 and PWR 0 in the opts field (0x03) which is SF12 and the next uplink is in SF12 with 0x03 0x07 in opts saying. “yes accepted the power and data rate”

[Jeff-UK]

@sniper4uonline

The above JSON shows the first three communication between node and server. First at bottom: ABP node first payload, then above that: Server Reply and the top most: the ABP node reply.

Loosing track of the thread at this point but…When you registered the node - did you register it as an ABP device - or does server think it is OTAA?

[sniper4uonline (Sniper)]

Its ABP so no join process. The server only tells about ADR PWR and RX delay.

[descartes (Nick McCloud)]

Just so everyone else can catch up, the LMIC code has been altered to try to block the SF12.

I’ll spin up an LMIC ABP over the weekend and see what happens.

[Jeff-UK]

@sniper4uonline

LoRaWAN MAC and PHY:

Current: MAC V1.0.3 and PHY V1.0.3 REV A

Activation mode:

ABP


History:

v2 gateway was running on the same version. Node was hardcoded to send

Yes you mentioned that early on and I can see a TTN DevAddrr called out - which I assume isnt changing as no Join to reset this

@sniper4uonline

"dev_addr": "260BB6E0"

Even so at this point am looking for silly gotchas that might explain…can you confirm you have ticked ABP in device registration…

[sniper4uonline (Sniper)]

Yes in console it is ABP because of which i get a NW key and APP key and the join setting in general setting is missing, which confirms it is ABP

[sniper4uonline (Sniper)]

@sniper4uonline

260BB6E0

One more update.

1. Tried to make ADRACKReq instead of ADR true from node. But server agains asks for SF12.
2. Tried to make FCtrl = 0. But still, the server wants the node to go to SF12.

Payload from Node before the server request to SF12 is given below:

Click to decoded payload of first node message

Assuming base64-encoded packet
QOC2CyYAAQABblHQvsb3fDp3yxS6O+/q

Message Type = Data
  PHYPayload = 40E0B60B26000100016E51D0BEC6F77C3A77CB14BA3BEFEA

( PHYPayload = MHDR[1] | MACPayload[..] | MIC[4] )
        MHDR = 40
  MACPayload = E0B60B26000100016E51D0BEC6F77C3A77CB14
         MIC = BA3BEFEA (from packet)
             = BA3BEFEA (expected, assuming 32 bits frame counter with MSB 0000)

( MACPayload = FHDR | FPort | FRMPayload )
        FHDR = E0B60B26000100
       FPort = 01
  FRMPayload = 6E51D0BEC6F77C3A77CB14 (from packet, encrypted)
             = 0B144D085F4803C615F172 (decrypted)

      ( FHDR = DevAddr[4] | FCtrl[1] | FCnt[2] | FOpts[0..15] )
     DevAddr = 260BB6E0 (Big Endian)
       FCtrl = 00
        FCnt = 0001 (Big Endian)
       FOpts = 

Message Type = Unconfirmed Data Up
   Direction = up
        FCnt = 1 (from packet, 16 bits) 
             = 1 (32 bits, assuming MSB 0x0000)
   FCtrl.ACK = false
   FCtrl.ADR = false

Message Type = Unconfirmed Data Up
   Direction = up
        FCnt = 1 (from packet, 16 bits) 
             = 1 (32 bits, assuming MSB 0x0000)
   FCtrl.ACK = false
   FCtrl.ADR = false

[sniper4uonline (Sniper)]

@Jeff-UK

Even so at this point am looking for silly gotchas that might explain…can you confirm you have ticked ABP in device registration…

[sniper4uonline (Sniper)]

Just now found something which helps me to be in SF7 itself without going to SF12.

Use adaptive data rate (ADR) - check this in console.

This enables ADR but now the request from the server is only for SF7 and there is also status request along with ADR request when ADR enabled in console.

But now the server is accepting the SF7 confirmation mac commands and server doesn’t keep asking for same ADR again as seen when it was disabled.

So for my GPS node the problem is solved in near range locations. (But this would let server to change data rate when the signal is low. So still I would be forced to go to higher airtime)

[wolfp (Wolfgang)]

I can confirm this behaviour. I had similar problems with my TTGO T-beam (lmic 4.1.1, ABP) for TTNmapper. After activating “ADR” in the console my T-beam seems to be stopped from switching from SF7 to SF12.

[wolfp (Wolfgang)]

btw: if my T-beam is not heard by a gateway when switching it on, it uses only the 3 basic-frequencies. If a gateway is reachable, the other frequencies are send by TTS and all frequencies are used.

[descartes (Nick McCloud)]

@wolfp

I can confirm this behaviour. I had similar problems with my TTGO T-beam (lmic 4.1.1, ABP) for TTNmapper. After activating “ADR” in the console my T-beam seems to be stopped from switching from SF7 to SF12.

Excellent stuff, we can now enter a scientific realm as we have two observations to form a working hypothesis from which we can perform a test.

I’ll work on the basis that LMIC & the NS are getting in a muddle (OK, maybe not so scientific, but I’m not accusing either party of being a fault). To evaluate that I’ll spin up my ATmega4808 with RFM95 which has enough space to allow me to use an untouched LMIC (mine has things turned on & off) but with printf and full debugging.

As a reference / point of comparison, I’ll use the same device on a STM32 B-L072Z aka Murata ABZ board using LoRaMAC-node.

I can decrypt/decoded stuff using the Runkit site, but does anyone know of a tool to decode the FOpts?

@wolfp

it uses only the 3 basic-frequencies

This is normal unless you have explicitly turned on the channels in LMIC.

It’s normal for the NS to send a channel list unless you have them configured on the device, then the NS regards this as a configuration the device has and doesn’t send it.

Because of the early excitement of the transition to TTS and at the bidding of our overlords, all my ABP devices have a full configuration on the server and I haven’t really used ABP since, if only because after a bun-fight with nonces on OTAA on devices with LoRaMAC-node, I’ve got the tools to reset the heck out of things - although there is a button on the console for that now.

I’ll try an ABP device with various sets of settings to see what occurs.