TTN Mapper with T-Beam V1.2

descartes · December 22, 2024, 2:35pm

You have shown screen shots from TTN & your gateway so you definitely have more than the serial monitor! This means you have access to TTN so you can look at the console on the page for the device that you created.

The AppKey doesn’t care which way round it goes but it is simpler if it’s left as MSB.

Feel free to add screenshots but it’s harder to read on mobile devices - like now - so maybe I’ll wait until I’m back at a desktop in January!

I have no idea how a JR ends up with the mangled Dev & Join EUIs - I’ve used LMIC on & off for years so I’ve seen a fair amount of interesting issues but never this.

Maybe @kersing or @Jeff-UK can advise as they are gateway masters.

kersing · December 22, 2024, 3:05pm

Please remove this when using OTAA.

Unless I’m mistaken LMIC (specifically older versions) are LoRaWAN 1.0.2, not 1.0.3.

Not entirely true. The main thing is you use valid values that don’t clash with ranges someone paid good money for. If your gateway runs reasonably recent software (not > 4 years old) you can safely use the suggested all zeros value.

I have never seen a JoinEUI being scrambled between the gateway and TTN in this way before. Can you check if there is a software update for your gateway?

vorderwaeldler · December 22, 2024, 3:13pm

No hurry.
We’re tinkering here, not on the run

The package looks like this on the MikroTik GW:
Type: RX
Time: Dec/22/2024 15:57:57
Gateway ID: REMOVED-FOR-PRIVACY
Message Type: Join-request
Major Version: LoRaWAN R1
Join EUI: D7 D2 59 C2 A5 C3 3C 42
Dev EUI: FF 79 99 DF 42 3C 43 C7
Dev Nonce: 15657
Freq (MHz): 868.100
Modulation: LoRa
Bandwith: 125 kHz
Datarate: SF7
Coderate: 4/5
IF Chain: 0
CRC Status: OK
Counter (us): 1172420260
RF Chain: 1
RSSI (dB): -101.00
SNR: 6.25
CRC: 5702
Size: 23
Payload: AEI8w6XCWdLXx0M8Qt+Zef8pPbBEV80=

Here are the event details of a JR in the TTN Console in JSON:

{
  "name": "gs.up.receive",
  "time": "2024-12-22T14:57:57.703236185Z",
  "identifiers": [
    {
      "gateway_ids": {
        "gateway_id": "REMOVED-FOR-PRIVACY",
        "eui": "REMOTE-FOR-PRIVACY"
      }
    }
  ],
  "data": {
    "@type": "type.googleapis.com/ttn.lorawan.v3.GatewayUplinkMessage",
    "message": {
      "raw_payload": "AEI8w6X/////x0M8Qt+Zef8pPbBEV80=",
      "payload": {
        "m_hdr": {},
        "mic": "sERXzQ==",
        "join_request_payload": {
          "join_eui": "FFFFFFFFA5C33C42",
          "dev_eui": "FF7999DF423C43C7",
          "dev_nonce": "3D29"
        }
      },
      "settings": {
        "data_rate": {
          "lora": {
            "bandwidth": 125000,
            "spreading_factor": 7,
            "coding_rate": "4/5"
          }
        },
        "frequency": "868100000",
        "timestamp": 1172420260
      },
      "rx_metadata": [
        {
          "gateway_ids": {
            "gateway_id": "REMOVED-FOR-PRIVACY",
            "eui": "REMOVED-FOR-PRIVACY"
          },
          "timestamp": 1172420260,
          "rssi": -101,
          "channel_rssi": -101,
          "snr": 6.25,
          "location": {
            "latitude": REMOVED-FOR-PRIVACY,
            "longitude": REMOVED-FOR-PRIVACY,
            "altitude": REMOVED-FOR-PRIVACY,
            "source": "SOURCE_REGISTRY"
          },
          "uplink_token": "CiQKIgoWc2Nob25hY2gtMDEtcmJ3YXByLTJuZBIIUDA1QUpIR1AQpO2GrwQaDAj10aC7BhDHgKHPAiCg4cXNjyI=",
          "received_at": "2024-12-22T14:57:57.703086663Z"
        }
      ],
      "received_at": "2024-12-22T14:57:57.703086663Z",
      "correlation_ids": [
        "gs:uplink:01JFQDZZY73NXEVJD94FGP1QQT"
      ]
    },
    "band_id": "EU_863_870"
  },
  "correlation_ids": [
    "gs:uplink:01JFQDZZY73NXEVJD94FGP1QQT"
  ],
  "origin": "ip-10-100-12-7.eu-west-1.compute.internal",
  "context": {
    "tenant-id": "CgN0dG4="
  },
  "visibility": {
    "rights": [
      "RIGHT_GATEWAY_TRAFFIC_READ"
    ]
  },
  "unique_id": "01JFQDZZY734CWF0EBB5FSY074"
}

What differs here are the following things:

JoinEUI
Payload
Dev-Nonce

vorderwaeldler · December 22, 2024, 3:19pm

I have commented out the line and uploaded it again.
Unfortunately, this does not change the behavior.
The packages continue to arrive broken.

I have installed the latest MikroTik OS (RBwAPR-2nD).
I have also just installed the update from 7.16.1 to 7.16.2.

descartes · December 22, 2024, 3:40pm

Please! ANY other users device that is heard by your gateway will include the details you’ve removed in their uplink meta data. And join requests are sent in plain text so will appear in any gateway logs that hear it. You’ve taken time to redact some information but then leave other info in. From experience this means there is a likely hood that there are other things being altered that need to be left alone.

The original (German) article links to version 3.2.0 of LMIC which is old but should be co-operating. At that time it reached LW v1.0.3 so that’s OK.

You could try RadioLib which has first class support for T-Beam AND has been shown to be completely compliant for all LoRa frequencies (FSK coming soon apparently) - this will sanitise / test your board & gateway combo. At RadioLib Development Organization · GitHub you can pickup the board definitions library and at a later date use the the persistence for ESP32.

After that you can patch over the GNSS elements or you can go back to LMIC.

descartes · December 22, 2024, 3:42pm

It has just dawned on me that both of your EUI’s are invalid - they should have bit 0 set to 0 and for local addresses, bit 1 set to 1. So as MSB, the second ‘digit’ can only ever be an even number.

See: DevEUI for non-hardware assigned values - #23 by terrillmoore which is why Random EUI or Key generator | descartes exists.

Also, LMIC has been shown to NOT cope at all with an all zero JoinEUI.

Jeff-UK · December 22, 2024, 4:21pm

Hummmm, never run a device (GW or node) without the antenna attached!

If you have done this there is a risk that you have damaged the front end of the device. Maybe not a catastrophic failure but potentially degraded performance(*). If the node receiver is impaired it may not be hearing any join ack correctly or without corruption which could also be a problem, in addition to all the other points called out above. Also if all zero JoinEUI a problem then people often find a simple 00…01 solves

(*) the other risk being an impairment that doesnt show straight away but whilch leads to an early life failure/degraded lifetime - you get working initially bit then fails days, week, years later - but earlier than ‘should’ be the case. Do you have another known good device that hasntt been run (powered up) Ant-less? If so move to that until you have all working correctly…

vorderwaeldler · December 22, 2024, 5:36pm

@descartes I have just generated a new JoinEUI via the site.
I have also created a new DevEUI and AppKey via the TTN console.
I loaded the JoinEUI and DevEUI back onto the T-Beam in LSB format and the AppKey in MSB format.

Now it looks even stranger.
In the TTN console, the JoinEUI and DevEUI now start with FF FF FF FF FF.

        "join_request_payload": {
          "join_eui": "FFFFFFFFB0FA710D",
          "dev_eui": "FFFFFFFFD006D09E",
          "dev_nonce": "C5C2"
        }

I have the feeling that the code no longer works with V1.2 of the T-Beam and / or the libraries used - which I have just downloaded - do not work with the device.
I’ll try my hand at RadioLib, but if I have to rewrite a lot of the code, it may take a while.
I have absolutely no idea how to write the code.
I have never worked with an Arduino in my life.

I downloaded not lmic 3.2.0, i get the newer version 5.0.1.

@Jeff-UK I only have one other LoRa device, which is a closed device without external antennas. This is currently working without any problems. However, I can’t contribute anything further to this topic here with this device as it is completely closed (waterproof).

descartes · December 22, 2024, 5:56pm

I’ve not tried the latest v5 series and I see from the release notes that part of the work done was to make it work with the new ESP32 release which was a bit of a hot potato. Given that Terri focuses LMIC efforts on his own range of boards, ESP

I’d be reasonably confident that a v2 of the ESP BSP with LMIC v4 would work. I’m pretty conservative with updating stuff so I know I’ve not tried a v3 / v5 combo yet. Maybe something to try in a moment over the mid-winter break.

RadioLib is an alternative to LMIC and has a notes.md that walks you through getting going - you’d only have code stuff to do if you chose to use that & wanted to do TTN Mapper work. But in the first instance RadioLib will let you know if your board is OK - which no antenna issues aside, I’d suspect it is.

Does this mean you can code in C but haven’t used Arduino or that the whole world of small microcontrollers with radios is outside your current experience?

What is it - make & model?

vorderwaeldler · December 22, 2024, 6:24pm

I can’t program C.
If you put a piece of paper in front of me and said “here you go”, it wouldn’t work
I can understand the code and change things with the help of ChatGPT.
(It can do Python pretty well … )

My hope was to use the instructions to build a TTN mapper so that I could measure my gateway.
I’m the only one here in town with a gateway and I want to know how well it works.
The topology is a bit tricky here.
The project is probably turning out to be an “advanced project”.

The other device I have is a Tabs TBOL100.
I can use it to track the cat

I have replaced the library with v3.2.0, v3.3.0, v4.0.0 and 4.1.0.
But I always get this error when compiling:

/home/user/.arduino15/packages/esp32/tools/esp-x32/2302/bin/../lib/gcc/xtensa-esp32-elf/12.2.0/../../../../xtensa-esp32-elf/bin/ld: /home/user/.arduino15/packages/esp32/tools/esp32-arduino-libs/idf-release_v5.1-632e0c2a/esp32/lib/libpp.a(hal_mac.o): in function `hal_init':
(.text.hal_init+0x40): multiple definition of `hal_init'; /home/user/.cache/arduino/sketches/E368F9F09813AEAD7DF6AA9E4EE4B471/libraries/arduino-lmic-4.1.0/hal/hal.cpp.o:/home/maxi/Arduino/libraries/arduino-lmic-4.1.0/src/hal/hal.cpp:416: first defined here
collect2: error: ld returned 1 exit status

exit status 1

Compilation error: exit status 1

descartes · December 22, 2024, 6:30pm

First steps, as you get more changes to make, you’ll start to figure out stuff yourself!

As long as you promise NEVER to attach that to any cat ever, I’m sure we can get you over the hump.

If you’ve followed the info for the LMIC version, try RadioLib, you’ve nothing to lose other than your sanity. And if you can find something else to do, like make wool balls for the cat, getting a TTN Mapper sorted for RadioLib has some merit.

terrillmoore · December 22, 2024, 8:46pm

@descartes / Actually it turned out that the AppEUI == 0 issue was an issue with the packet forwarder for Multi-Tech gateways – we had a monoculture and we didn’t realize that this was the problem until I got a TTIG. It dated back to a bug fix for the USB version of the MultiTech LoRa radios – sometimes a packet of all zeros came in and it was mistaken for a join request. The solution in the code was to suppress packets with zero appeuis (which are first in the packet). But it was reported as a bug… But all good advice, thanks for supporting users!

terrillmoore · December 22, 2024, 8:50pm

/home/user/.arduino15/packages/esp32/tools/esp-x32/2302/bin/../lib/gcc/xtensa-esp32-elf/12.2.0/../../../../xtensa-esp32-elf/bin/ld: /home/user/.arduino15/packages/esp32/tools/esp32-arduino-libs/idf-release_v5.1-632e0c2a/esp32/lib/libpp.a(hal_mac.o): in function `hal_init':
(.text.hal_init+0x40): multiple definition of `hal_init'; /home/user/.cache/arduino/sketches/E368F9F09813AEAD7DF6AA9E4EE4B471/libraries/arduino-lmic-4.1.0/hal/hal.cpp.o:/home/maxi/Arduino/libraries/arduino-lmic-4.1.0/src/hal/hal.cpp:416: first defined here
collect2: error: ld returned 1 exit status

exit status 1

Compilation error: exit status 1

This is a problem with conflicting names., issue ESP32 is failing regression tests · Issue #966 · mcci-catena/arduino-lmic · GitHub. It was fixed in V5. Try using v5.0.1, which is current, and which I know people are using on ESP32.

descartes · December 22, 2024, 8:53pm

Has it been changed to cope with all zeros - a convenient solution to TTS v3 that now asks for a Join EUI up front?

Will endeavour to get myself up to date on v5 over the break.

PS, we had a brief chat at the TTN conference - always feels good to have colleagues around the globe!

terrillmoore · December 22, 2024, 10:56pm

Has it been changed to cope with all zeros?

If “it” refers to the LMIC, it turns out it wasn’t broken. The Semtech-protocol packet fowarder for the MultiTech was dropping JoinRequests with AppEUI == 0, and by sheer bad luck, the reporter and we at MCCI were all using MultiTech. So I mistakenly thought that the LMIC was broken. Changing the MultiTech gateway to use basic station solves the problem.

Embarassingly, the gateways run by TTN New York are all MultiTechs and the most are still on packet forwarder. But this won’t affect users in Europe!

Yes, I remember we met, hope all is well & best wishes for Yuletide & 2025!

kersing · December 22, 2024, 11:44pm

Updating to the release of packet forwarder released two years ago will fix it as well. And that will be the better option for MultiTech gateways with old concentrator cards as you need new hardware for basics station. Or consider replacing the gateways as my two MultiTech gateways from 2015 both died over the last year. (Flash worn out)

kersing · December 22, 2024, 11:47pm

Given that the gateway shows valid data that seems unlikely.

Which packet forwarder are you using on the gateway?

terrillmoore · December 23, 2024, 4:29am

Updating to the release of packet forwarder released two years ago will fix it as well.

Yes, I’d forgotten that, but that’s true. Thanks!

vorderwaeldler · December 23, 2024, 9:58am

Hi!

I have established a secure uplink connection on my MikroTik with the following configurations:

Address: eu1.cloud.thethings.network
Protocol: LNS
Port 8887
API Key: yes
SSL: yes

vorderwaeldler · December 23, 2024, 4:23pm

Small update

I have tested the project TTGO T-Beam Tracker for The Things Network again.
The display seems to work, only the display for the battery does not work.
Apparently a new library has to be implemented.

But GPS reception is there, the time has been determined and a location is available after a few minutes.
grafik

The display shows the current position data quite quickly every 20 seconds.
These values can then be recognized:

Latitude: 48.XXXXXXXX
Longitude: 8.XXXXXXXX
Error: 1.00m
Message queued

I think “error” means as much as accuracy.
1.00m varies from 0.9m to 1.9m

But well, joining the LoRa network still doesn’t work.