TTN Mapper with T-Beam V1.2

vorderwaeldler · December 22, 2024, 11:02am

Hello forum,
I have a T-Beam from Lilygo here in version 1.2.
I wanted to use it to fill TTN Mapper with some information.
I used this project here as a code base:
Part 1 - Part 2 - Part 3
(Attention, all in German!)

The access data for OTAA has been entered correctly
The frequency has been changed to 868MHz
I have left CFG_sx1276_radio active

After uploading the code to the T-Beam, I only see the following two lines:

EV_TXSTART
EV_JOIN_TXCOMPLETE: no JoinAccept

On my MikroTik gateway, however, I can see that the join requests are arriving and the correct frequency is being used.
The EUIs look correct.
I have tried several settings, but nothing has worked.

Then I tried my hand at this project here:
https://github.com/kizniche/ttgo-tbeam-ttn-tracker
It is no longer up to date for V1.2, but at least the LoRa story works.
But even with “TTGO T-Beam Tracker for The Things Network” I can’t join the network.
The join requests there also always come to nothing.

What am I doing wrong?
Does V1.2 still work at all with the codes?

Finally, a question about the first project:
How can I integrate the display there so that it shows me current values?
This doesn’t seem to work with the available code.

stevencellist · December 22, 2024, 11:32am

I’d start off with checking your antenna connections and a forum search about distance between device & gateway

descartes · December 22, 2024, 11:50am

This ^^ - using the search term “no JoinAccept”.

When you say the EUI’s look correct, it would be more technically definitive if you told us what was appearing on the gateway & device consoles.

vorderwaeldler · December 22, 2024, 12:31pm

@stevencellist The connection to the gateway is perfect.
RSSI: -98.00dB
SNR: 4.50dB to 6.75dB
Without an antenna, there are no join requests at the gateway.
If I go a little closer to the gateway, I have the following values:
RSSI: -83.00dB
SNR: 9.50dB

Another LoRa device, which is a little closer to the gateway, works without any problems.

@descartes I don’t see much in the console of the device.

13:26:18.477 -> ets Jul 29 2019 12:21:46
13:26:18.477 ->
13:26:18.477 -> rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
13:26:18.477 -> configsip: 0, SPIWP:0xee
13:26:18.477 -> clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
13:26:18.537 -> mode:DIO, clock div:1
13:26:18.537 -> load:0x3fff0030,len:4832
13:26:18.537 -> load:0x40078000,len:16460
13:26:18.537 -> load:0x40080400,len:4
13:26:18.537 -> load:0x40080404,len:3504
13:26:18.537 -> entry 0x400805cc
13:26:18.638 -> Starting
13:26:18.671 -> Payload: 00 00 00 00 00 00 00 00 00 00 00
13:26:18.671 -> Packet queued
13:26:18.671 -> 2674: EV_JOINING
13:26:24.346 -> 358635: EV_TXSTART
13:26:30.440 -> 739282: EV_JOIN_TXCOMPLETE: no JoinAccept

This is what I see on the gateway:
grafik
grafik

I can also see this in the TTN console:

The only thing I find strange is that I see FF FF FF FF FF in TTN in the JoinEUI, whereas the correct ID is on the gateway.
Is that normal?

descartes · December 22, 2024, 1:00pm

RSSI & SNR are OK - how far apart is device & gateway?

We’d refer to that as the serial debug but any & all information is good.

The Join & Dev EUI are transmitted in plain text so are public information. The only things to redact are the AppKey & any API keys you generate. A gateway EUI appears all over the place and helps us with checking status / debugging. But anyway, you’ve told us the payload …

I presume this is the gateway console - please help the volunteers to help you with precision. What does the device console say - if it doesn’t say anything, no picture needed.

No, that’s totally wrong. Everything has to match from device, any outputs on any serial logs or the TTN consoles.

The Join Request appears to be:

00 42 3C C3 A5 C2 59 D2 D7 C7 43 3C 42 DF 99 79 FF 2D 4E 64 80 20 42

So your Join EUI starting D7 ending 42 and the Dev EUI starting FF and ending C7 on your gateway screen shot seems to match up with the payload but then gets horribly mangled at TTN.

What version of LW have you set on the console?

Where did you get these EUI’s? TTN allocates EUI’s starting 70 B3 D5

vorderwaeldler · December 22, 2024, 1:52pm

The device is about 20 meters away from the gateway.
At the best value it was perhaps 10 meters.

According to the instructions, I should set this here:
grafik

The instructions say that I should enter something in the JoinEUI.
The main thing is that it is in HEX and I can press Confirm.
One statement in a video was also “that the console would report if the JoinEUI is not okay”.
So I used the password manager to generate a 16-digit hex value.

The code for the T-Beam also contains the following line:

  //Set FS for RX2 (SF9 = TTN,TTS EU)
  LMIC.dn2Dr = DR_SF9;

In my opinion, the set frequency should fit.

grafik

descartes · December 22, 2024, 2:15pm

It is sufficient to just give text answers for things like what version - hard to read these screen shots on a mobile device and it’s more efficient for all - you don’t have to take the pics!

The settings are as per LMIC, the AppKey doesn’t come in to play until you receive a Join Request but, as I said above, assuming you showed a screen shot of the TTN gateway console, there’s no Join Accept being generated.

So need confirmation that is the gateway console and if there is anything on the device console please.

Also, did you see the notes in the code base about which way round the EUI’s need to be - LSB, not MSB!

vorderwaeldler · December 22, 2024, 2:29pm

I only have the serial monitor of the Arduino IDE and it doesn’t show me any further information.
That’s why I’m extremely confused and can’t find the error.

According to the instructions, APPEUI and DEVEUI must be specified in LSB and APPKEY in MSB.
That’s what I did.
I also entered the APPKEY in LSB once, but unfortunately that didn’t help either.

I like to add screenshots, as they sometimes give hints about errors that you don’t even think about at that moment.
I also like to keep track of things, as the selection fields of the TTN Console, for example, have changed somewhat in recent years.
The instruction videos on YouTube look a little different in some places.

descartes · December 22, 2024, 2:35pm

You have shown screen shots from TTN & your gateway so you definitely have more than the serial monitor! This means you have access to TTN so you can look at the console on the page for the device that you created.

The AppKey doesn’t care which way round it goes but it is simpler if it’s left as MSB.

Feel free to add screenshots but it’s harder to read on mobile devices - like now - so maybe I’ll wait until I’m back at a desktop in January!

I have no idea how a JR ends up with the mangled Dev & Join EUIs - I’ve used LMIC on & off for years so I’ve seen a fair amount of interesting issues but never this.

Maybe @kersing or @Jeff-UK can advise as they are gateway masters.

kersing · December 22, 2024, 3:05pm

Please remove this when using OTAA.

Unless I’m mistaken LMIC (specifically older versions) are LoRaWAN 1.0.2, not 1.0.3.

Not entirely true. The main thing is you use valid values that don’t clash with ranges someone paid good money for. If your gateway runs reasonably recent software (not > 4 years old) you can safely use the suggested all zeros value.

I have never seen a JoinEUI being scrambled between the gateway and TTN in this way before. Can you check if there is a software update for your gateway?

vorderwaeldler · December 22, 2024, 3:13pm

No hurry.
We’re tinkering here, not on the run

The package looks like this on the MikroTik GW:
Type: RX
Time: Dec/22/2024 15:57:57
Gateway ID: REMOVED-FOR-PRIVACY
Message Type: Join-request
Major Version: LoRaWAN R1
Join EUI: D7 D2 59 C2 A5 C3 3C 42
Dev EUI: FF 79 99 DF 42 3C 43 C7
Dev Nonce: 15657
Freq (MHz): 868.100
Modulation: LoRa
Bandwith: 125 kHz
Datarate: SF7
Coderate: 4/5
IF Chain: 0
CRC Status: OK
Counter (us): 1172420260
RF Chain: 1
RSSI (dB): -101.00
SNR: 6.25
CRC: 5702
Size: 23
Payload: AEI8w6XCWdLXx0M8Qt+Zef8pPbBEV80=

Here are the event details of a JR in the TTN Console in JSON:

{
  "name": "gs.up.receive",
  "time": "2024-12-22T14:57:57.703236185Z",
  "identifiers": [
    {
      "gateway_ids": {
        "gateway_id": "REMOVED-FOR-PRIVACY",
        "eui": "REMOTE-FOR-PRIVACY"
      }
    }
  ],
  "data": {
    "@type": "type.googleapis.com/ttn.lorawan.v3.GatewayUplinkMessage",
    "message": {
      "raw_payload": "AEI8w6X/////x0M8Qt+Zef8pPbBEV80=",
      "payload": {
        "m_hdr": {},
        "mic": "sERXzQ==",
        "join_request_payload": {
          "join_eui": "FFFFFFFFA5C33C42",
          "dev_eui": "FF7999DF423C43C7",
          "dev_nonce": "3D29"
        }
      },
      "settings": {
        "data_rate": {
          "lora": {
            "bandwidth": 125000,
            "spreading_factor": 7,
            "coding_rate": "4/5"
          }
        },
        "frequency": "868100000",
        "timestamp": 1172420260
      },
      "rx_metadata": [
        {
          "gateway_ids": {
            "gateway_id": "REMOVED-FOR-PRIVACY",
            "eui": "REMOVED-FOR-PRIVACY"
          },
          "timestamp": 1172420260,
          "rssi": -101,
          "channel_rssi": -101,
          "snr": 6.25,
          "location": {
            "latitude": REMOVED-FOR-PRIVACY,
            "longitude": REMOVED-FOR-PRIVACY,
            "altitude": REMOVED-FOR-PRIVACY,
            "source": "SOURCE_REGISTRY"
          },
          "uplink_token": "CiQKIgoWc2Nob25hY2gtMDEtcmJ3YXByLTJuZBIIUDA1QUpIR1AQpO2GrwQaDAj10aC7BhDHgKHPAiCg4cXNjyI=",
          "received_at": "2024-12-22T14:57:57.703086663Z"
        }
      ],
      "received_at": "2024-12-22T14:57:57.703086663Z",
      "correlation_ids": [
        "gs:uplink:01JFQDZZY73NXEVJD94FGP1QQT"
      ]
    },
    "band_id": "EU_863_870"
  },
  "correlation_ids": [
    "gs:uplink:01JFQDZZY73NXEVJD94FGP1QQT"
  ],
  "origin": "ip-10-100-12-7.eu-west-1.compute.internal",
  "context": {
    "tenant-id": "CgN0dG4="
  },
  "visibility": {
    "rights": [
      "RIGHT_GATEWAY_TRAFFIC_READ"
    ]
  },
  "unique_id": "01JFQDZZY734CWF0EBB5FSY074"
}

What differs here are the following things:

JoinEUI
Payload
Dev-Nonce

vorderwaeldler · December 22, 2024, 3:19pm

I have commented out the line and uploaded it again.
Unfortunately, this does not change the behavior.
The packages continue to arrive broken.

I have installed the latest MikroTik OS (RBwAPR-2nD).
I have also just installed the update from 7.16.1 to 7.16.2.

descartes · December 22, 2024, 3:40pm

Please! ANY other users device that is heard by your gateway will include the details you’ve removed in their uplink meta data. And join requests are sent in plain text so will appear in any gateway logs that hear it. You’ve taken time to redact some information but then leave other info in. From experience this means there is a likely hood that there are other things being altered that need to be left alone.

The original (German) article links to version 3.2.0 of LMIC which is old but should be co-operating. At that time it reached LW v1.0.3 so that’s OK.

You could try RadioLib which has first class support for T-Beam AND has been shown to be completely compliant for all LoRa frequencies (FSK coming soon apparently) - this will sanitise / test your board & gateway combo. At RadioLib Development Organization · GitHub you can pickup the board definitions library and at a later date use the the persistence for ESP32.

After that you can patch over the GNSS elements or you can go back to LMIC.

descartes · December 22, 2024, 3:42pm

It has just dawned on me that both of your EUI’s are invalid - they should have bit 0 set to 0 and for local addresses, bit 1 set to 1. So as MSB, the second ‘digit’ can only ever be an even number.

See: DevEUI for non-hardware assigned values - #23 by terrillmoore which is why Random EUI or Key generator | descartes exists.

Also, LMIC has been shown to NOT cope at all with an all zero JoinEUI.

Jeff-UK · December 22, 2024, 4:21pm

Hummmm, never run a device (GW or node) without the antenna attached!

If you have done this there is a risk that you have damaged the front end of the device. Maybe not a catastrophic failure but potentially degraded performance(*). If the node receiver is impaired it may not be hearing any join ack correctly or without corruption which could also be a problem, in addition to all the other points called out above. Also if all zero JoinEUI a problem then people often find a simple 00…01 solves

(*) the other risk being an impairment that doesnt show straight away but whilch leads to an early life failure/degraded lifetime - you get working initially bit then fails days, week, years later - but earlier than ‘should’ be the case. Do you have another known good device that hasntt been run (powered up) Ant-less? If so move to that until you have all working correctly…

vorderwaeldler · December 22, 2024, 5:36pm

@descartes I have just generated a new JoinEUI via the site.
I have also created a new DevEUI and AppKey via the TTN console.
I loaded the JoinEUI and DevEUI back onto the T-Beam in LSB format and the AppKey in MSB format.

Now it looks even stranger.
In the TTN console, the JoinEUI and DevEUI now start with FF FF FF FF FF.

        "join_request_payload": {
          "join_eui": "FFFFFFFFB0FA710D",
          "dev_eui": "FFFFFFFFD006D09E",
          "dev_nonce": "C5C2"
        }

I have the feeling that the code no longer works with V1.2 of the T-Beam and / or the libraries used - which I have just downloaded - do not work with the device.
I’ll try my hand at RadioLib, but if I have to rewrite a lot of the code, it may take a while.
I have absolutely no idea how to write the code.
I have never worked with an Arduino in my life.

I downloaded not lmic 3.2.0, i get the newer version 5.0.1.

@Jeff-UK I only have one other LoRa device, which is a closed device without external antennas. This is currently working without any problems. However, I can’t contribute anything further to this topic here with this device as it is completely closed (waterproof).

descartes · December 22, 2024, 5:56pm

I’ve not tried the latest v5 series and I see from the release notes that part of the work done was to make it work with the new ESP32 release which was a bit of a hot potato. Given that Terri focuses LMIC efforts on his own range of boards, ESP

I’d be reasonably confident that a v2 of the ESP BSP with LMIC v4 would work. I’m pretty conservative with updating stuff so I know I’ve not tried a v3 / v5 combo yet. Maybe something to try in a moment over the mid-winter break.

RadioLib is an alternative to LMIC and has a notes.md that walks you through getting going - you’d only have code stuff to do if you chose to use that & wanted to do TTN Mapper work. But in the first instance RadioLib will let you know if your board is OK - which no antenna issues aside, I’d suspect it is.

Does this mean you can code in C but haven’t used Arduino or that the whole world of small microcontrollers with radios is outside your current experience?

What is it - make & model?

vorderwaeldler · December 22, 2024, 6:24pm

I can’t program C.
If you put a piece of paper in front of me and said “here you go”, it wouldn’t work
I can understand the code and change things with the help of ChatGPT.
(It can do Python pretty well … )

My hope was to use the instructions to build a TTN mapper so that I could measure my gateway.
I’m the only one here in town with a gateway and I want to know how well it works.
The topology is a bit tricky here.
The project is probably turning out to be an “advanced project”.

The other device I have is a Tabs TBOL100.
I can use it to track the cat

I have replaced the library with v3.2.0, v3.3.0, v4.0.0 and 4.1.0.
But I always get this error when compiling:

/home/user/.arduino15/packages/esp32/tools/esp-x32/2302/bin/../lib/gcc/xtensa-esp32-elf/12.2.0/../../../../xtensa-esp32-elf/bin/ld: /home/user/.arduino15/packages/esp32/tools/esp32-arduino-libs/idf-release_v5.1-632e0c2a/esp32/lib/libpp.a(hal_mac.o): in function `hal_init':
(.text.hal_init+0x40): multiple definition of `hal_init'; /home/user/.cache/arduino/sketches/E368F9F09813AEAD7DF6AA9E4EE4B471/libraries/arduino-lmic-4.1.0/hal/hal.cpp.o:/home/maxi/Arduino/libraries/arduino-lmic-4.1.0/src/hal/hal.cpp:416: first defined here
collect2: error: ld returned 1 exit status

exit status 1

Compilation error: exit status 1

descartes · December 22, 2024, 6:30pm

First steps, as you get more changes to make, you’ll start to figure out stuff yourself!

As long as you promise NEVER to attach that to any cat ever, I’m sure we can get you over the hump.

If you’ve followed the info for the LMIC version, try RadioLib, you’ve nothing to lose other than your sanity. And if you can find something else to do, like make wool balls for the cat, getting a TTN Mapper sorted for RadioLib has some merit.

terrillmoore · December 22, 2024, 8:46pm

@descartes / Actually it turned out that the AppEUI == 0 issue was an issue with the packet forwarder for Multi-Tech gateways – we had a monoculture and we didn’t realize that this was the problem until I got a TTIG. It dated back to a bug fix for the USB version of the MultiTech LoRa radios – sometimes a packet of all zeros came in and it was mistaken for a join request. The solution in the code was to suppress packets with zero appeuis (which are first in the packet). But it was reported as a bug… But all good advice, thanks for supporting users!