MIC Mismatch when joining with custom Device over OTAA (Arduino & RFM95Hope)

Hi Guys,

In a previous thread today, I had some problems migrating my custom hardware from v2 to v3.

I can now send again to v3 using the maniacbug-bootloader for my custom PCB with an ATMEGA1284p MCU.

However, now, I am getting a MIC mismatch error:
grafik

That indicates something with ma AppEUI, AppKEY or DevEUI is wrong, I checked it multiple times. I Generated the DevEUI using the old v2 console. as AppEUI I only took zeroes.

I could add and send reasonable payload using ABP but because there is no option to remove the frame counter check, I cannot really use that option.

What am I doing wrong with the OTAA activation? I used LoRaWAN 1.0.2 protocol as previously, tested PhysA & B.

EDIT: when changing the DevEUI from msb to lsb, I receive the following response:
grafik
But I don’t get the payload using the standard LMIC-OTAA example:

/*******************************************************************************
 * Copyright (c) 2015 Thomas Telkamp and Matthijs Kooijman
 * Copyright (c) 2018 Terry Moore, MCCI
 *
 * Permission is hereby granted, free of charge, to anyone
 * obtaining a copy of this document and accompanying files,
 * to do whatever they want with them without any restriction,
 * including, but not limited to, copying, modification and redistribution.
 * NO WARRANTY OF ANY KIND IS PROVIDED.
 *
 * This example sends a valid LoRaWAN packet with payload "Hello,
 * world!", using frequency and encryption settings matching those of
 * the The Things Network.
 *
 * This uses OTAA (Over-the-air activation), where where a DevEUI and
 * application key is configured, which are used in an over-the-air
 * activation procedure where a DevAddr and session keys are
 * assigned/generated for use with all further communication.
 *
 * Note: LoRaWAN per sub-band duty-cycle limitation is enforced (1% in
 * g1, 0.1% in g2), but not the TTN fair usage policy (which is probably
 * violated by this sketch when left running for longer)!

 * To use this sketch, first register your application and device with
 * the things network, to set or generate an AppEUI, DevEUI and AppKey.
 * Multiple devices can use the same AppEUI, but each device has its own
 * DevEUI and AppKey.
 *
 * Do not forget to define the radio type correctly in
 * arduino-lmic/project_config/lmic_project_config.h or from your BOARDS.txt.
 *
 *******************************************************************************/

#include <lmic.h>
#include <hal/hal.h>
#include <SPI.h>

//
// For normal use, we require that you edit the sketch to replace FILLMEIN
// with values assigned by the TTN console. However, for regression tests,
// we want to be able to compile these scripts. The regression tests define
// COMPILE_REGRESSION_TEST, and in that case we define FILLMEIN to a non-
// working but innocuous value.
//
#ifdef COMPILE_REGRESSION_TEST
# define FILLMEIN 0
#else
# warning "You must replace the values marked FILLMEIN with real values from the TTN control panel!"
# define FILLMEIN (#dont edit this, edit the lines that use FILLMEIN)
#endif

// This EUI must be in little-endian format, so least-significant-byte
// first. When copying an EUI from ttnctl output, this means to reverse
// the bytes. For TTN issued EUIs the last bytes should be 0xD5, 0xB3,
// 0x70.
static const u1_t PROGMEM APPEUI[8]={0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
void os_getArtEui (u1_t* buf) { memcpy_P(buf, APPEUI, 8);}

// This should also be in little endian format, see above.
static const u1_t PROGMEM DEVEUI[8]={0xE5, 0x60, 0xFF, XXX};
void os_getDevEui (u1_t* buf) { memcpy_P(buf, DEVEUI, 8);}

// This key should be in big endian format (or, since it is not really a
// number but a block of memory, endianness does not really apply). In
// practice, a key taken from ttnctl can be copied as-is.
static const u1_t PROGMEM APPKEY[16] = {0x22, 0x73, 0xBB, 0xB1, 0x7C, 0x05, XXX};
void os_getDevKey (u1_t* buf) {  memcpy_P(buf, APPKEY, 16);}

static uint8_t mydata[] = "Hello, world!";
static osjob_t sendjob;

// Schedule TX every this many seconds (might become longer due to duty
// cycle limitations).
const unsigned TX_INTERVAL = 10;

// Pin mapping
const lmic_pinmap lmic_pins = {
  .nss = 12,
  .rxtx = LMIC_UNUSED_PIN,
  .rst = 13,
  .dio = {2, 3, 15},
};

void onEvent (ev_t ev) {
    Serial.print(os_getTime());
    Serial.print(": ");
    switch(ev) {
        case EV_SCAN_TIMEOUT:
            Serial.println(F("EV_SCAN_TIMEOUT"));
            break;
        case EV_BEACON_FOUND:
            Serial.println(F("EV_BEACON_FOUND"));
            break;
        case EV_BEACON_MISSED:
            Serial.println(F("EV_BEACON_MISSED"));
            break;
        case EV_BEACON_TRACKED:
            Serial.println(F("EV_BEACON_TRACKED"));
            break;
        case EV_JOINING:
            Serial.println(F("EV_JOINING"));
            break;
        case EV_JOINED:
            Serial.println(F("EV_JOINED"));
            {
              u4_t netid = 0;
              devaddr_t devaddr = 0;
              u1_t nwkKey[16];
              u1_t artKey[16];
              LMIC_getSessionKeys(&netid, &devaddr, nwkKey, artKey);
              Serial.print("netid: ");
              Serial.println(netid, DEC);
              Serial.print("devaddr: ");
              Serial.println(devaddr, HEX);
              Serial.print("artKey: ");
              for (int i=0; i<sizeof(artKey); ++i) {
                Serial.print(artKey[i], HEX);
              }
              Serial.println("");
              Serial.print("nwkKey: ");
              for (int i=0; i<sizeof(nwkKey); ++i) {
                Serial.print(nwkKey[i], HEX);
              }
              Serial.println("");
            }
            // Disable link check validation (automatically enabled
            // during join, but because slow data rates change max TX
	    // size, we don't use it in this example.
            LMIC_setLinkCheckMode(0);
            break;
        /*
        || This event is defined but not used in the code. No
        || point in wasting codespace on it.
        ||
        || case EV_RFU1:
        ||     Serial.println(F("EV_RFU1"));
        ||     break;
        */
        case EV_JOIN_FAILED:
            Serial.println(F("EV_JOIN_FAILED"));
            break;
        case EV_REJOIN_FAILED:
            Serial.println(F("EV_REJOIN_FAILED"));
            break;
        case EV_TXCOMPLETE:
            Serial.println(F("EV_TXCOMPLETE (includes waiting for RX windows)"));
            if (LMIC.txrxFlags & TXRX_ACK)
              Serial.println(F("Received ack"));
            if (LMIC.dataLen) {
              Serial.print(F("Received "));
              Serial.print(LMIC.dataLen);
              Serial.println(F(" bytes of payload"));
            }
            // Schedule next transmission
            os_setTimedCallback(&sendjob, os_getTime()+sec2osticks(TX_INTERVAL), do_send);
            break;
        case EV_LOST_TSYNC:
            Serial.println(F("EV_LOST_TSYNC"));
            break;
        case EV_RESET:
            Serial.println(F("EV_RESET"));
            break;
        case EV_RXCOMPLETE:
            // data received in ping slot
            Serial.println(F("EV_RXCOMPLETE"));
            break;
        case EV_LINK_DEAD:
            Serial.println(F("EV_LINK_DEAD"));
            break;
        case EV_LINK_ALIVE:
            Serial.println(F("EV_LINK_ALIVE"));
            break;
        /*
        || This event is defined but not used in the code. No
        || point in wasting codespace on it.
        ||
        || case EV_SCAN_FOUND:
        ||    Serial.println(F("EV_SCAN_FOUND"));
        ||    break;
        */
        case EV_TXSTART:
            Serial.println(F("EV_TXSTART"));
            break;
        default:
            Serial.print(F("Unknown event: "));
            Serial.println((unsigned) ev);
            break;
    }
}

void do_send(osjob_t* j){
    // Check if there is not a current TX/RX job running
    if (LMIC.opmode & OP_TXRXPEND) {
        Serial.println(F("OP_TXRXPEND, not sending"));
    } else {
        // Prepare upstream data transmission at the next possible time.
        LMIC_setTxData2(1, mydata, sizeof(mydata)-1, 0);
        Serial.println(F("Packet queued"));
    }
    // Next TX is scheduled after TX_COMPLETE event.
}

void setup() {
    Serial.begin(9600);
    Serial.println(F("Starting"));

    #ifdef VCC_ENABLE
    // For Pinoccio Scout boards
    pinMode(VCC_ENABLE, OUTPUT);
    digitalWrite(VCC_ENABLE, HIGH);
    delay(1000);
    #endif

    // LMIC init
    os_init();
    // Reset the MAC state. Session and pending data transfers will be discarded.
    LMIC_reset();

    // Start job (sending automatically starts OTAA too)
    do_send(&sendjob);
}

void loop() {
    os_runloop_once();
}

Is this device still present on v2?

You need to ensure that the device is effectively disabled on v2, you can do this (for example) by changing one byte in de app key on the V2 console. Then, when it joins, you can be sure it is actually joined to the v3 backend (not to v2) and uses session keys from v3.

The other possibility is of course a typing error, or MSB/LSB flip in the application EUI, device EUI or the application key.

@bertrik thank you. I have deleted the device on v2 that I used to generate the DevEUI for the new v3 device and checked it again, I still receive js.join.accept .

Here are more remarks for troubleshooting:

  1. The appkey needs to be MSB, otherwise i receive a MIC Mismatch / ns.up.join.cluster.fail
  2. msb/lsb of AppEUI doesn’t matter since it is filled with zeroes.
  3. DevEUI needs to be lsb → otherwise I receive no reaction on the console at all.

EDIT: My gateway is connected but doesn’t report the join request:
grafik

But I know that it always (including today) receives the payload just fine. If i switch to ABP again, I could see the proper payload.

I left the test device connected to my laptop and saw this morning that it successfully sent a message:
grafik
The payload is “48656C6C6F2C20776F726C6421” in hex which converts to “Hello, world!”, as provided in the OTAA example of the LMIC library, newest version 4.0.0. was used.

After opening the Arduino Serial Monitor, which resets the MCU, I get the same problem again that the device hangs at the join request.

  1. I could not find additional troubleshooting ideas from the [add devices documentation]
    (Devices | The Things Stack for LoRaWAN).
  2. I tried different settings in the add devices option, e.g. LoRaWAN 1.0.2 and 1.0.3 that should both be supported by the LMIC library.

May I ask one of the Captains here for help? @kersing @cslorabox @descartes @bluejedi
I can exclude that it’s a hardware problem. I’ve also tested the OTAA connection with custom PCB devices that worked well on ABP v2.

As I’ve performed sooooo many tests over the last few months I’ve lost track, but I vaguely recall that LMiC doesn’t understand all 00’s for the App/JoinEUI. Which would chime with the ABP working, OTAA not.

Try getting one from the v2 console and using that.

I’ve looked at LMiC 4.0.0 but not yet used it, so if that’s not a solution, you might want to try the latest 3.3.0 which I have been using to good effect.

I’ve copied the DevEUI and the App/JoinEUI from v2 console and tried v3 again.
grafik
for LMiC 3.3.0 and 4.4.0.

I’m now getting an unknown event error even for AppEUI filled up with 0s, which did not happen before. Eventually the connection is refused now because of too many attempts. I’ll give it a fresh try later today or tomorrow.

Please do not post ANYTHING that can be copied & pasted as an image.

Unknown event isn’t actually an error - I’ve not figured that one out yet.

Did you put the App/JoinEUI in to the v3 console?

Can you double-triple check your DIO1 connection just in case the RFM isn’t able to signal that it has an incoming message.

Nope, MCCI LMIC (v3.3.0, v4.0.0) does not have a problem with AppEUI containing all zero’s. At least I did not experience any.

I have devices successfully running on V3 with AppEUI containing all zero’s (EU868, LoRaWAN 1.0.3).

1 Like

Thank you @descartes I put the App/JoinEUI and checked the DIO1 for different devices. I also took devices that worked well on v2 ABP for months.

@bluejedi thanks for clarifying the AppEUI may contain zeroes on v3 with LMIC v3.3.0 and v4.0.0. May I ask you what your setup is (MCU or node), does the standard LMIC OTAA example work?

The gateway is not a problem, I’ve added commercial devices from dragino.

I’ll probably just get an Arduino Mega and use it with an RFM on a breadboard to see if I can constrain the problem to my custom PCBs (which worked the last months) or my LMIC library/wrong TTN keys.

As frequency plan I always took EU 868 SF9 Rx2 - recommended. That should be fine for my EU868 RFM.

My test setup consists of different types of nodes with different MCU types. The software used is LMIC-node which is a cross-platform example for LMIC. It uses OTAA by default but also supports ABP and is an improvement over the standard ttn-otaa.ino and ttn-abp.ino examples.

Information about LMIC-node can be found here:

and here:

FWIW, I have consistently good results with a Pro Mini + RFM95 using up to 3.3.0

Captains (@bluejedi & @descartes & @arjanvanb ): Setting #define LMIC_ENABLE_arbitrary_clock_error 1 in the config file and LMIC_setClockError(MAX_CLOCK_ERROR * 10 / 100); in the arduino code, as recommended in a previous topic by @arjanvanb. I’ve used the latest 4.0.0 version.

Thanks guys! Maybe this can have something to do with the fact that I’m using the internal 8MHz oscillator of the ATMEGA1284p. Eventually the clock timing becomes an issue then. Can this be the problem or is this hypothesis nonsense?

That makes sense to me - all the Pro Mini’s and Nano Every’s I used with an RFM95 have had a crystal (or similar) which will be much much more accurate than an internal clock on an older design Atmel part.

Well done!