TTN V3 - Au1 cluster throws Duplicate_Identifier issue on new device registration

radiobridge · June 30, 2021, 6:22pm

I am integrating the TTN v3 API in my application, Though i am seeing many issues.

When I register a device in the Au1 cluster then it’s creating a device with the Activation method ABP, while I want to use the OTAA method. and there is no way to update the activation method later through API nor console.
As per another topic I got to know that we need to call 4 APIs to register a device in a cluster, I’ve followed the same, but I am seeing “Duplicate identifier” error, in my account this is the first device that i am trying to register. So not sure why i am seeing this error. Please correct me if i am doing anything wrong in the below API calls.

Identity Server API Call: POST
https://eu1.cloud.thethings.network/api/v3/applications/radiobridge-app-aus1/devices

Request JSON:

{
  "end_device": {
    "ids": {
      "device_id": "au1-4491600000f7c5b9",
      "dev_eui": "4491600000F7C5B9",
      "join_eui": "0101010101010101"
    },
    "join_server_address": "au1.cloud.thethings.network",
    "network_server_address": "au1.cloud.thethings.network",
    "application_server_address": "au1.cloud.thethings.network",
    "name": "Door Window Sensor"
  },
  "field_mask": {
    "paths": [
      "join_server_address",
      "network_server_address",
      "application_server_address",
      "ids.dev_eui",
      "ids.join_eui",
      "name"
    ]
  }
}

Response:

{
  "ids": {
    "device_id": "au1-4491600000f7c5b9",
    "application_ids": {
      "application_id": "radiobridge-app-aus1"
    },
    "dev_eui": "4491600000F7C5B9",
    "join_eui": "0101010101010101"
  },
  "created_at": "2021-06-30T18:09:30.248Z",
  "updated_at": "2021-06-30T18:09:30.248Z",
  "name": "Door Window Sensor",
  "version_ids": {},
  "network_server_address": "au1.cloud.thethings.network",
  "application_server_address": "au1.cloud.thethings.network",
  "join_server_address": "au1.cloud.thethings.network"
}

Network Server API Call: POST
https://au1.cloud.thethings.network/api/v3/ns/applications/radiobridge-app-aus1/devices

Request:

{
  "end_device": {
    "multicast": false,
    "supports_join": true,
    "lorawan_version": "MAC_V1_0_2",
    "ids": {
      "device_id": "au1-4491600000f7c5b9",
      "dev_eui": "4491600000F7C5B9",
      "join_eui": "0101010101010101"
    },
    "mac_settings": {
      "supports_32_bit_f_cnt": true
    },
    "supports_class_c": false,
    "supports_class_b": false,
    "lorawan_phy_version": "PHY_V1_0_2_REV_A",
    "frequency_plan_id": "AU_915_928_FSB_2"
  },
  "field_mask": {
    "paths": [
      "multicast",
      "supports_join",
      "lorawan_version",
      "ids.device_id",
      "ids.dev_eui",
      "ids.join_eui",
      "mac_settings.supports_32_bit_f_cnt",
      "supports_class_c",
      "supports_class_b",
      "lorawan_phy_version",
      "frequency_plan_id"
    ]
  }
}

Response:

{
  "code": 6,
  "message": "error:pkg/networkserver/redis:duplicate_identifiers (duplicate identifiers)",
  "details": [
    {
      "@type": "type.googleapis.com/ttn.lorawan.v3.ErrorDetails",
      "namespace": "pkg/networkserver/redis",
      "name": "duplicate_identifiers",
      "message_format": "duplicate identifiers",
      "correlation_id": "7ea7a6461d3f4eddaf7ffe33057ed833",
      "code": 6
    }
  ]
}

Application Server API Call: POST
https://au1.cloud.thethings.network/api/v3/as/applications/radiobridge-app-aus1/devices

Request:

{
  "end_device": {
    "ids": {
      "device_id": "au1-4491600000f7c5b9",
      "dev_eui": "4491600000F7C5B9",
      "join_eui": "0101010101010101"
    }
  },
  "field_mask": {
    "paths": [
      "ids.device_id",
      "ids.dev_eui",
      "ids.join_eui"
    ]
  }
}

Response:

{
  "code": 6,
  "message": "error:pkg/applicationserver/redis:duplicate_identifiers (duplicate identifiers)",
  "details": [
    {
      "@type": "type.googleapis.com/ttn.lorawan.v3.ErrorDetails",
      "namespace": "pkg/applicationserver/redis",
      "name": "duplicate_identifiers",
      "message_format": "duplicate identifiers",
      "correlation_id": "e515b4b170de434489d054455e32790e",
      "code": 6
    }
  ]
}

Join Server API call: POST
https://eu1.cloud.thethings.network/api/v3/js/applications/radiobridge-app-aus1/devices

Request:

{
  "end_device": {
    "ids": {
      "device_id": "au1-4491600000f7c5b9",
      "dev_eui": "4491600000F7C5B9",
      "join_eui": "0101010101010101"
    },
    "network_server_address": "au1.cloud.thethings.network",
    "application_server_address": "au1.cloud.thethings.network",
    "network_server_kek_label": "",
    "application_server_kek_label": "",
    "application_server_id": "",
    "net_id": null,
    "root_keys": {
      "app_key": {
        "key": "1F7FA60649841A4095164139B3B1D5AF"
      }
    }
  },
  "field_mask": {
    "paths": [
      "network_server_address",
      "application_server_address",
      "ids.device_id",
      "ids.dev_eui",
      "ids.join_eui",
      "network_server_kek_label",
      "application_server_kek_label",
      "application_server_id",
      "net_id",
      "root_keys.app_key.key"
    ]
  }
}

Response:

{
  "code": 6,
  "message": "error:pkg/joinserver/redis:duplicate_identifiers (duplicate identifiers)",
  "details": [
    {
      "@type": "type.googleapis.com/ttn.lorawan.v3.ErrorDetails",
      "namespace": "pkg/joinserver/redis",
      "name": "duplicate_identifiers",
      "message_format": "duplicate identifiers",
      "correlation_id": "5aa904b5afba4d90b70b449b77c68412",
      "code": 6
    }
  ]
}

Note: To resolve this problem, I tried to delete the devices from AS, NS, and JS servers, but that returns device not found error, which means the device is not registered on these servers, so not sure why I am seeing this error.

Also, I am calling these APIs in the following sequence:
Identity server, Join Server, Network server, Application Server

Can anybody please help me? I am trying to solve this from 2 Days but no luck.

If I try to register this same device in the EU and North America Cluster then that seems to be fine. I would like to know what I am missing or what’s wrong happening.

radiobridge · June 30, 2021, 7:11pm

Just to add more information to this, I am even not able to add the device through TTN Console as well. Please check the attached screenshot.

Screenshot 2021-07-01 at 12.39.43 AM

radiobridge · July 1, 2021, 2:36pm

@descartes Sorry to bother you, tagging you so that you can take a look at my post. I am stuck due to this issue for 2 days and I am not sure what I am doing wrong. I’ve tried all the ways to resolve the issue.

I am not sure if the problem is that i am passing the same dev EUI and Join EUI to register a device. But i deleted the device from the console and it’s not visible there.

The scenario is our JoinEUI will be the same for all devices, only Dev EUI will change and it’s possible we may be adding the devices in different Clusters. is that a problem?

descartes · July 1, 2021, 2:46pm

The ID’s are global to the whole of the TTS CE and can’t be recycled if deleted.

You can only add one DevEUI / AppEUI unique combo whole of TTS CE.

Your JoinEUI is clearly made up so should be all zeros OR an official one (preferably) OR an unofficial one created using this Random EUI or Key generator (least preferable).

What’s your language of choice - I’ve got as far as a Python one and there’s a PHP one I made on the forum that’s temporarily disappeared from my hard disk.

radiobridge · July 1, 2021, 5:07pm

Thank you for the reply. I understand the ID part and Unique DevEUI/AppEUI combo.

The problem is I am able to use the same ID and DevEUI/AppEUI combo in EU1 and NAM1 cluster and the API call works fine. Only I am getting the issue when I make calls to AU1 cluster servers.

Join Server, Application Server and Network server returns the Duplicate identifier for AU cluster. But if I make the call to NAM1 and EU1 cluster with the same parameters then that works fine and returns a proper response.

That’s why I am wondering if there is any problem in the AU1 cluster. Also, I don’t see any way to choose the OTAA method for registration.

I am using PHP and got the code that you posted, but here the main problem is TTN console is also returning the same error. I also posted a screenshot in this thread to show the error i see when i try to add the device.

descartes · July 1, 2021, 5:10pm

Maybe the problem is in reverse, maybe the second registration shouldn’t work.

There has been some discussion on Slack about having an application that spans continents and the short answer is you can, but preferable that you regionalise if you can.

But overall, I’d be pressed to see why you’d try to register a device in three regions - what are you trying to achieve.

Perhaps @htdvisser could comment …

radiobridge · July 1, 2021, 5:15pm

Also i see we can reuse the deleted Device IDs as per the documentation When an end device is deleted, the ID and EUI are released and can be reused

My Scenario is:

I’ve a Door Sensor device, At a time I want to register the device in one cluster. and if needed I will be deleting the device from one Cluster and then add the device to another cluster.

When I try to perform the above then I am unable to do that, first, I deleted the device from the EU1 cluster and then trying to add in the AU1 cluster. but somehow identity server returns success but Join Server, Application Server, and Network Server returns “Duplicate Identifier” error.

descartes · July 1, 2021, 5:16pm

Are you deleting them from all four places?

radiobridge · July 1, 2021, 5:19pm

Yes, I deleted the device from all 4 places, Identity Server, corresponding Network, Join, and Application server.

radiobridge · July 3, 2021, 7:09am

@htdvisser Need your help with this issue, can you please check and suggest about what we are doing wrong or if there is an issue in TTN network?

radiobridge · July 19, 2021, 8:42am

@descartes
is there any chance that I can search for the devices using the devEUI and joinEUI to find out deviceID which is registered on the Application Server, join server, and network server, so that I can delete them to register my device properly?

It will be very helpful if you can show me the right direction.

radiobridge · July 19, 2021, 8:50am

@bafonins @htdvisser @johan
I’ve registered a device on TTN Network with some Device ID and I missed deleting the device from Join, Application, and Network server, Now when I try to add the device again then I am seeing 409 Conflict - Duplicate Identifier error,

Can you please help me in resolving this issue, is there any way to clean the registration of the following device EUI and join EUI

Device EUI: 4491600000F7C5B9
Join EUI/App EUI: 0101010101010101

Sorry, I was looking for some help on this, but I think only core team members can guide me in the right direction, that’s why tagged you guys.

descartes · July 19, 2021, 9:11am

This doesn’t inspire - either use all Zeros as per console or a proper EUI. See & TR007 (to which @johan was party to its creation)

Others can tell you the same things to be found on the forum using search - the database integrity is fully documented and the ability of TTI to dip in to the live data is restricted to ensure privacy & therefore non-trivial.

We run the risk of TTI staffers being asked to clean up after us all if they, well, start accepting clean up requests. Can you not use a different ID?