AWS IoT integration problem when switching to another AWS account

TTN Network and Services The Things Stack v3
1

Hello!

Is it possible that The Things Network v3 AWS IoT integration is caching something that makes it unable to change AWS account once one account has been used?

I have successfully setup the AWS IoT integration and received messages from my TTN Node to AWS MQTT. I created this integration to my dev AWS account. Now I’m trying to update the integration to use my prod AWS account, where I have the same setup, and it doesn’t work. I keep seeing “Fail to process upstream message” in the live data view. When I switch the role and stack name settings back to the dev account, it works again.

I have also tried to create multiple roles in the dev account and I’m able to switch the role there succesfully. But using the prod account always fails.

In my CloudTrail I can see that The Things Network is successfully calling AssumeRole in the prod account and gets the temporary credentials. But then nothing happens and the error message appears in the live data view.

Here is a piece of the error data shown in the live data view. It doesn’t have any error detail, so I have trouble figuring out the exact problem.

"data": {
    "@type": "type.googleapis.com/ttn.lorawan.v3.ErrorDetails",
    "namespace": "pkg/applicationserver/io/packages/awsiot",
    "name": "publish",
    "message_format": "publish message: {message}",
    "attributes": {
      "message": ""
    },
    "correlation_id": "bb164264e7a64b5f9615d65d066aceb5",
    "code": 10
  },
2

How do you update the integration? Do you manually change the role ARN, or do you leave the Configure lambda in the prod account configure the integration for you?

Did you try deleting the AWS IoT integration from The Things Stack Console, and then deploy the CloudFormation template in your prod account?

3

I manually changed the Role ARN and the Stack name in the TTN console. I don’t really want all the CloudFormation resources, I only want to receive the raw messages through MQTT.

4

When you say “delete AWS IoT integration”, do you mean the “Disable AWS IoT integration” button? I did try to disable and re-enable it, and it did not seem to have effect.

5

I see. I looked at this and it appears that the AWS IoT Core MQTT endpoint isn’t described when these fields change. So indeed, The Things Stack keeps using the old endpoint.

I do. Can you also try with CLI please?

$ ttn-lw-cli app pkg def-assoc rm <app-id> 198
6

Thanks, it looks like the CLI worked. I ran the def-assoc rm command, and then re-added the integration from the Console UI. Now I’m receiving the MQTT messages in AWS IoT successfully.

I guess the TTN backend system should “forget” the IoT endpoint address when the IAM role is changed from the UI…

7

Glad that it helped. I filed an internal issue for this.

1 Like