What is your vision on how to connect TTN to AWS IoT?
Currently, we are linking the TTN server to AWS IoT using a node bridge. Often, AWS IoT support is made via a AWS IoT thing. However, we have discussed this with solution architects from AWS and they advised us to publish on a regular MQTT topic, e.g. uplink/<AppEUI>/<devEUI>, instead of creating a thing for every device we have. They claim AWS thing should only be used if you also need to publish a state back to the device, e.g. when downlink is required.
You mention using a node bridge, but not sure if you mean this one: https://www.npmjs.com/package/ttn-aws-iot (which is built by the core team) or something you’re putting together yourself.
AWS is stating you should not create a thing for every device, but if you look a t the ttn aws iot bridge that @gonzalocasas is mentioning then this is exactly connecting a LoRa device to a AWS iot Thing with that name. So this is inflexible in my opinion.
Maybe it is changed currently, but for our bridge we took the ttn-aws-iot as a starting pointed and changed it in such a way that it becomes flexible. and just listens to all devices of an appliction and sends them to a topic on AWS IoT like: uplink / appEUI / devEUI
The downside of this is that we do not automatically support multiple applications to be passed on to AWS IoT. So that’s why we would really look forward to AWS IoT support built in the TTN server already. My point is that the topic to be sent to AWS should be somehow configurable and not a hard coded AWS Thing, e.g. loriot is doing it like that and in my opinion that is not the always the good way to go with AWS IoT
I’m not sure why AWS solution architects are claiming that you should not sync TTN devices and AWS IoT things. I’m working with AWS on an integration with our v2 (preview) environment, and it will automatically sync devices and things, as things are meant in AWS IoT.
With the current connector, for v1-staging, you have to create a thing first in AWS IoT. This will be done automatically in v2.
Maybe I misinterpret @arjanvanb, but do I understand it correctly that with TTN v2 and AWS IoT connector will publish on the same topic on the AWS MQTT queue as on the TTN queue? So the message are published on the < AppEUI >/devices/< devEUI >/up topic on AWS IoT. And next to this the state is reported on a generated AWS IoT thing?
It is not that you should never sync devices and things. But in some use cases it is not by definition the best practice to do so. I do not know the exact specifications of TTN v2, but I can explain a bit on the reasons why we are not using AWS things currently. Note: We do sync devices and things for the devices whose state can be set via downlink, but this is only a small part of the total of devices.
A thing id is not very flexible, so you cannot register a rule on an application topic, nor a device type topic. This would mean a rule per device, in case of 1000s of devices, this would imply maintaining many IoT rules.
We want to store the information from different kinds of devices in a uniform format, we do this on AWS as TTN payload functions were not adequate at that time. I understood that the payload functions will be device specific in v2? Will there also be sort of payload functions per device type? So in case of 1000s of devices with the same payload function, would it be easy to update the code of the payload function?
Inspecting the latest (and historical devices readings, e.g. temp levels) can not be done fast and in bulk when storing the state in things. So that’s why we are storing them in dynamo and our business logic. So skipping the thing will improve throughput time and lesten AWS costs.
Our solution is also multi-tenant. We are handling multiple applications on our AWS platform. It is undoable to authorize customers to only publish on the topics of their own things when handling 1000s of things. With MQTT topics like < appEUI >/devices/< deviceEUI >, authorization certificates can be created per customer for their own queue < appEUI >/+
What the connector does, is creating a thing type on AWS IoT for TTN devices. All devices are created as thing with that thing type.
On uplink, the connector reports state in AWS IoT default format and topic structure: $aws/things/<device_id>/shadow/update.
I’m myself not entirely familiar with the query engine for rules, but if I understand you right, you want to be able to create different rules for different applications.
Would publishing to the v2 topic structure (<AppID>/devices/<DevID>/up, see https://www.thethingsnetwork.org/docs/v2-preview/mqtt/), query on the AppID and then republish to the right thing topic work better for you?
Regarding payload functions; in v2 you will be able to take the FPort into account. Your device can use the FPort to indicate the payload type while the device still falls within the same application. The payload function can switch on the FPort to decode the right binary format.
Yes, publishing to the v2 topic structure /devices//up on AWS IoT would be perfect.That way we are in control by using the AWS IoT ruling mechanism to store the payload in dynamo and decide for some devices to also republish to the corresponding thing structure.
Additionally, I see that in v2 TTN you can also subscribe to topics for individual per payload field, which is a nice feature. However, with respect to publishing to AWS IoT it might be wise to not publish too much and have the consumer subscribe to AWS IoT topics that are used. As with AWS IoT you have to pay per million messages, it would be a pity if there are many messages that are never listened to.
Maybe off topic, but the payload functions are still a bit unclear to me. If i understand correclty you can register a payload function per Fport per application? But what if you have 1000 devices of reseller A sending a temperature, and 500 of reseller B sending a humidity and both are sending on Fport 1. How would you then know how to decode the message in the payload function?
OK, we’ll give people the option there; either a raw MQTT bridge or a device registry sync using things and types. That raw bridge would discard the per-field publishes.
Payload functions are indeed per application, and you will be able to take the FPort into account in the Decoder. If you have conflicting device payloads in your application as you illustrate, then payload functions are not the way to go, and you would need to work it out upstream.
That would be a great solution providing flexibility for different types of users of the TTN network.
I agree with you that for complex, large applications the decoding of the bytes is not the responsibility of the LoRA server but of the application developers.
Johan, I would be highly interested in a more direct integration between TTN and AWS IoT (TTN MQTT to AWS IoT MQTT with automatic sync of things/devices) … curious about the status of the integration in the V2 environment, can you provide some more info on this?
I think the creation of an MQTT channel is a good alternative. But what do I need for this? Do I need any additional service or “bridge” running on the external server?
I am working to communicate my TTN applications with AWS IoT.
I think the creation of an MQTT channel is a good alternative. But what do I need for this? Do I need any additional service or “bridge” running on an external server?
I was wondering if anybody had any experience with Amazon AWS IoT.
I want to send my data to AWS IoT and store it in a database.
There is documentation available from Amazon, but it’s very difficult to understand it all.
The guides that you can find online are for very specific use cases or are not free.
You need to use lambdas to then store the data in a database
You can have a look at IoT platforms such as Caynne MyDevices which simplify
NOTE: IoT AWS is designed as a piecemeal solution with developers in mind so programming experience is essential to work with this service (it’s also one of the reasons why it’s so cheap - no frills)
Edit: I lie it supports AWS IoT supports HTTP, WebSockets, and MQTT
It supports thing registry synchronization, thing shadow reports on uplink and shadow delta updates as downlink, bridge uplink and downlink messages on AWS IoT MQTT and it reports metrics to CloudWatch.