marcoz
(stopped with active lora development and left ttn)
1
I am little frustrated with the new platform. On the old platform i was just able to get all information from all gateways and nodes which i used to create a nice monitoring system to see the coverage in e.g. Flevoland. I do not own any equipment so i relied on equipment from others to get the data, they shared me the AppKey, With the new approach it looks like i cannot get this data anymore. I am just wondering why we really need to be so paranoic and close all these interfaces. To me the encrypted data part is already sufficient to prevent that someone sees the actual data. I hope that in the short future there will be a solution to get all data via MQTT again. If not then my application server is a dead project.
Using ttnctl the other parties can share access to their applications using:
$ ttnctl applications authorize --help
ttnctl applications authorize lets you authorize a user for an application.
Usage:
ttnctl applications authorize [eui] [e-mail] [flags]
This way you are still able to get data send by devices registered to that application.
Being able to see a node is transmitting data and with what interval is a security/privacy concern in itself. If a node is used to signal movement of an object or a button press the presence of a packet on the network provides information as well. So I’m glad we have the option to share if we want and keep private if we don’t.
2 Likes
marcoz
(stopped with active lora development and left ttn)
3
Thanks for sharing, Still à pitty that the data is getting so restricted. For my application iT means that for every gateway i need to get permission from the gateway owners too. The gateway info is used in my application to show the coverage area.
could you explain why gateway info is restricted also? Most important for me are just the gps coordinates and the eui. This way i can calculate the distance between the ‘authorized’ node and the gateway. See my pictures before.
No, gateway owners cant give permission. I own two gateways in Almere and they just pass traffic, I don’t have a clue what data.
The application owners need to give you permission. But still this isn’t a very solid basis for a monitoring system you require readable data per node.
What exactly is it that you want to monitor? Gateway uptime? Coverage?
That you still have, for gateway(s) that received a packet from such node. If you’re using TTN’s MQTT, you’ll get that in the metadata array in the JSON message.
marcoz
(stopped with active lora development and left ttn)
6
Thanks for the info. I will think it over to see if irs worthwhile to continue with the application. If i had know before that TTN will become like a real KPN like environment ( the API and MQTT were just open from the start) then i would not have started i quess.
I think we can close the discussion.
Marco, are you planning to pop into our next monthly meetup? We have yet to announce it via Facebook and the mailinglist, but it has been planned for 24/5/2016: http://www.meetup.com/StartupAlmere/events/230870979/
Unfortunately I won’t be there, but there will be plenty of people to discuss your application. I
m sure there are solutions, don’t stop now. Btw: it was clear from the start that TTN would be secured, the Croft API was just open as a demo environment. No one would be using TTN for serious applications if all the data would be there for anyone to grab, would they?
I think it’s clear for everyone the payload will be encrypted in real usage, but the question is about the metadata (in particular gateway status updates).
In theory it’s probably possible to get an overview of (large parts of) the network, that is, the gateways. That would also be very useful for status monitoring, debugging and the community platform. For scalability reasons though there won’t be a central point anymore passing all data, so there might be a solution in subscribing to multiple known public routers. I’m not sure what the current plans are for monitoring and what part of the (meta)data will be public, so maybe @johan / @htdvisser can give some pointers here.
Message payloads will always be encrypted. Using the default keys will be discouraged and eventually blocked (because it will break routing).
Messages will only be delivered to the intended application. There are no plans to support any kind of public “firehose”. If you want to make your data public, you have to do that yourself, or authorize others to your application with the solution @kersing proposed.
Messages will contain some metadata from the gateways that received the message (gps coordinates, signal strength etc.)
Gateway stat messages will not be publicly available over MQTT. We absolutely want to make it possible to see some stats about gateways and coverage, but this won’t be in the same way as the old backend did. @turiphro already mentioned that the distributed nature of the new backend makes it difficult to get statistics from one central point. Another issue is that gateway owners currently can’t control what information about their gateway is publicly available. We want to give them this control before we can start publishing gateway information.
As @arjanvanb already explained, you can get GPS coordinates of the gateways (if available) and signal strength from the metadata attached to the uplink messages. You can use this data to estimate the location of a node.
If you want to get a more accurate location of your device, add GPS to your node
1 Like
marcoz
(stopped with active lora development and left ttn)
10
I think that if you want to do some planning of iot nodes in the field it would be interesting if that area is covered by any gateway. I was using the gps coordinates of the gateway to see where there are gateways in the neighbourhood and to see if we have a good signal. We did some war driving in the past so we could draw up a picture i showed before where we calculate the distance of the node towards the gateway and the snr ratio which belongs to that packet. I will have a look if i can use the info you all provided. for the record, i understand the need for security, only disagree that the gateway information is not public available. Anyway thanks for your input
Indeed, for quite some time now http://noc.thethingsnetwork.org:8085/api/v2/gateways provides some statistics, and can also be REST-like queried for a specific gateway. Of course, this only applies to gateways, and does not supply as many details as in the early days, like htdvisser already explained May 5th 2016 above.
However, that API is not official, I think. March 2017:
Meanwhile, development of the Network V3 Stack is in progress for which I guess one could deploy one’s own Gateway Server and route, say, all gateways for some community through that server to get many more details about those. Also the V3 Console will be open source, maybe making the APIs it uses suitable to reliably get data for one’s own gateways.
The v3 stack will not contain a NOC component as we have it now. A centralized infrastructure that analyzes, stores and aggregates traffic just doesn’t scale.
Instead, for v3 we are spending a lot of time on designing and implementing the infrastructure and APIs for hooking into event streams of network clusters. These APIs would allow you to subscribe to (for example) gateway events, such as the gateway connecting/disconnecting to the server, sending uplink/status messages or receiving downlink messages. With sufficient rights you may even hook into gateway traffic directly to analyze it. These APIs allow developers to build their own stream/long-term analytics tools on top of TTN APIs. Those tools could then work with different public community clusters, but also with private clusters.
