Sorry for the vague title, but I hope this can be a good discussion of options for analyzing and visualizing data. Spoiler alert: I’m fairly new to AWS, so I can see high level possibilities, but don’t understand enough about implementation to know which direction to go. In addition, while I may be able to make things work, I’m also interested in learning better ways to do things.
For reference, I’m working on “yet another vineyard project”: collecting weather and other data from tens of sensors on roughly a minute schedule. I’ve been pleased by the ease of setup and the general reliability of TTN. I can make some graphs (see below), but ultimately, I’d like to make spatial graphs and other things beyond simple line graphs. AWS QuickSight seems like a good platform for this, but that’s just one option is a sea of possibilities, I’m sure.
Here’s what I have done:
- I’ve created an AWS IoT integration and even actions to send the data to a DynamoDB. I can see the data in the tables, but I don’t know where to go from there.
- I’ve created an HTTP integration that sends data to a LAMP stack on an EC2 instance and stores data in a MySQL database. From that, I can make basic line graphs (temperature over time) using highcharts.
But that’s where I’m stuck. Every web search brings up a different solution, so I’d like to get opinions.
- QuickSight has easy integrations with RDS, but not (amazingly) DynamoDB. So getting the data into RDS seems like a good idea. I don’t have so much data that I need a hugely scalable NoSQL database.
- To get data into RDS, I see two options:
- Keep doing essentially what I’m doing with the HTTP integration, but connect to RDS instead of the MySQL database on my virtual EC2 instance. I’ve done a tutorial on that – it mostly makes sense but I need to sit down and figure out security groups.
- Implement a lambda function to send data directly to RDS. This sounds like the most “pro” option, but is also the least understood by me (remedial python knowledge doesn’t help).
Any thoughts or recommendations? Feel free to proselyte your favorite solution or just lay out some of the pros and cons. I’ve looked through some web pages, but I’d be happy to hear from you all.
