Visualizing AWS IoT Button data

I recently wrote two introductory posts about configuring the Amazon IoT button and using AWS SNS to receive SMS and email notifications on click.
To log the clicks and battery voltage over time, the payload sent with every click can be time-stamped and stored in a database. A script can be used then to query the database, retrieves the data and plot the time series of this data (using one of many d3js libraries, for example, plot.ly). (Will write about the script later in this post)
Some posts about the Dash button speculated that the battery would last for over 1000 clicks. So far, after >150 clicks the battery voltage shows no signs of decreasing.


At a starting point, we have a fully configured Amazon IoT button with uploaded certificates. We tested that it works and can send SMS notifications on click. We know that the button publishes the following message (payload) in the ‘iotbuttons’ topic of the IoT message broker:

{
  "serialNumber": "G030JF058432MLDB",
  "batteryVoltage": "1739mV",
  "clickType": "LONG"
}

We need to time-stamp this data and store it in a database. Let’s use Amazon’s NoSQL DynamoDB database to store and query the time-series data.


Create new DynamoDB table to store data

Go to the DynamoDB dashboard and click Create table button. The table requires a name and a primary key. Let’s name the table iot-buttons (we might need to log more than one button for a potential application). While the name choice is rather arbitrary, the partition key is important – elements with different partition keys should have uniform access frequency (more about partition keys). In our case, the serial number is a good partition key. Additionally, we can use a timestamp as a sort key to search within a partition. Both the serial number and the timestamp are strings. After specifying the keys, we can create a table using default remaining settings.

iot-buttons
device_id (String)
timestamp (String)

Create IoT rule

With the table created and keys specified, we now can create a rule in the AWS IoT console to write the data into DynamoDB on click. Clicking on the Create resource and choosing the Create a rule  opens a dialog window for the new rule. Name and Description fields are again arbitrary. The SQL query should capture all messages sent by (multiple) buttons to our MQTT broker without additional conditions. This is achieved by:

SELECT * FROM 'iotbutton/+'

We need to select the DynamoDB Action from the drop-down menu to insert  messages into a database. This selection extends the dialog by several fields. First, choose a table name  from the drop-down list (there is also an option to create new table, but we have taken care of this already). We decided to use the serial number as a hash key. It is transmitted in the “serialNumber” field of the payload, so we will specify ${serialNumber} as a hash key value. To specify time-stamp as a range key value,  type ${timestamp()} in the appropriate field. Finally, the payload field can be just data, meaning that the entire payload will be stored in the database.

The last step is to specify the role. Luckily, we can click on Create new role and simply specify role name. IoT interface will take care of the rest and create a new IAM role which allows writing data into a particular database.

Click Add action to add this DynamoDB action to the new rule (a rule can have several actions) and Create rule to complete the process. Pressing AWS IoT button now should create a new record in the DynamoDB table with the following content:

dataMap{3}
         batteryVoltage String: 1760mV
         clickType String: LONG
         serialNumber String: G030JF05XXXXXXXX
device_id String: G030JF05XXXXXXXX
timestamp String:1474212345678

Every click is now recorded into the database. Let’s now look at the ways to retrieve the data and visualize voltage and clicks as a function of  time.


To be continued…

Mapping the NSF funding data

NSF Funding levels:  funds obligated and estimated max per year (1996 - 2015)The National Science Foundation funds ~ 1/4 of all federally supported basic research, most of it in physical science and math. As a PhD student in physics,  I am supported in part by the NSF grants awarded to my PI and Brandeis University MRSEC. How does current level of  basic research funding compares to historical data? All the funding data by the government agencies like NSF is publicly available, but not easy to digest and analyze. I created a simple web-page that allows one to search NSF funding by PI,  institution or state. It also makes visualizing funding data on the state or city level very easy with interactive maps.

I looked up all NSF funding from 1996 through 2016 and sorted it by city and state. Looks like the amount of NSF funding peaked in 2009 and has been decreasing recently. Also there are two visible spikes in funding levels, in 2002 and 2010. I mapped the funding levels by city using plot.ly interactive maps. Here is one such map for 2015. You can find the rest of them on my plot.ly page.

2015 NSF and NASA Funding ($ Millions) - US Cities
2015 NSF and NASA Funding ($ Millions) – US Cities