Visualizing AWS IoT Button data (part II)

The IoT button is connected to the AWS cloud and every click is recorded in the DynamoDB. It’s now time to query the database and present the data, preferably in a visual form. Here, we will do everything in the browser, building our app using JavaScript AWS SDK. Thus, the first step is to add the script to the HTML page:

<script src="https://sdk.amazonaws.com/js/aws-sdk-2.5.4.min.js">
</script>

Configure the SDK

Create and configure user credentials

We would like to send requests to the DynamoDB, but the SDK has to be configured before we can do that. The two required pieces of information are the region and credentials. There are several ways to provide credentials securely:

  1. Using Amazon Cognito to authenticate users
  2. Using web identity federation to authenticate users
  3. Hard-coded in your application

We are going to pick the easiest one and hard-code the credentials into the application. This is also the least desirable method from the security standpoint, used here for the sake of convenience and speed. To do so, open the Identity and Access Management (IAM) console, click on Users menu and Create User. Immediately after the user is created, you can copy (or download) the security credentials and hard-code them into the application:

var user_credentials = {
        accessKeyId: 'AKIAIV6CXXXXXXXXXXXX', 
        secretAccessKey: 'i8+vtmKkXXXXXXXXXXXXX'
};

Allow read-only access to DynamoDB

This newly created user still does not have any service permissions and thus can’t perform any actions. We need to attach a specific policy to allow DynamoDB access. Click on the Users name in the IAM, select the user, open Permissions tab and click Attach policy. Type AmazonDynamoDBReadOnlyAccess, select it and click Attach policy. We are now officially allowed to query the DynamoDB. Permissions are set as read only to mitigate the security risk due to hard-coding the credentials.

Let’s finish configuring the JavaScript SDK by executing a command to update user credentials:

AWS.config.region = 'us-east-1';
AWS.config.update(user_credentials);

List DynamoDB tables

To test the newly created user credentials and verify the read-only access to Dynamo, lest first list all Dynamo tables. This is done by calling the listTables() method of the DynamoDB object. Let’s also put all tables into the drop-down menu for easy selection.

var dynamodb = new AWS.DynamoDB();
dynamodb.listTables(function(err, data) {
 var html_str;
 if (err == null){
   html_str = ' DynamoDB tables: <select>';
   $.each(data.TableNames, function(index, value){
     html_str += '<option value="' + value + '">' + index+1 + '. '+ value + '</option>';
   });
   html_str += '</select>';
  }
  else{
   console.log(err);
   html_str = err;
  }
  $('#table-list').html(html_str); 
 });

If the credentials are invalid, we will see an error message. If the credentials were crated properly, we can select the table iot-buttons from the menu (that’s the name we decide on in the first part).

Display table info

Let’s now obtain table info by calling describeTable() method and passing the table name as parameter. For example, let’s get the total number of records in a table:

var params = {
   TableName: $(this).val()
};
dynamodb.describeTable(params, function(err, data) {
   if (err) {
      //an error occurred 
       $('#table-list').append('<p>' + JSON.stringify(err) + '</p>');
    }
    else {
       //successful response
       var table = data.Table;
       $('#table-list').append('<p> '+table.TableName + ": # of items= "+table.ItemCount + '</p>');
    }
 });

Query the table

Finally, let’s query the table to get the data for a particular sensor. Device serial number serves as the primary partition key. Let’s obtain all data from an IoT button with a given S/N. First, we need to create a DocumentClient object:

var docClient = new AWS.DynamoDB.DocumentClient();

We need to pass query parameters to the query() method of this object. This includes the  deviceS/N:

var device_sn = 'G030JF05XXXXXXXX';
var params = {
    TableName : table_name,
    KeyConditionExpression: "#device = :dev_sn",
    ExpressionAttributeNames:{
       "#device": "device_id"
    },
    ExpressionAttributeValues: {
       ":dev_sn":device_sn
    }
 };

Finally, we can get the data from the table:

docClient.query(params, function(err, data) {
   if (err) {
      //output error message if query failed
      $('#query').append('<p> Unable to query. Error: </br>'+JSON.stringify(err, null, 2));
   } 
   else {
      $('#query').append(JSON.stringify(data.Items));
      //data.Items now contains the array with returned item
    } 
 });

After the successful query, returned data.Items  contains all returned items. The variable structured as an array of JSON object, each corresponding to a row in the DynamoDB table:

data.Items = [Object, Object,... Object]
Object = {"data":
             {
                "serialNumber":"G030JF05XXXXXXXX",
                "clickType":"SINGLE",
                "batteryVoltage":"1704mV"
             },
             "timestamp":"1472127527955",
             "device_id":"G030JF058432MLDB"
          }

The only thing left is to represent the data in a convenient way.

Process and visualize the data

There are many possible ways to process and visualize the data. For example, one can parse it and represent as HTML table. The voltage and click type can be represented as time-series using one of many available charting libraries.

I wrote two classes for data processing. The first one process data as a HTML table and then adds more functionality to it using dynatable jQeuery plug-in. The second one plots clicks and voltage vs time using plot.ly JavaScript.

To take advantage of these, add the following scripts:

<script src="http://cdnjs.cloudflare.com/ajax/libs/numeral.js/1.4.5/numeral.min.js"></script>
<script src="http://momentjs.com/downloads/moment.js" type="text/javascript"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/Dynatable/0.3.1/jquery.dynatable.min.js" type="text/javascript"></script>
<script src="https://cdn.plot.ly/plotly-latest.min.js"></script>

<script src="plotly_scatter.js" type="text/javascript"></script>
<script src="tableprocessor.js" type="text/javascript"></script>

Scripts table_processor.js and plotly_scatter.js  require moment.js and numeral.js to propely process the time-stamp data.

Pass the acquired data into the new FH.dataprocess object together with a jQuery object for the output HTML div, and the script takes care of tables and plotting:

new FH.dataprocess(query_data, $('#processed-data'));

To see the details of the data processing scripts, download the repository on GitHub. If your AWS IoT and DynamoDB endpoints are set-up correctly, this sample code will produce IoT button voltage and click time-series plots like the one below.