Remote Storage Integrations

Remote storage services allow our partners to store and retrieve any amount of data, at any time.

Storage Integrations

 Integrating a remote storage service to one of your projects will allow our cloud to write the data we receive to your remote storage. Bluzone aims to integrate with several leading providers. At this time, Bluzone supports Amazon Simple Storage Service (S3)  and Azure Storage

Amazon Simple Storage Service (S3)

Integrating a S3 bucket into our project is extremely straightforward. The following tutorial describes how to integrate a S3 bucket into your project through the Bluzone Portal.

The following steps outline how to configure a Bluzone project to archive data to a S3 bucket. 

  1. Generate IAM credentials for connecting to Amazon S3. Refer to the Setting up for Amazon Simple Storage Service for step-by-step instructions.
    1. Select IAM Management Console
    2. Select a user
      1. The user must have, at least, the following permissions for Bluzone to write data to a bucket:

        {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Sid": "VisualEditor0",
                    "Effect": "Allow",
                    "Action": [
                        "s3:PutObject",
                        "s3:ListBucket"
                    ],
                    "Resource": [
                        "arn:aws:s3:::my-example-bucket/*",
                        "arn:aws:s3:::my-example-bucket"
                    ]
                }
            ]
        }
    3. Select "Security Credentials" tab
    4. Click "Create Access Key"
    5. Take note of the "Access Key ID" and the "Secret Key" values, as they are needed for a later step
  2. Create a S3 bucket.
    1. From the Amazon S3 Console click "Create bucket"
    2. Give the bucket a name (remember the name, it will be used later)
    3. Select a Region
    4. Click "Create"
  3. Add the S3 bucket to Bluzone.
    1. Login to the Bluzone Portal
    2. Open the project setting view
    3. Select "Remote Storage"
    4. Create a new configuration by clicking "Create"
    5. Enter the following values:
      1. Storage Provider (Currently, only AWS is supported)
      2. S3 bucket name (This should match the name that you assigned to your bucket through Amazon)
      3. Storage Region (Make sure to enter a region that Amazon supports)
      4. Either you can give AWS Access Key and AWS Secret key or Enable Assume Role, Refer How to create assume role.
    6. Click "Save"
    7. Once you have successfully create a remote storage configuration, it will show up in the stream list with the Enabled status. If you choose to disabled a configuration:
      1. Select the configuration from the grid

      2. Uncheck the "Enable this remote storage" box

      3. Click "Save"

The Bluzone app will take a few minutes to setup the remote storage. Once it's finished, our cloud will begin writing data to your bucket.

Azure Blob Storage

The following steps outline how to configure a Bluzone project to archive data to Azure Blob Storage Containers.

  1. Creating Blob Container.
    1. Create the Azure Blob Storage by following the link Setting up Azure Blob Storage Container for step-by-step instructions.
    2. Once you create your storage account and blob container, your azure cloud portal screen will be displayed like below.

              

    3. Take a note of Storage account name, container name and your storage account key, go to the left pane of the storage account screen click on access key option, you can see the storage account key.

              

  2. Add Blob Storage to Bluzone.
    1. Login to Bluzone Portal.

             

    2. Open the project setting view.

             

    3. Select “Remote Storage” and click “Create” button for new configuration.

             

    4. Create Remote Storage Screen will be displayed.

             

    5. Select Azure from the Cloud Provider dropdown in the screen.
      1. Enter the Blob Container name in Container Name text box.
      2. Enter the Storage Account Name in Account Name text box.
      3. Enter the Storage Account Key in Account Key text box.

    6. Test your storage credentials by clicking on “Test Remote Storage”.
    7. Click Save to enable the remote storage account for Bluzone.

The Bluzone app will take a few minutes to setup the remote storage. Once it's finished, our cloud will begin writing data to your bucket.

Data Processing

The data stored in the S3 bucket is in Protocol Buffer format (https://developers.google.com/protocol-buffers/). Furthermore, the data can be either High Speed Motion Data or Location Event Data. The "protobuf" file that can be used to generate stub code for parsing the message structure of High Speed Motion Data is beaconmetrics.proto, the file necessary for Location Event data is locpb.proto and similiarly file necessary for Temperature and battery data is telemetry.proto (more information regarding the syntax of the .proto file can be found here).

Beacon Metrics Proto File


Location Data Proto File

Telemetry Data Proto File


Example usage in Java

To generate the classes you'll need to read BeaconMetrics messages you need to install the protocol buffer compiler, protoc. See here for detailed installation instructions: https://github.com/google/protobuf, e.g.

Install protoc
wget https://github.com/google/protobuf/releases/download/v3.4.0/protoc-3.4.0-osx-x86_64.zip ~/Downloads
mkdir -p ~/Downloads/protoc 
cd $_ 
unzip ~/Downloads/protoc-3.4.0-osx-x86_64.zip cp bin/protoc /usr/local/bin 
rm -fR ~/Downloads/protoc-3.4.0-osx-x86_64.zip ~/Downloads/protoc

To create a Java class from the proto file run the following command. A Java class will be created, e.g. Beaconmetrics.java.

protoc -I=$SRC_DIR --java_out=$DST_DIR $SRC_DIR/beaconmetrics.proto

Once you have the Java class you can begin parsing BeaconMetric files stored in Protocol Buffer Format. For example,

BeaconMetrics.BeaconRawMetricMap beaconRawMetricMap = BeaconMetrics.BeaconRawMetricMap.parseFrom(new FileInputStream("<FILE_PATH>");
beaconRawMetricMap.getResultsMap();

Alternatively, you can use protoc to parse the data. For example,

 protoc --decode BeaconRawMetricMap beaconmetrics.proto < ~/Downloads/2017-09-29.proto.gz


Sample Command to Parse Telemetry Data files,

 protoc --decode BeaconTelemetryMap telemetry.proto < ~/Downloads/<your-file-name>.proto.gz


For a detailed tutorial on Protocol Buffers go here.