Thursday, November 20, 2014

Sequence Control Using Arduino

I am considering Arduino as one of the method to making controller programming easier.

Nowadays it seems that everyone is using Arduino, you can even see elementary school kids programming using Arduino.

Accessibility of Arduino is incredible.
Leaving performance out of discussion, I have never seen anything like Arduino, which makes programming sooooo easy, in my lifetime. 

Thus, I started to think "what if I make controller using Arduino...?"

MPU and libraries used in Arduino are good enough to use for controller.
This is very obvious because ATmega328 that are being used for Arduino are already widely used for controller.

However, MPU is too exposed in Input/Output circuit. CANNOT BE USED
Also, it seems that designer has not thought of any power system. CANNOT BE USED

If you think about it in another way, you can use Arduino as controller if you make modification to Input/Output circuit and power system.

Modifying Arduino as whole will be too tiresome so I am suggesting a following experimental changes.

Input/Output Circuit -> Remote IO Module for FA
Power System -> Power Supply for FA

This is the first example.

I have tried Feedback Control and Sequence Control using Arduino.

The controlled system is Drilling Machine.

Drilling Machine

Drilling Machine Structure



Machine is composed of 3 parts.


PRESS

Drilling Machine Press - holds samples that are going to be drilled firmly

DC 24V Motor with reduction gear and worm gear drives metal structure to move up and down with powerful strength.

Top part has a limit switch that decides the stop location. However bottom part does not have limit switch. Location of bottom limit changes depending on the thickness of the sample, thus it is impossible to put limit switch.

Instead of limit switch, we measure the load on the press motor to decide the bottom limit. We need to constantly measure the current in the press motor and stop the motor when the current skyrockets. This step requires lots of site tuning.


ELEVATOR

Drilling Machine Elevator - holds drill motor

This also has a DC 24V Motor with reduction gear and rack and pinion gear drives drill motor to move metal structure up and down.

It has limit switch at both top and bottom.


DRILL

Drilling Machine Drill

On platform, there is big motor with AC 220V. On the axis there is drill bits.
The motor load needs to be monitored all the time, because if the sample does not get drilled easily because it is too firm the drill bits needs to move up and down multiple times and also because if this does not work the drilling needs to be stopped.
To measure the motor current, the motor power line wraps CT twice.



Arduino Interface


For direct connection with the machine I am going to use ADAM-4050 and ADAM-4017, which are both made for industrial purposes. ADAM-4050 has 8 channel Digital Out and 7 channel Digital In and ADAM-4017 has 8 channel Analog In. Both use RS-485 connections.

Below is channel list.

ADAM-4050

DO0: Elevation Motor ON
DO1:                         Direction(+)
DO2:                         Direction(-)
DO3: Press Motor       ON
DO4:                         Direction(+)
DO5:                         Direction(-)
DO6: Drill Motor        ON
DO7: Fan Motor         ON

DI0: Limit Switch Position UP
DI1:                                 MID
DI2:                                 BOTTOM
DI3: Manual Switch           START
DI4:                                 STOP
DI5:
DI6:

Elevator Motor and Press Motor both moves up and down, which means it is controlled both ways thus it needs 3 ch per motor. Drill Motor is one way, since it only needs to start the motor. Fan Motor is 24V DC. It also only needs to be turned on. It sends air to the drill bits cooling it down. Fan Motor is kept ON for 30 seconds even after the drilling.

We use three limit switch: PRESS' midpoint, top, and bottom.

There are two switches for manual control. One for starting and another one for stop.


ADAM-4017

VIN0: Elevation Motor Current
VIN1: Press Motor Current
VIN2: Drill Motor Current
VIN3:
VIN4:
VIN5:
VIN6:
VIN7: 

It reads current flowing in each motor.
Elevation Motor and Press Motor puts shunt resistance on the motor drive and measures the current. For Drill Motor we use CT sensor to read the converted current value. Readings from CT sensor are AC, thus it shows value that goes through full-wave rectifier and integrated using 60Hz LPF. In this case we only need to know the relative value of the load thus we do not measure the RMS value.


Below are connection diagram for ADAM-4050/ADAM-4017 and the machine.
(Detailed circuit diagram for the machine is company's asset thus it is not released publicly; however, if you really need the diagram please contact privately.)

ADAM-4050/ADAM-4017 and the Machine Connection Diagram

OR

ADAM-4050/ADAM-4017 and the Machine Connection Diagram

OR

ADAM-4050/ADAM-4017 and the Machine Connection

ADAM-4050/ADAM-4017 and the Machine Connection


Sequence Control

STANDBY

At standby PRESS and ELEVATOR are all resting at the top. All motors are switched OFF and waits for the START switch to be ON.


PRESS DOWN

START switch is ON. Until one of the stop condition is satisfied run the Press Motor downward.

Stop Conditions:
  1. If the Press Motor Shunt current value has passed 120% of moving average line.
    Moving average line is the calculated average value before the press reaches the sample. When the press starts to push down on the sample the motor current rapidly increases and when it exceeds 120% it should stop.
  2. If the Press Motor Shunt voltage has passed 3V.
    This is FAIL-SAFE. For some reason if there are too much pressure pushing down on the sample with large moving average line the Press should stop.
  3. If it has been over 17 seconds after the press motor has started working.
    This is also FAIL-SAFE. The press should not be working more than 17 seconds.


ELEVATOR DOWN

Now we start to lower the elevator that holds the drill. Turn ON the Drill Motor and Fan Motor. Until one of the stop condition is satisfied run the Elevator Motor downward.

There is only one stop condition:
  1. If the elevator has reached the bottom point.
    Bottom limit switch decides this. The drill bits have successfully penetrated the sample.

For the following cases, we higher the drill for a little bit and lower it again. When it does not drill well it is better to move up and drill again rather than continuously trying to drill. 
  1. If the Drill Motor's current rises more than 7% of the moving average line.
    Raise the drill (raise elevator) until the current falls below the 7% line and then lower the drill (lower elevator) again.
  2. If the Drill Motor has been on for more than 17 seconds.
    Move the drill up until the top line (raise the elevator) and then lower (lower the elevator). This is the same as restarting. Thus set the maximum of this action to 6. If it exceeds 6, we can safely conclude that the sample is impenetrable.

ELEVATOR UP

Drill work has finished whether it succeeded or not. Let's quickly put the elevator back to where it was.

Run the Elevator motor in UP mode until it reaches the top point. Keep the Drill Motor ON. If the Drill Motor has been turned OFF it might get stuck in the sample thus preventing it from being raised.

When it reaches the top point, turn both Elevator and Drill Motor OFF.


PRESS UP

Run the Press Motor in UP mode until it reaches the MID point.
When it reaches MID point, turn the Press Motor OFF. Leave the Fan Motor ON for extra 30 seconds.


In any stages, if the STOP switch is ON, go back to STANDBY mode.



SOURCE CODE

Source code can be viewed in the following site.
https://github.com/michelleseo/Arduino_Web/blob/master/SequenceControl/MicroFactory.ino

Explanation of code will be difficult since there are too much. If necessary I will answer any questions.



The following is the working video.


Thursday, June 19, 2014

DHT Subscriber - MQTT Arduino Gateway

Use WMQTT Utility to Subscribe Uploaded DHT-11 Temperature & Humidity Data


Download WMQTT Utility

http://www14.software.ibm.com/cgi-bin/weblap/lap.pl?popup=Y&li_formnum=L-TMAN-5N7JL9&accepted_url=ftp://public.dhe.ibm.com/software/integration/support/supportpacs/individual/ia92.zip
Dowload ia92.zip. No need to install, you just need to unzip.

For example, if you unzipped in C:\Evaluation\preview\MQTT\ia92 in DOS Box
cd C:\Evaluation\preview\MQTT\ia92\J2SE
java -jar wmqttSample.jar

Manual is in page 49 of IBM MQTT RedBook. (The page number refers to PDF Viewer's page)


Subscribe

Broker TCP/IP address - broker.mqttdashboard.com
Connect, if green light turns on,
Subscribe Topic - DHT11/temphum
and you will see DHT-11 Temperature & Humidity data in History


Subscribe Using WebSocket

For full source code refer to the link at the bottom.

If you want to see only the MQTT part -> https://gist.github.com/matbor/8837988



<!DOCTYPE html>
<html>
  <head>
    <title>Temperature & Humidity Subscriber</title>
    
    <script type="text/javascript" src="jquery-2.1.1.min.js"></script>
    <script type="text/javascript" src="globalize.min.js"></script>
    <script type="text/javascript" src="dx.chartjs.js"></script>
    <script type="text/javascript" src="knockout-3.0.0.js"></script>
    <script type="text/javascript" src="mqttws31.js"></script>
    
    <script type="text/javascript">
        var client;
        var t, h;

        $(document).ready(function() {
            t = $('<div style="width: 50%; height: 100%; float: left;" />').appendTo('#chartContainer');
            h = $('<div style="width: 50%; height: 100%; float: left;" />').appendTo('#chartContainer');
            var options = { scale: { startValue: 0, endValue: 40, majorTick: { tickInterval: 5}} };

            t.dxCircularGauge($.extend(true, {}, options, {
                geometry: { startAngle: 180, endAngle: 90 },
                value: 0,
                rangeContainer: {
                    ranges: [
                      { startValue: 28, endValue: 40, color: '#CE2029' },
                      { startValue: 0, endValue: 10, color: '#0077BE' }
                  ]
                },
                valueIndicator: { type: 'rangeBar' },
                title: { text: 'Temperature' },
                scale: {
                    label: { customizeText: function(arg) {
                    return arg.valueText + '&deg;C';
                    }
                    }
                }
            }));

            h.dxCircularGauge($.extend(true, {}, options, {
                scale: { startValue: 100, endValue: 0, majorTick: { tickInterval: 10 },
                    label: { customizeText: function(arg) {
                        return arg.valueText + '%';
                    }
                    }
                },
                geometry: { startAngle: 90, endAngle: 0 },
                value: 0,
                valueIndicator: { type: 'rangeBar' },
                rangeContainer: {
                    ranges: [
                        { startValue: 33, endValue: 0, color: '#77DD77' },
                        { startValue: 67, endValue: 33, color: '#E6E200' },
                        { startValue: 100, endValue: 67, color: '#92000A' }
                    ]
                },
                title: { text: 'Humidity' }
            }));
        });

        client = new Messaging.Client("broker.mqttdashboard.com", 8000, "HTMLTempHumSensor" + parseInt(Math.random() * 100, 10));

        // Connection Lost Message
        client.onConnectionLost = function(responseObject) {
            alert("Connection Lost: " + responseObject.errorMessage);
        };

        // Message Arrival
        client.onMessageArrived = function(message) {
            var data = message.payloadString.split(",");
            var temperature = parseInt(data[0]);
            var humidity = parseInt(data[1]);
            $(t).dxCircularGauge('instance').value(temperature);
            $(h).dxCircularGauge('instance').value(humidity);
        };

        // Connect Options
        var options = {
            timeout: 3,
            onSuccess: function() {
                alert("Connected");
            },
            onFailure: function(message) {
                alert("Connection failed: " + message.errorMessage);
            }
        };

        var publish = function(payload, topic, qos) {
            var message = new Messaging.Message(payload);
            message.destinationName = topic;
            message.qos = qos;
            client.send(message);
        }
        
    </script>
  </head>
  
  <body>
    <div id="chartContainer" style="height:450px;width:1000px;" align="center"></div>
    
    <table width="1000px">
      <tr>
        <td align="right">
          <button type="button" style="width:200px;" onclick="client.connect(options);">
            Connect
          </button>
        </td>
        <td align="center"> 
          <button type="button" style="width:200px;" onclick="client.subscribe('DHT11/temphum', { qos: 2 }); alert('Subscribed');">
            Subscribe
          </button>
        </td>
        <td align="left">
          <button type="button" style="width:200px;" onclick="client.disconnect();">
            Disconnect
          </button>
        </td>
      </tr>
      
      <tr>
        <td colspan="3">
          <div id="messages"></div>
        </td>
      </tr>
    </table>
  </body>
</html>



Full codes can be found in following links





Wednesday, June 18, 2014

DHT Publisher - MQTT Arduino Gateway

Uploading DHT-11 Temperature & Humidity Data using MQTT

Refer to Log Data Upload - Humidity & Temperature Web Logger posting for more information about Arduino & DHT-11.


Download MQTT Publisher Library for Arduino



Choose MQTT Broker

There are several public MQTT brokers; however, we are going to use HTML/JS to see Temperature & Humidity Data thus we need to use WebSocket.
I am going to use broker.mqttdashboard.com because it supports WebSocket.
IP is 212.72.74.21

Other good Brokers:
  • m2m.eclipse.org (IP: 198.41.30.241)
  • test.mosquitto.org (IP: 85.119.83.194)

If you want your own Broker, use Mosquitto.
Install and run mosquitto.exe.


TOPIC NAME

broker.mqttdashboard.com only supports 1 sub-tree, which means that it is possible to use "DHT11/temphum" as TOPIC, but "DHT11/temphum/room" will not work.



For full source code refer to the link at the bottom.
#include <SPI.h>
#include <Ethernet.h>
#include <dht.h>
#include <PubSubClient.h>

#define DHT11_PIN     A0
#define POLLINTERVAL  10000   // 10seconds interval

#define CLIENTID      "ArduinoTempHumSensor"
#define TOPICNAME     "DHT11/temphum"

byte mac[]    = { 0x90, 0xA2, 0xDA, 0x0F, 0x25, 0xC4 };
byte server[] = { 212, 72, 74, 21 };    // "broker.mqttdashboard.com"

dht DHT;

PubSubClient arduinoClient(server, 1883, callback); // We don't use it

char charSensorVal[20];
double humidity    = 0.0; // Sensed Humidity
double temperature = 0.0; // Sensed Temperature
int seq = 0;


void setup() {
  Serial.begin(9600);
  Ethernet.begin(mac);
  
  // Connect to the MQTT Server - broker.mqttdashboard.com
  beginConnection();
}

void loop() {
  if((millis() % POLLINTERVAL) == 0) {
    if(getSensorVal()){
      dtostrf(temperature, 5, 2, charSensorVal);
      charSensorVal[5] = ',';
      dtostrf(humidity, 5, 2, charSensorVal + 6);
      // One line publishing
      arduinoClient.publish(TOPICNAME, charSensorVal);
      Serial.print(seq++);
      Serial.print(":");
      Serial.println(charSensorVal);    
    }
  }
}

void beginConnection() {

  Serial.println("Try connect to MQTT server");    
  
  int connRC = arduinoClient.connect(CLIENTID); // MQTT Broker connection
  
  if(!connRC) {
    Serial.println(connRC);
    Serial.println("Could not connect to MQTT Server");
    
    delay(100);
    exit(-1);
  }
  else {
    Serial.println("Connected to MQTT Server...");
  }
}

void callback(char* topic, uint8_t* payload, unsigned int length) {}

byte getSensorVal() {
  if(DHT.read11(DHT11_PIN) == DHTLIB_OK) {
    humidity = DHT.humidity;
    temperature = DHT.temperature;
    return 1;
  }
  else return 0;
}
Full codes can be found in following links






Saturday, June 14, 2014

Introduction - Heart Rate Monitor by MQTT

The wristband looks splendid, but the accuracy of measurement is apparently crappy.

Do wristband heart trackers actually work? A checkup.
According to the above link, finger tip works the best (Samsung Galaxy S5).

Performance wise we can say: finger tip > chest strap > wristband.
Wristband is is most wearable.
Unless you are planning on doing intense workout chest strap seems inconvenient.
Getting further into it will just be a waste of time.
Thus I'm going to ignore how pretty each things are and get into how each things will be implemented (if we start getting into how to make things more prettier, we will be creating super thimble at some point).



1. Photoplethysmographic Sensor



It calculates heart rate using the change in reflectance of light source as blood pressure changes.
There are plenty of cases relating to this on the internet.

Few examples:
Apparently all the batteries will be used by IR LED.
In normal condition we should constantly increase the measuring rate, and in abnormal condition we are in emergency state anyways.
In that condition battery should not be the problem, thus let's assume that we are going to measure constantly.



2. Bluetooth

Due to problem with battery we will have to use Bluetooth Low Energy (BLE), which only has 10 m long output power ( it will decrease even more because we are using PCB antenna ).
Depending on the situation we might have to use 2 APs.
The protocol will be MQTT-SN.



3. Battery

If we want to use Bluetooth to keep 24/7 watch, it will be impossible to use same battery for months.
Henceforth, when we are picking elegant design while increasing the volume we can use Lithium Polymer (Li-Po) battery.
Or even later on when we are mass producing we can customize the battery shape so that it fits into the case ( This is the ultimate goal ).
Two set of heart rate monitor should be one package, so that when one is being used the other can be charging.
Then it should contain a LED light that will let you know when the charging is needed.




4. MQTT

I'm going to use MQTT, because I don't need to build separate Service Server.
People who have huge interest in someone's heart can just register as subscriber to the broker.
If there is problem, Broker will send an e-mail or SMS immediately.
If there is no message, you could ask Broker.
However, the most simple way will be to call the person directly.




Friday, June 13, 2014

Summary of MQTT

MQTT (Message Queue Telemetry Transport)

The official Website

http://mqtt.org/

The final version of MQTT is 3.1


The final version of MQTT-SN (SN or S, Sensor Network) is 1.2
The client is WSN node (Wireless Sensor Network, like ZigBee), and we need broker like MQTT-S Gateway.

The spec sheet for MQTT is only 42 pages long and MQTT-S is only 28 pages long.

PT Informations to Help you Understand Better

Low Latency Messaging for Mobile Apps or When HTTP and Push Notification are Simply Not Enough
Building WSN with MQTT, RPi & Arduino

MQTT - MQ Telemetry Transport
Connecting Cars with IoT MQTT

Important Documents

IBM MQTT RedBook.  
Building Smarter Planet Solutions with MQTT and IBM WebSphere MQ Telemetry



Tips

.MQTT uses Network Byte / Bit ordering. (MSB is equal to bit<0>)
However, remaining length is in LSB-MSB order.





Thursday, June 12, 2014

HOW? - Arduino Remote Sensor Logger

It's going to cost some money to follow easy and simple way.
I'm going to purchase design kit (or evaluation board) that has been prepared by companies.
Sometimes it's costly; however, it's worth it.


TMP451

Costs $75.
USB Interface can be used on something else later on.




nRF24LE1

It contains 2 sets.
To send and receive I'll need 2.
Also it comes with a lot of things that seems unnecessary.


If we connect it, it'll look something like the following.




The next step...
Try this and that and if you get the feeling that this is the one, make it smaller. 





Wednesday, June 11, 2014

Construct - Arduino Remote Sensor Logger



There is not much difference from Components - Arduino Web Logger posting after adding Arduino.
Ethernet Shield has been replaced for WiFi Shield and RS-485 Shield is replaced by nRF24LE1 Shield.

Let's focus on Temperature Remote Sensor.
It is composed of 3 parts.

  1. Temperature Sensor
  2. RF Sender
  3. Battery

Temperature Sensor

I'm going to use TMP451 from TI.
This is introduction on their site.  http://www.ti.com/product/tmp451
And the following is the datasheet.  http://www.ti.com/lit/ds/symlink/tmp451.pdf


If you look at Power, it doesn't exceed 500 uA.
This is perfect for Battery.



Sending RF

I'm going to use nRF24LE1 from Nordic.
This is the chip that has been used in wireless mouse, keyboard, and remote controller for long time.
For this model, you can program internal 8051, thus you don't need additional Controller.
Power ranges from 1.9 V - 3.6 V, which is perfect for battery. I assume the price will be perfect also.
This is introduction for this model.  http://www.nordicsemi.com/eng/Products/2.4GHz-RF
And the following is the datasheet.  http://www.nordicsemi.com/eng/Products/2.4GHz-RF/nRF24LE1

However, power consumption rate could be a bit better.
If we add more peripherals it will exceed 20 mA.



Battery

I am wanting Battery that will last at least a year.
We are going to use it once in every 10 minutes and it will be used for 10 seconds using 20 mA.
Using simple math:
20 mA * 10 seconds * 6 times / hours * 24 hours * 365 days = 10,512,000 mA / sec = 2920 mAH
Thus we need Battery with size 3000 mAH.
3000 mAH is very common in AA size batteries now.
In Alkaline, DURACELL Normal or Energizer will be good enough.
There are lots of nikel-hydride battery, http://www.nordicsemi.com/eng/Products/2.4GHz-RF/nRF24LE1
However, the size of AA is not that small.  Thus the size of remote sensor will be the size of AA battery.  Of course the weight will be the same as AA battery.





Tuesday, June 10, 2014

Create Graph with DB Data using Chart JS - Humidity & Temperature Web Logger

Let's modify WSDHT1.0.html little bit to create graph.



There is nothing much different from other previous graphs, except that we have two Data and we have two vertical axis.

First do Chart Configuration:

        $(function() {
            $("#chartContainer").dxChart({
                title: {
                    text: 'Temperature & Humidity Graph'
                },
                // Left y-axis: temperature, right y-axis: humidity
                valueAxis: [{ name: 'mainAxis', position: 'left', title: { text: 'Temperature (C)'} },
                        { name: 'secondaryAxis', position: 'right', title: { text: 'Humidity (%)'}}],
                dataSource: chartDataSource,
                legend: { verticalAlignment: "bottom", horizontalAlignment: "center" },
                // Humidity & Temperature have common x-axis: time
                commonSeriesSettings: { argumentField: 'time' },
                // Use 'spline' type to make smooth curves
                series: [{ name: 'Temperature', valueField: 'temperature', point: { visible: false }, type: 'spline', axis: 'mainAxis' },
                     { name: 'Humidity', valueField: 'humidity', point: { visible: false }, type: 'spline', axis: 'secondaryAxis'}]
            });
        });


Then Add Chart Data:

        function graphData() {
            var len, i;
            var chartData = [];
            blockNo = document.getElementById("DataBlockNo").value;
            db.transaction(function(tx) {
                tx.executeSql('SELECT * FROM LogData WHERE block="' + blockNo + '"', [], function(tx, results) {
                    len = results.rows.length;
                    if (len > 42) len = 42;
                    for (i = 0; i < len; i++) {
                        chartData.push({ time: i, temperature: parseFloat(results.rows.item(i).temperature), humidity: parseFloat(results.rows.item(i).humidity) });
                    }
                    observableChartDataSource = ko.observableArray(chartData);
                    var chart = $("#chartContainer").dxChart("instance");
                    var dataSource = chart.option('dataSource', chartData);
                }, null);
            });
        }


Full codes can be found in following links





Monday, June 9, 2014

Log Data Upload - Humidity & Temperature Web Logger

I have purchased DHT11 Humidity & Temperature Sensor from the following site.
http://www.dfrobot.com/index.php?route=product/product&product_id=174#.U4lzu3J_uSq

They deliver only the sensor.  It has Connector and Wire; however, there is no PCB.



BUT, it doesn't have cable to connect to Arduino.
So I had to purchase the Arduino jumper wire set, which I thought was useless. :(
http://www.eleparts.co.kr/EPXCTKGY ( the following site is Korean site. You might have to search for other site in your own country if you need to purchase one. )



Using the correct colors ( it's not a must ), connect it to Arduino.  I attached it to Connector on RTC Shield.

Red cable: +5V Power.  Top left connector, fifth pin from the top.
Black cable: GND.  Bottom left connector, first pin from the top.
Green cable: A0.  Top right connector, fourth pin from the top.

Download library and import it.


If you upload the dht.ino included in the zip file,



  1. Temperature: It was very hot today; however, it looks somewhat correct.  When I get an actual thermometer I will re-check.
  2. Humidity: Humidity calculation is based on Temperature, thus if the temperature is not correct humidity cannot be correct.
  3. Although there is a decimal point, the decimal value is always 0.  If we change to better sensor I assume it will give us the decimal values so I'm going to stick with the float.  Accuracy of sensor is not the point in this posting so I'm going to move on.

In dht.ino change #define DHT11_PIN 4 to #define DHT11_PIN A0, and 
chk = DHT.read11(DHT11_PIN); to int chk = DHT.read11(DHT11_PIN);.


Protocol

I have modified Get Data Block "GETn".
D = <time stamp> + <Temperature> + <Humidity>
We are going to send time stamp, temperature, and humidity all at the same time.



Arduino Code

Following is excerpts from the code.  The entire code can be found in the link at the very bottom.  I modified WSAdam5.ino to create the following code.
#include <dht.h>
... OMIT ...
#define LOG_BLOCK_SIZE 96
... OMIT ...
//-------------------------------------DHT-11
#define DHT11_PIN A0
//#define SAMPLING_INTERVAL (3*1000)

// If you set it to this, it will read sensor value every minute.
// It takes forever to debug if you set it to 10 minutes.
// Arduino default is 16 bit int. If you don't do (uint32_t)
// it becomes -5536
#define SAMPLING_INTERVAL (uint32_t)60*1000
#define MAX_REC_SIZE  42 // 512/12 = 42.666...

dht DHT;
float humidity;
float temperature;
void DHTPoll();
... OMIT ...
void loop() {
 wsServer.listen();
 
 if(sdStatus) {
  if((millis() - lastPollTime) >= SAMPLING_INTERVAL) {
   lastPollTime = millis();
   DHTPoll();
  }
 }
}

/*
  ----------------------------------------------------------------
   D H T - 1 1   h a n d l e r
  ----------------------------------------------------------------
*/
void DHTPoll() {
 if(DHT.read11(DHT11_PIN) == DHTLIB_OK) {
  humidity = DHT.humidity;
  temperature = DHT.temperature;

  if(sdStatus) makeLogBlock();
 }
}
... OMIT ...
void SDbegin(void) {
... OMIT ...
 //load last block
 sdCard.readBlock(currBlock, sdBuffer);
 pB = sdBuffer;

 if(*((uint32_t*)pB) != USED_MARK) { // New data block
  *((uint32_t*)pB) = USED_MARK; // Used Flag
  *((uint32_t*)(pB + 4)) = 0; // for this Logging Block
                              // set the record to 0 and
                              // add 3 bytes padding
 }

 LogBlockSize = sdBuffer[4]; // Used data block
 sdStatus++;  
 printCurrBlock(); //debug purpose
}
... OMIT ...
void makeLogBlock(void) {
... OMIT ...
 //------------------------- push Data into Buffer
 pB = sdBuffer + 8 + (LogBlockSize * 12);
 now = rtc.now();

 *((uint32_t*)pB) = now.unixtime(); // Time-stamp 
 *((float*)(pB + 4)) = temperature; // Temperature
 *((float*)(pB + 8)) = humidity; // Humidity
... OMIT ...
 //------------------------- write to SD card everytime
 LogBlockSize++;
 sdBuffer[4] = LogBlockSize;
 sdCard.writeBlock(currBlock, sdBuffer);

 //------------------------- If the Data Block is complete, 
 //           fix the Free block location and reset Buffer
 if(LogBlockSize >= MAX_REC_SIZE) {
  currBlock++;
  update_currBlock();

  pB = sdBuffer;
  *((uint32_t*)pB) = USED_MARK; // Used Flag
  *((uint32_t*)(pB + 4)) = 0; // Set this logging Block's record
                              // to 0 and add 3 bytes padding
  LogBlockSize = 0;
 }  
... OMIT ...
void upload_DataBlock(uint32_t reqBlock) {
... OMIT ...
 cnt = *pS;
 pS += 4;
 printLn[0] = 'D';

 while(cnt--) {
  // time stamp (seconds)
  ltoa(*((uint32_t*)pS), printLn + 2, 10);
  pC = printLn + strlen(printLn);
  *pC++ = '=';
  pS += 4;
  
  // Temperature
  dtostrf(*((float*)pS), 6, 2, pC); // -12.34C
  pC = printLn + strlen(printLn);
  *pC++ = '=';
  pS += 4;

  // Humidity
  dtostrf(*((float*)pS), 6, 2, pC); // 12.34%
  pS += 4;
  WSsend(printLn, strlen(printLn));
 }

 printLn[2] = '0'; //timestamp == 0, end-of-data
 WSsend(printLn, 3);
... OMIT ... 


HTML / JS Code

This code shows how to upload data using Web Socket, and how to create Web DB using the uploaded data.  Codes below are only part of the code.  If you want to see the entire code look at the link at the very bottom.
This code is modified from WSAdam6_5.html.

Clicking [GetDataBlock] button will display the Temperature & Humidity Data in DB.



... OMIT ...
      var db = openDatabase('WebLogger10', '1.0', 'Web Logger', 512*1024);

      $(document).ready(function() {
        WebSocketConnect();
        db.transaction(function (tx) {
          tx.executeSql('CREATE TABLE IF NOT EXISTS LogIndex(block, timeStamp)');
          tx.executeSql('CREATE TABLE IF NOT EXISTS LogData(block, timeStamp, temperature, humidity)');
        });
      });
... OMIT ...
      function WebSocketConnect() {
... OMIT ...
            // receive Data Block
            if(ar[0] == 'D'){
              data[rcvCnt++] = ar[1];
              data[rcvCnt++] = ar[2];
              data[rcvCnt++] = ar[3];
              if(ar[1] == '0'){
                $("#recordCntPara").empty();  
                $("#recordCntPara").append('Received:' + rcvCnt);
                if(blockNo != 0){
                  db.transaction(function (tx) {                  
                    if(rcvCnt>126) rcvCnt = 126;
                    rcvCnt /= 3;
                    $("#recordCntPara").append(',Record Count:' + rcvCnt);
                    for(i=0;i<rcvCnt;i++){
                      tx.executeSql("INSERT INTO LogData VALUES(?, ?, ?, ?)", [blockNo, data[i*3], data[i*3+1], data[i*3+2]]);
                    } 
                  });
                  db.transaction(function (tx) {
                    tx.executeSql("INSERT INTO LogIndex VALUES(?, ?)", [blockNo, data[0]]);
                  });
... OMIT ...
      function viewData() {
... OMIT ...
            for(i=0;i<len;i++){
              timeStamp = results.rows.item(i).timeStamp;
              tod.setTime((timeStamp - (9*3600)) * 1000);
              $("#dumpPara").append('<p>' + tod.toLocaleString() + '  ' + results.rows.item(i).temperature + 'C ' + results.rows.item(i).humidity + '%<p>');
            }
... OMIT ...



Full codes can be found in following links





Sunday, June 8, 2014

Construct - Humidity & Temperature Web Logger

This time I'm going to create Humidity & Temperature Logger to outline how easy it is to create Web Logger.



Humidity & Temperature Sensor

I have used DHT11, the most common Arduino Humidity & Temperature sensor.
http://www.dfrobot.com/index.php?route=product/product&product_id=174#.U4--irkU9aR
The above site has many more interesting gadgets to look at.

DHT11 datasheet can be downloaded from following site, or it can also be found in the link at the bottom.
http://www.micro4you.com/files/sensor/DHT11.pdf

Item Measurement Range Humidity Accuracy Temperature Accuracy Resolution Package
DHT11 20-90% RH
0-50°C
±5% RH ±2°C 1 4 Pin Single Row

Accuracy for both Temperature & Humidity sucks; however, DHT11 is easy to purchase and connect.  I will try working with higher accuracy model next time.

It says Resolution for both are 8 bit; however, if you look at the data format it is a different story.

Data Format: 

  • 8 bit integral RH data 
  • + 8 bit decimal RH data 
  • + 8 bit integral T data 
  • + 8 bit decimal T data 
  • + 8 bit check sum

If we start thinking about decimal points it gets confusing when coding.  Thus we are going to use floating just like we did for ADAM-4017.




Data Block Components

It is highly unlikely Temperature & Humidity changes drastically.
For this sensor, the response time for humidity is 15 seconds and temperature is 30 seconds ( in data sheet ).
I am going to measure every 10 minutes.

If we measure every 10 minutes, it is going to take multiple hours to fill up 1 sector.  And if for some reason if you decide to reboot, you are going to lose hours worth of Data.
This is very unwanted situation, thus we are going to write to SD every time we measure.
Add time stamp to each measured data.


  • Used Flag: 4 bytes
    Set it to 0xaa55a55b
  • Data Count: 1 byte
    Number of Data in sector. 0, 1 - 42
  • Extra: 3 bytes
    Fill it with Null.
  • Data: 12 bytes
    42 will fit in 1 Data Block (8 + 42 * 12 = 512)
    • Time Stamp: 4 bytes
      Use only sec part of POSIX time_t.
      Must *1000 because we are not using millisecond part.
      It is 32 bit unsigned integer.
    • Temperature: 4 bytes floating
    • Humidity: 4 bytes floating

Each sector will contain 42 * 10 min = 420 min = 7 hrs worth of data.
Each Data has its own time stamp, thus sector is not ordered in time or date.



Full codes can be found in following links




Saturday, June 7, 2014

RTC Connection Code - Arduino Web Logger

Add the protocol to check and modify RTC time.

Get RTC Time

When you send "RTC0", it replies with T=<time stamp>
<time stamp> is only the sec part of POSIX time_t.

Set RTC Time

When you send "RTC<time stamp>", it replies with T=<time stamp>


Original Protocol

Get Last Data Block Number

When you send "GET0", it replies with N=<Data Block Number>

<Data Block Number>: "N=1" - "N=4294967295" unsigned 32 bit integer.
Of course, it will not get this big.

In the Data Log Code - Arduino Web Logger posting source code, SD_BASE_BLOCK becomes 0.

Get Data Block

When you send "GETn", it replies with
  • N = <Data Block Number>
  • T = <time stamp>
  • D = <data>
<time stamp> is only the sec part of POSIX time_t.  Do *1000 since we don't use millisecond part.  Of course it will be very difficult to see "T=4294967295" in our lifetime.

<data> is in +dd.ddd format.
  • "D=+12.345"
  • "D=" no data
  • "D=-99.999" termination.  That certain Data Block send is completely finished.

Digital Out

When turning on "DOn=ON",
when turning off "DOn=OFF".

n is channel number between 0 to 7.  There is no reply.

Digital In

When you send "DI", it replies with "bbbbbbbb".
b represents each input channel, and it becomes either '0' or '1'.

Analog In

When you send "AIn", it replies with "+dd.ddd".
n is channel number between 0 and 7, and "+dd.ddd" is from -10.000 to +10.000.


When you click [GetRTC] button display RTC time.



The incorrect RTC time gets set to current time when [SetRTC] button is clicked.




Now let's look at modified code.  ( Look at link below for complete code )

Fixed code from Arduino

#include <SPI.h>
#include <Ethernet.h>
#include <WebSocket.h>
#include <SoftwareSerial.h>
#include "Sd2Card.h"
#include <minWire.h> // use the modified library code from RTC Preparation posting
#include <minRTClib.h>

// ... OMIT ...

//-------------------------------------DS-1307
RTC_DS1307 rtc;

// ... OMIT ...

//------------------------------------------------ setup()
void setup() {
 Serial.begin(9600);
 Wire.begin();
 rtc.begin();

 // ... OMIT ...

//-----------------------------------------------
// Create DATA Logging Block
//-----------------------------------------------
void makeLogBlock(void) {
 byte i;
 byte *pB;
 float fdata;
 DateTime now;

 // ... OMIT ...

 //------------------------- buffer is full
 if(LogBlockSize >= LOG_BLOCK_SIZE) {
  pB = sdBuffer;
  *((uint32_t*)pB) = USED_MARK;  //Used Flag
  pB += 4;
  now = rtc.now();
  *((uint32_t*)pB) = now.unixtime(); //Time-stamp

  // ... OMIT ...

void printTOD(void) {
 DateTime now = rtc.now();
 Serial.print(now.year(), DEC);
 printProg(DASH_MSG);
 Serial.print(now.month(), DEC);
 printProg(DASH_MSG);
 Serial.print(now.day(), DEC);
 printProg(SPACE_MSG);
 Serial.print(now.hour(), DEC);
 printProg(COMMA_MSG);
 Serial.print(now.minute(), DEC);
 printProg(COMMA_MSG);
 Serial.println(now.second(), DEC);
}

// ... OMIT ...

void onData(WebSocket &socket, char* rxLn, byte rxSz) {
 uint32_t para;
 DateTime now;

 //-----------------------------------
 // GET
 //-----------------------------------
 if((rxLn[0] == 'G') && (rxLn[1] == 'E') && (rxLn[2] == 'T')) {
  *(rxLn + rxSz) = 0x00;
  para = atol(rxLn + 3);
  
  if(para) 
   upload_DataBlock(para + SD_BASE_BLOCK);
  else 
   upload_lastBlockNo();
 }

 //-----------------------------------
 // RTC
 //-----------------------------------
 else if((rxLn[0] == 'R') && (rxLn[1] == 'T') && (rxLn[2] == 'C')) {
  *(rxLn + rxSz) = 0x00;
  para = atol(rxLn + 3);

  // set RTC
  if(para) {
   rtc.adjust(DateTime(para));
   delay(100);
  }

  // send current RTC
  printLn[0] = 'T';
  printLn[1] = '=';
  now = rtc.now();
  ltoa((long int)(now.unixtime()), printLn+2, 10);
  WSsend(printLn, strlen(printLn));
 }
}


Fixed code from HTML/JS

        // ... OMIT ...
        
        // receive Time Stamp
        if(ar[0] == 'T') {
            timeStamp = ar[1];
            // Locale standard is GMT. 
            // In order to view as Korean time, we need to -9 hrs.
            tod.setTime((timeStamp - (9 * 3600)) * 1000); 
            
    // ... OMIT ...
    
    function getRTC() {
        clearDspPara();
        ws.send('RTC0');
    }
    
    function setRTC() {
        clearDspPara();
        var t = new Date();
        // GMT problem. Ths time +9 hrs.
        ws.send('RTC' + (t / 1000 + 9 * 3600));
    }
    
// ... OMIT ...

<button type="button" onclick="getRTC()">Get RTC</button>
<button type="button" onclick="setRTC()">Set RTC</button>

// ... OMIT ...



Full codes can be found in following links





Friday, June 6, 2014

RTC Preparation - Arduino Web Logger

This is somewhat out of order, BUT let's connect RTC.

I purchased RTC Shield from following site.
https://store.open-electronics.org/index.php?_route_=RTC%20shield

Then the following gets delivered.


They gave me extra connector! :)

Also it comes with manual that I cannot read.  Italian maybe...?


Fortunately, there is no problem reading the schematic.


After some soldering...




Now we have four story high building.
We have on the first floor Arduino, second floor Ethernet Shield, third floor RTC Shield, and finally on the fourth floor we have RS485 Shield.




The library for RTC can be downloaded in the following site.
You might have to modify the name of the folder after downloading it.  Arduino sketch won't let you import 'RTClib-master', thus fix the name to 'RTClib_master'.

However, there is a problem.  This library takes up too much memory.
This library brings our memory down to 170 bytes ( before this library we have 377 bytes ).  We are not able to use this like this.

Let's look through, and find who uses up the most memory.

1. We are only going to use IIC Master mode and not the Slave.  So let's get rid of all Slave part.


2. Reduce memory from writing to RTC part.


RTClib.cpp
void RTC_DS1307::adjust(const DateTime& dt) {
 WIRE.beginTransmission(DS1307_ADDRESS);
 WIRE.write(0);
 WIRE.write(bin2bcd(dt.second()));
 WIRE.write(bin2bcd(dt.minute()));
 WIRE.write(bin2bcd(dt.hour()));
 WIRE.write(bin2bcd(0));
 WIRE.write(bin2bcd(dt.day()));
 WIRE.write(bin2bcd(dt.month()));
 WIRE.write(bin2bcd(dt.year() - 2000));
 WIRE.write(0);
 WIRE.endTransmission();
} // The import thing is that the maximum length is 9 bytes

Wire.cpp
size_t TwoWire::write(uint8_t data) {
 // ... OMIT ...
 txBuffer[txBufferIndex] = data; // size of txBuffer[] is 
                                    // #define BUFFER_LENGTH 32 in
                                 // Wire.h
                                    // Reduce to size 9
 ++txBufferIndex;
 txBufferLength = txBufferIndex;

// ... OMIT ...

uint8_t TwoWire::endTransmission(uint8_t sendStop) {
 int8_t ret = twi_writeTo(txAddress, txBuffer, txBufferLength, 1, sendStop);

twi.c
uint8_t twi_writeTo(uint8_t address, uint8_t* data, uint8_t length, uint8_t wait, uint8_t sendStop) {
 // ... OMIT ...

 for(i = 0; i < length; ++i) {
  // Because ISR sends actual IIC, it has it's own Buffer.
  // If you make sure that you won't send until it is
  // completely finished, then you don't need that Buffer.
  twi_masterBuffer[i] = data[i];
 }

 // ... OMIT ...


3. Reduce memory from reading from RTC part.


RTClib.cpp
DateTime RTC_DS1307::now() {
 WIRE.beginTransmission(DS1307_ADDRESS);
 WIRE.write(0); 
 WIRE.endTransmission();
 WIRE.requestFrom(DS1307_ADDRESS, 7); // Receiving 7 bytes
 uint8_t ss = bcd2bin(WIRE.read() & 0x7F);
 uint8_t mm = bcd2bin(WIRE.read());
 uint8_t hh = bcd2bin(WIRE.read());
 WIRE.read();
 uint8_t d = bcd2bin(WIRE.read());
 uint8_t m = bcd2bin(WIRE.read());
 uint16_t y = bcd2bin(WIRE.read()) + 2000;
 return DateTime (y, m, d, hh, mm, ss);
}

Wire.cpp
uint8_t TwoWire::requestFrom(uint8_t address, uint8_t quantity, uint8_t sendStop) {
 // ... OMIT ...

 // Reduce the size of rxBuffer[] to 7
 uint8_t read = twi_readFrom(address, rxBuffer, quantity, sendStop);

 // ... OMIT ...
}

twi.c
uint8_t twi_readFrom(uint8_t address, uint8_t* data, uint8_t length, uint8_t sendStop) {
 //... OMIT ...

 for(i = 0; i < length; ++i){
  // Received from ISR. There is no actual benefits for
  // moving to pointer, so let's keep 2 buffer.
  // fix the size of twi_masterBuffer[] to 7
  data[i] = twi_masterBuffer[i];
 }

 // ... OMIT ...
}


We have reduced memory to 142 bytes.

I have uploaded the modified library with "min" added to the original name.
Import the attached library.

Full codes can be found in following links






Thursday, June 5, 2014

Drawing DB Data with Chart JS - Arduino Web Logger



This time we are going to draw graph with the data we have seen as text.
I have used Spline Charts.  I chose this graph because it has smooth curve lines.
http://chartjs.devexpress.com/Demos/VizGallery/#chart/chartslineandpointseriesspline
Refer to below site for how to make chart.
http://chartjs.devexpress.com/Documentation/Tutorial/Configure_Charts?version=13_2

Use the code in Web SQL Database - Arduino Web Logger posting after deleting upload part and inserting DB part.

For detailed information about how to use Chart JS refer to HTML / JavaScript - 4017 Arduino Gateway posting.  Include knockout-3.0.0.js this time.

<!DOCTYPE html>
<html>
    <head>
        <title>WebLogger 7.1</title>
        // ******************** 
        //  Below js library are in
        //  DevExpressChartJS-13.2.9\Lib\js.
        // ********************
        <script src="jquery-2.0.3.min.js"></script>
        <script src="globalize.min.js"></script>
        <script src="knockout-3.0.0.js"></script>
        <script src="dx.chartjs.js"></script>
        
        <script>
            var blockNo;
            var data = new Array();
            var tod = new Date();
            var db = openDatabase('WebLogger8', '1.0', 'Web Logger', 512 * 1024);

            $(document).ready(function() {
                db.transaction(function(tx) {
                    tx.executeSql('CREATE TABLE IF NOT EXISTS LogIndex(block, timeStamp)');
                    tx.executeSql('CREATE TABLE IF NOT EXISTS LogData(block, seq, data)');
                });

            });

            // ********************
            // Create Chart.
            // ********************
            var chartDataSource = [];

            $(function() {
                $("#chartContainer").dxChart({
                    title: {
                        text: 'Logged Data Graph'
                    },
                    dataSource: chartDataSource,
                    commonSeriesSettings: {
                        type: 'spline',
                        argumentField: 'time'
                    },
                    series: [{ name: 'ADAM-4017', valueField: 'data', point: { visible: false}}]
                });
            });

            function viewIndex() {
                var len, i;

                db.transaction(function(tx) {
                    tx.executeSql("SELECT * FROM LogIndex", [], function(tx, results) {
                        len = results.rows.length;
                        clearDspPara();

                        for (i = 0; i < len; i++) {
                            tod.setTime(results.rows.item(i).timeStamp * 1000);
                            $("#dumpPara").append('<p>' + results.rows.item(i).block + ',' + tod.toLocaleString() + '<p>');
                        }
                    }, null);
                });
            }

            function viewData() {
                var len, i;
                blockNo = document.getElementById("DataBlockNo").value;

                db.transaction(function(tx) {
                    tx.executeSql('SELECT * FROM LogData WHERE block="' + blockNo + '"', [], function(tx, results) {
                        len = results.rows.length;

                        if (len > 120) len = 120;

                        clearDspPara();

                        for (i = 0; i < len; i++) {
                            data[results.rows.item(i).seq] = results.rows.item(i).data;
                        }

                        for (i = 0; i < len; i++) {
                            $("#dumpPara").append('<p>' + data[i] + '<p>');
                        }
                    }, null);
                });
            }

            function graphData() {
                var len, i;
                var chartData = [];
                blockNo = document.getElementById("DataBlockNo").value;

                db.transaction(function(tx) {
                    tx.executeSql('SELECT * FROM LogData WHERE block="' + blockNo + '"', [], function(tx, results) {
                        len = results.rows.length;

                        if (len > 120) len = 120;

                        // ********************
                        //  Create Data for Chart
                        // ********************
                        for (i = 0; i < len; i++) {
                            // If string goes into chart data
                            // everything screws up. Here I 
                            // converted string to float.
                            // More about conversion:
                            // http://www.javascripter.net/faq/convert2.htm
                            data[results.rows.item(i).seq] = parseFloat(results.rows.item(i).data);
                        }

                        for (i = 0; i < len; i++) {
                            chartData.push({ time: i * 0.5, data: data[i] });
                        }

                        observableChartDataSource = ko.observableArray(chartData);
                        var chart = $("#chartContainer").dxChart("instance");
                        var dataSource = chart.option('dataSource', chartData);
                    }, null);
                });
            }

            function clearDspPara() {
                $("#dumpPara").empty();
            }
        </script>
    </head>
    
    <body>
        <!--Create space for Chart-->
        <div id="chartContainer" style="width: 720px; height: 440px;"></div>
        <br />
        
        <button type="button" onclick="viewIndex()">View Index</button>
        <input type="text" id="DataBlockNo" value="1">
        <button type="button" onclick="viewData()">View Data</button>
        <button type="button" onclick="graphData()">Graph Data </button>
        <p id="dumpPara"></p>
    </body>
</html>


Full codes can be found in following links





Wednesday, June 4, 2014

Web SQL Database - Arduino Web Logger

The most basic HTML5.0 - Web SQL Database Tutorial.
http://www.tutorialspoint.com/html5/html5_web_sql.htm

1. Set DB's Name and Size.

    var db = openDatabase('WebLogger8', '1.0', 'Web Logger', 512 * 1024);

DB's name is WebLogger8 and the size is 512K bytes.

2. Create 2 DB Table. One with Index and the other with Data.

    db.transaction(function(tx) {
        tx.executeSql('CREATE TABLE IF NOT EXISTS LogIndex(block, timeStamp)');
        tx.executeSql('CREATE TABLE IF NOT EXISTS LogData(block, seq, data)');
    });

  • LogIndex: this is an index table with 2 columns
    1. block number &
    2. timeStamp
  • LogData: this is an data table
    1. block number,
    2. index within the block (500 ms = index 0, 1000 ms = index 1...)
    3. actual data value

3. Add Data Block to DB Table.

    db.transaction(function(tx) {
        for (i = 0; i < rcvCnt; i++) {
            tx.executeSql('INSERT INTO LogData VALUES(?, ?, ?)', [blockNo, i, data[i]]);
        }
    });

Append the data table.  You must write to DB after you have received entire Data Block.  If you try to write data values every time you receive it, you will lose a lot of data.  The confusing thing is that you receive correct number of data; however, you don't see the actual values anywhere.

We don't define data type for each column.  This makes things simpler and confusing at the same time.  When you actually write to DB the type gets defined.  I am assuming that block becomes String, seq becomes Number, and data becomes String.

    db.transaction(function(tx) {
        tx.executeSql('INSERT INTO LogIndex VALUES(?, ?)', [blockNo, timeStamp]);
    });

Lastly append the index.
block and timeStamp are both String type.

Caution:
    db.transaction(function(tx) {
        tx.executeSql('INSERT INTO LogData VALUES(?, ?, ?)', [blockNo, i, data[i]]);
        tx.executeSql('INSERT INTO LogIndex VALUES(?, ?', [blockNo, timeStamp]);

Inside one db.transaction, do not append DB twice.  Theoretically this is not a problem; however, if you continue to use that statement it causes error so let's avoid it.

4. Check DB Table.

    db.transaction(function(tx) {
        // Check all index.
        tx.executeSql('SELECT * FROM LogIndex', [], function(tx, results) {
            len = results.rows.length;

            for (i = 0; i < len; i++) {
                // we don't use millisecond part thus do *1000
                tod.setTime(results.rows.item(i).timeStamp * 1000);
                // if you do toLocaleString() to
                // day-time format variable, it returns a string
                // with a language sensitive representation of this
                // date.  There are many others. 
                // http://www.w3schools.com/jsref/jsref_obj_date.asp
                $("#dumpPara").append('' + results.rows.item(i).block + ',' + tod.toLocaleString() + '

');
            }
        }, null);
    });

    db.transaction(function(tx) {
        // search DB with block
        tx.executeSql('SELECT * FROM LogData WHERE block="' + blockNo + '"', [], function(tx, results) {
            len = results.rows.length;

            for (i = 0; i < len; i++) {
                // There is no guarantee that DB will read in order we
                // wrote. Thus we index it inside block.
                data[results.rows.item(i).seq] = results.rows.item(i).data;
            }

            for (i = 0; i < len; i++) {
                $("#dumpPara").append('

' + data[i] + '

');
            }
        }, null);
    });






5. For Less Confusion while Debugging...

  1. When you try to change the name of DB, close all Chrome windows and re-open it.  Otherwise new DB Table does not get created.
  2. If HTML file is not in the same Folder path, close all Chrome windows and re-open it.  If you don't do this WebSocket doesn't get connected.

Get the last SD logged Block Number from Arduino.

Click the [Get Last Data Block Number] button.



Enter the Block Number and Upload that certain Data Block from Arduino SD.

Enter the Block Number and click [Get Data Block] button.

If you upload, it gets inserted into both Index table and Log table.


Check Index Table.

Click [View Index] button.
It is displayed in Block, Time Stamp order.


Enter Data Block number to check Data Table.

Enter Block Number and click [View Data] button.

You will be able to see 120 values from that Block.  Only the data will be shown.


Below is HTML/JS Code

The link to the code is posted in the end.
<!DOCTYPE html>
<html>
    <head>
        <title>WebLogger 6.3</title>
        <script type="text/javascript" src="jquery-2.0.3.min.js"></script>
        <script>
            var ws;
            var blockNo, rcvCnt, timeStamp;
            var data = new Array();
            var tod = new Date();

            var db = openDatabase('WebLogger8', '1.0', 'Web Logger', 512 * 1024);

            $(document).ready(function() {
                db.transaction(function(tx) {
                    tx.executeSql('CREATE TABLE IF NOT EXISTS LogIndex(block, timeStamp)');
                    tx.executeSql('CREATE TABLE IF NOT EXISTS LogData(block, seq, data)');
                });

                WebSocketConnect();
            });

            function WebSocketConnect() {
                var ar = new Array();
                var arSz, i;

                try {
                    ws = new WebSocket('ws://192.168.219.16:80/');
                    ws.onopen = function() {
                        status('Connected...');
                    }

                    ws.onclose = function() { status('Closed...'); }
                    ws.onerror = function(evt) { status('Error ' + evt.data); }
                    ws.onmessage = function(evt) {
                        ar = evt.data.split('=');

                        // receive Last Data Block Number
                        if (ar[0] == 'N') {
                            blockNo = ar[1];
                            $("#blockNoPara").empty();
                            $("#blockNoPara").append('Block Number:' + blockNo);
                        }

                        // receive Time Stamp
                        if (ar[0] == 'T') {
                            timeStamp = ar[1];
                            tod.setTime(timeStamp * 1000);
                            $("#timestampPara").empty();
                            $("#timestampPara").append(tod.toLocaleString());
                        }

                        // receive Data Block
                        if (ar[0] == 'D') {
                            data[rcvCnt] = ar[1];

                            if (ar[1] != '-99.999') rcvCnt++;
                            else {
                                $("#recordCntPara").empty();
                                $("#recordCntPara").append('Record Count:' + rcvCnt);

                                if (blockNo != 0) {
                                    db.transaction(function(tx) {
                                        if (rcvCnt > 120) rcvCnt = 120;
                                        for (i = 0; i < rcvCnt; i++) {
                                            tx.executeSql("INSERT INTO LogData VALUES(?, ?, ?)", [blockNo, i, data[i]]);
                                        }
                                    });

                                    db.transaction(function(tx) {
                                        tx.executeSql("INSERT INTO LogIndex VALUES(?, ?)", [blockNo, timeStamp]);
                                    });
                                }
                            }
                        }
                    }
                } catch (exception) { status('Exception' + exception); }
            }

            function upload_lastBlockNo() {
                ws.send("GET0");
            }

            function upload_DataBlock() {
                blockNo = document.getElementById("DataBlockNo").value;
                ws.send("GET" + blockNo);
                blockNo = 0;
                rcvCnt = 0;
                $("#timestampPara").empty();
                $("#recordCntPara").empty();
                $("#dumpPara").empty();
            }

            function status(str) {
                $("#status").empty();
                $("#status").append(str);
            }

            function viewIndex() {
                var len, i;

                db.transaction(function(tx) {
                    tx.executeSql("SELECT * FROM LogIndex", [], function(tx, results) {
                        len = results.rows.length;
                        clearDspPara();

                        for (i = 0; i < len; i++) {
                            tod.setTime(results.rows.item(i).timeStamp * 1000);
                            $("#dumpPara").append('<p>' + results.rows.item(i).block + ',' + tod.toLocaleString() + '<p>');
                        }
                    }, null);
                });
            }

            function viewData() {
                var len, i;
                blockNo = document.getElementById("DataBlockNo").value;

                db.transaction(function(tx) {
                    tx.executeSql('SELECT * FROM LogData WHERE block="' + blockNo + '"', [], function(tx, results) {
                        len = results.rows.length;

                        if (len > 120) len = 120;

                        clearDspPara();

                        for (i = 0; i < len; i++) {
                            data[results.rows.item(i).seq] = results.rows.item(i).data;
                        }

                        for (i = 0; i < len; i++) {
                            $("#dumpPara").append('<p>' + data[i] + '<p>');
                        }
                    }, null);
                });
            }

            function clearDspPara() {
                $("#blockNoPara").empty();
                $("#timestampPara").empty();
                $("#recordCntPara").empty();
                $("#dumpPara").empty();
            }
        </script>
    </head>

    <body>
        <p id="status"></p>
        <button type="button" onclick="upload_lastBlockNo()">Get Last Data Block Number</button>
        <br />
        
        <input type="text" id="DataBlockNo" value="1">
        <button type="button" onclick="upload_DataBlock()">Get Data Block</button>
        <br />
        
        <button type="button" onclick="viewIndex()">View Index</button>
        <button type="button" onclick="viewData()">View Data</button>
        <p id="blockNoPara"></p>
        <p id="timestampPara"></p>
        <p id="recordCntPara"></p>
        <p id="dumpPara"></p>
        
    </body>
</html>

Full codes can be found in following links