Monday, June 2, 2014

Data Log Code - Arduino Web Logger

What does this code do...?


  1. Reads ADAM-4017's channel 0 every 500 ms.
  2. Store the data as float in buffer.
  3. After repeating 60 * 2 = 120 times, a minute long Data Block has been created.  Write the Data Block to SD.

Can we use the SD library...?

In order to use SD FAT32 we absolutely have to use SD Library.  However, do we have enough memory?

Let's check the size of Arduino UNO's RAM size with using just following code.
void setup() {}
void loop() {}

RAM: 2,048

9#include <SPI.h>
71#include <SPI.h> <Ethernet.h>
262#include <SPI.h> <Ethernet.h> <WebSocket.h>
330#include <SPI.h> <Ethernet.h> <WebSocket.h> <SoftwareSerial.h>
330#include <SPI.h> <Ethernet.h> <WebSocket.h> <SoftwareSerial.h> <Time.h>
1,109#include <SPI.h> <Ethernet.h> <WebSocket.h> <SoftwareSerial.h> <Time.h> <SD.h>

It may seem as if there might not be any problem, but let's check what really happens in actual code from Arduino code - 4017 Arduino Gateway posting.

1,126#include <SPI.h> <Ethernet.h> <WebSocket.h> <SoftwareSerial.h> <Time.h> excluding <SD.h>

If we include setTime() to actually use <Time.h>, the memory becomes 1,163

And then we include <SD.h>...1,769
Open a file...1,863

We only have 212 bytes left.  Now we don't even have 512 bytes to create a Data Block.  Even if we don't create Data Block, how are we going to manage Stack & Heap with this memory?  What memory are we going to use for additional codes?
It might work sometimes... ( doesn't actually run on Arduino )

Therefore, we are going to give up on FAT32.


Now, are we going to use Arduino with bigger RAM?
http://arduino.cc/en/Main/ArduinoBoardMega2560
The biggest they have is 8K anyways.
If we have to improve on hardware continuously, why use Arduino?
Thus alternatively we are going to leave out FAT32 file system.

We are going to use 512 bytes block without using File.  Why do we need File if we are only going to log?

In Arduino library copy the following 4 files into the current sketch folder.
\arduino-1.5.6-r2\libraries\SD\src\utility\Sd2Card.cpp
Sd2Card.h
Sd2PinMap.h
SdInfo.h

Now we are only going to use the following 3 functions.
  • init()
  • readBlock()
  • writeBlock()

Current Time

Eventually we are going to use RTC; however, for this code we are using System Clock.
Library is available on the following site: http://www.pjrc.com/teensy/td_libs_Time.html
Download the Time.zip file unzip it and then Sketch > Import Library > Add Library.

You can also find it in Arduino Cookbook pg. 404.

Source Code

This is only part of the code.  I will post the link to the full code at the bottom.
#include <SPI.h>
#include <Ethernet.h>
#include <WebSocket.h>
#include <SoftwareSerial.h>
#include "Sd2Card.h"
#include <Time.h>

byte mac[] = { 0x90, 0xA2, 0xDA, 0x0F, 0x25, 0xC4 };
byte ip[] = { 192, 168, 219, 16 };

WebSocket wsServer;

// ------------------------------------------- SD Card
#define SD_BASE_BLOCK  1024 // 1024 is not a special #
#define LOG_BLOCK_SIZE 120

byte sdBuffer[512]; // SD card only writes in 512 bytes block.
                    // Thus we have to have 512 bytes.
byte sdStatus = 0;
byte LogblockSize = 0;
uint32_t currBlock;
long lastPollTime = 0;

Sd2Card sdCard;

void SDbegin(void);
void makeLogBlock(void);
void update_currBlock(void);

void setup() {
 Serial.begin(9600);

 Soft485.begin(9600);
 pinMode(DE485, OUTPUT);
 ADAM4017_config();

 Ethernet.begin(mac, ip);
 wsServer.registerConnectCallback(&onConnect);
 wsServer.registerDataCallback(&onData);
 wsServer.registerDisconnectCallback(&onDisconnect);
 wsServer.begin();
 delay(100);

 rx485ix = -1;

 // setTime(hours, minutes, seconds, days, months, years);
 // Time is not going to be right until we add RTC
 setTime(12, 0, 0, 11, 5, 14);
                               
 SDbegin();
 if(!sdStatus)
  printProgln(SD_FAIL_MSG);
}

void loop() {
 wsServer.listen();

 if(sdStatus) {
  if((millis() - lastPollTime) >= 500) {
   lastPollTime = millis();
   AdamPoll('0');
  }
 }

 AdamReceiver();
}

/*
 -----------------------------------------------------
 D A T A  B L O C K  M A K I N G
 -----------------------------------------------------
*/

// ---------------------------------------------------
// SD Card Initialize
// ---------------------------------------------------
void SDbegin(void) {
 byte *pB;

 sdStatus = 0;
 pinMode(10, OUTPUT); // SS

 if(!(sdCard.init(SPI_HALF_SPEED, 4)))
  return; // SD Card Initialization Failed

 if(!(sdCard.readBlock(SD_BASE_BLOCK, sdBuffer)))
  return; // SD Card Read Failed

 pB = sdBuffer;

 // No Used Flag. New SD Card!!
 if(*((uint32_t*)pB) != 0xaa55a55a) {
  printProgln(NEW_SD_MSG);
  *((uint32_t*)pB) = 0xaa55a55a;
  currBlock = SD_BASE_BLOCK + 1;
  *((uint32_t*)(pB + 4)) = currBlock;

  if(!(sdCard.writeBlock(SD_BASE_BLOCK, sdBuffer)))
   return; // SD Card Write Failed
 }

 else
  currBlock = *((uint32_t*)(pB + 4)); // First Free Block

 LogBlockSize = 0;
 sdStatus++;
 printCurrBlock(); // debugging purpose
}

// ---------------------------------------------------
// Create DATA Logging Block
// ---------------------------------------------------
void makeLogBlock(void) {
 byte i;
 byte *pB;
 float fdata;

 // --------- Reset buffer if it is the first time.
 if(!LogBlockSize) {
  pB = sdBuffer + 8; // insert Used Flag &
                     // time-stamp when finished
  fdata = -99.999 * 10.; // -99.999 means that there is no
                         // data. When we read them we do
                         // /10 thus we have to do *10.
  i = 120; // fill it in 120 times
  
  while(i--) {
   *((float*)pB) = fdata;
   pB += 4;
  }

  i = 24; // 24 bytes padding
  
  while(i--) {
   *pB = 0x00;
  }
 }
 /*
  ADAM-4017 returns +dd.ddd
  It uses fixed 3 decimal points.

  To convert decimal string to float:
  double atof(const char* nptr)
  http://www.nongnu.org/avr-libc/user-manual/group__avr__stdlib.html#ga689c9d3c4c04463aa31d329937789d06

  But there is a problem. It only works with 2 decimal points.
  It rounds up third decimal point.
  Thus atof("12.345") will become 12.35.

  As a solution we do *10 then convert it to float.
  0123456    0123456
  +98.765 -> +987.65
 */

 // ----------------------- Insert Data into buffer
 //             0123456
 // rx485Ln[] ">+dd.ddd" <cr>
 rx485Ln[3] = rx485Ln[4];
 rx485Ln[4] = '.'; // *10

 pB = sdBuffer + 8 + (LogBlockSize * 4);
 *((float*)pB) = atof(rx485Ln);

 LogBlockSize++;

 // -------------------------------- buffer is full
 if(LogBlockSize >= LOG_BLOCK_SIZE) {
  pB = sdBuffer;
  *((uint32_t*)pB) = 0xaa55a55a; // Used Flag
  pB += 4;
  *((time_t*)pB) = now(); // Time Stamp

  sdCard.writeBlock(currBlock, sdBuffer);
  currBlock++;
  LogBlockSize = 0;
  update_currBlock(); // Fix the next free block location
 }
}

void update_currBlock(void) {
 byte *pB;
 byte status = 0;

 if(sdCard.readBlock(SD_BASE_BLOCK, sdBuffer)) {
  pB = sdBuffer;

  if(*((uint32_t*)pB) == 0xaa55a55a) {
   *((uint32_t*)(pB + 4)) = currBlock;

   if(sdCard.writeBlock(SD_BASE_BLOCK, sdBuffer)) {
    status++;
    printTOD(); // display time stamp
    printCurrBlock(); // display next free block
   }
  }
 }

 if(!status)
  // something is going terribly wrong
  printProgln(BLOCK_UPDATE_FAIL_MSG);
}

void printTOD(void) {
 tmElements_t tm;

 breakTime(now(), tm);

 Serial.print(tm.Year - 30); // size 1970
 printProg(DASH_MSG);
 Serial.print(tm.Month);
 printProg(DASH_MSG);
 Serial.print(tm.Day);
 printProg(SPACE_MSG);
 Serial.print(tm.Hour);
 printProg(COMMA_MSG);
 Serial.print(tm.Minute);
 printProg(COMMA_MSG);
 Serial.print(tm.Second);
}

void AdamReceiver(void) {
 char rxChr;

 if(Soft485.available()) {
  rxChr = Soft485.read();

  if(rxChr == '>')
   rx485ix = 0;
  
  else if(rx485ix >= 0) {
   if(rxChr == 0x0d) { //     0123456
    if(rx485ix == 7) { // >+dd.ddd<cr>
     wsServer.send(rx485Ln, 7);
     
     if(sdStatus)
      // when AD read value arrives from
      // ADAM-4017, log it
      makeLogBlock();
    }
    rx485ix = -1;
   }

   else {
    rx485Ln[rx485ix++] = rxChr;

    if(rx485ix > 7)
     rx485ix = -1;
   }
  }
 }
}

Full codes can be found in following links





Sunday, June 1, 2014

Modification to RS485 Shield - Arduino Web Logger

Finally, we are using SD card.

In Ethernet Shield use Pin 4 as SD_CS.  If we just leave everything as it is, we are not going to be able to use them.

Originally:

  • Pin 3: RX
  • Pin 4: TX
  • Pin 2: DE
We are going to modify it so that it becomes:
  • Pin 3: RX
  • Pin 5: TX
  • Pin 2: DE

I'm going to cut RS-485 Shield.





In all previous codes I used Pin 4 as TX, and because I don't want to fix all the codes I soldered a wire onto the pin.  So now I can move TX to Pin 4 or Pin 5 depending on what I need.







Saturday, May 31, 2014

Components - Arduino Web Logger

Diagram of Arduino Web Logger Component

Let's log ADAM-4017's AD value every 500 ms.
We are going to make 512 bytes long block every 1 minute and write that block to the SD card in Ethernet Shield.

Data Block Component.

  • Used Flag: 4 bytes
    Set it to 0xaa55a55a.
  • Time Stamp: 4 bytes
    This is only the sec part of POSIX time_t value.  We are not going to use the millisecond, thus we are going to multiply the value by 1000 (*1000).  It is 32 bit unsigned integer.
  • Data: 4 bytes float value.  It is going to be sampled twice every second.
    1 minute measurement: 2 * 60 * 4 = 480 bytes
    Second is going to be offset.
    For example, The offset for second measurement in 27 second will be 8 + 27 * 8 + 4 = 228.
    Set the unmeasured values to -99.999.  When initializing the block set everything to -99.999.
  • Backup: 24 bytes, set it to Null.

Data File.

Logging is going to be one File.

In FAT32, the biggest file size is 512 * 232 = 241 = 2,199,023,255,552 = 2GB.  Even the smallest SD card is bigger than 2GB.  With 2GB, we can log for 2,199,023,255,552 / 512 / 60 / 24 / 365 = 8,217 years.  Thus memory is going to be full approximately after 8,000 years so we don't have to worry about that.

Data inside File are going to be in time order, which will allow us to retrieve Data Block from certain time very quickly using the simplest Binary Search.  However, adjusting the RTC clock due to whatever reasons will screw this up.  ( Getting deeper into this is too much for this posting, so I am going to skip it. )

Web DB.

Arduino cannot hold all this log data like Browser might hope.  If the browser wants to look through a year long data "very quickly" it's impossible.  Thus, browser is going to keep log data in its database and once in a while connect with Arduino to get the most recent log and update its DB.





Friday, May 30, 2014

Testing: Temporary Test Board(volume) + ADAM-4017 + RS485 + Arduino UNO + Ethernet Shield

Temporary Test Board + ADAM 4017 + RS485 + Arduino UNO + Ethernet Shield

I tested connecting volume to CH0 of ADAM-4017.
I used same source voltage.
Because input range is set to ±10V, if voltage is over +10V Arduino code shows it as +10V.
I didn't cut the board because I was too lazy it is easier to turn the knob with the board like that.

Schematic for volume attaching

I am attaching schematic for volume attaching just in case.
CH0 - CH5 on ADAM-4017 is differential, and CH6 - CH7 is single ended.
Thus, Vin0- has to be set to GND.





Thursday, May 29, 2014

HTML / JavaScript - 4017 Arduino Gateway

Screen Capture of WSAdam4_0.html

I want to emphasize how easy it is to draw that circular gauge!

First of all choose the chart that you like the most.
http://chartjs.devexpress.com/Demos/VizGallery/#index

If you have picked Circular Gauge, let's have a look at 12 samples.
http://chartjs.devexpress.com/Demos/VizGallery/#chart/circulargaugesbasicelementspaletteforranges

Now we have to download the library.
http://chartjs.devexpress.com/Download

Last updated download file is DevExpressChartJS-13.2.9.zip

Next, read the tutorial very thoroughly. After reading this you can do everything!
http://chartjs.devexpress.com/Documentation/Tutorial/Configure_Gauges?version=13_2

If you want to know more, read the API reference.
http://chartjs.devexpress.com/Documentation/ApiReference/dxCircularGauge?version=13_2

I have divided up what we have to do roughly. It could easily be done by just simply following the tutorial.

  1. Load the needed js library.
  2. Create the gauge.
  3. Using HTML, create space for gauge.

    If all the above is done,
  4. Send data that gauge is going to represent.

<!DOCTYPE html>
<html>
    <head>
        <title>ADAM-4017 4.0</title>
        <!--****************************
        1. Load the needed js library
           The following 3 js library is included 
           in DevExpressChartJS-13.2.9\Lib\js
           I have attached it in this posting.
        ****************************-->
        <script type="text/javascript" src="jquery-2.0.3.min.js"></script>
        <script type="text/javascript" src="globalize.min.js"></script>
        <script type="text/javascript" src="dx.chartjs.js"></script>
        <script>
            var ws;
            var poll = 0;
            var intervalTimer;
            
//          ****************************
//          2. Create the gauge
//          ****************************
            $(function() {
                $('#gaugeContainer').dxCircularGauge({
                    scale: {
                        startValue: 0,
                        endValue: 10,
                        majorTick: { tickInterval: 1 },
                        minorTick: { visible: true, tickInterval: 0.1 }
                    },
                    rangeContainer: {
                        ranges: [{ startValue: 0, endValue: 9, color: 'royalblue' },
                                 { startValue: 9, endValue: 10, color: 'red'}]
                    },
                    valueIndicator: { color: 'red' },
                    title: {
                        text: 'ADAM-4017',
                        font: { size: 28 },
                        position: 'bottom-center'
                    }
                });
            });

            $(document).ready(function() {
                WebSocketConnect();
            });

            function WebSocketConnect() {
                try {
                    ws = new WebSocket('ws://192.168.219.16:80/');
                    ws.onopen = function() {
                        status('Connected...');
                        ws.send("AI0");
                    }
                    ws.onclose = function() { status('Closed...'); }
                    ws.onerror = function(evt) { status('Error ' + evt.data); }
                    ws.onmessage = function(evt) {
//                  ****************************
//                  4. Send data that gauge is going to represent.
//                  ****************************
                        $('#gaugeContainer').dxCircularGauge('instance').value(evt.data);
                    }
                }
                catch (exception) {status('Exception ' + exception);}
            }

            function DI_poll() {
                ws.send("AI0");
                intervalTimer = setTimeout(function() { DI_poll() }, 500);
            }

            function DI_pollEnable() {
                if (!poll) {
                    poll++;
                    DI_poll();
                }
            }

            function DI_pollDisable() {
                clearTimeout(intervalTimer);
                poll = 0;
            }

            function status(str) { $('#status').append(str); }
            
        </script>
    </head>
    <body>
    <p id="status">Connection Status: </p>
    <button type="button" onclick="DI_pollEnable()">Input Refresh ON</button>
    <button type="button" onclick="DI_pollDisable()">OFF</button>
    
    <!--****************************
    3. Using HTML create space for gauge.
    ****************************-->
    <div id="gaugeContainer" style="height:380px"></div>
    </body>
</html>



Full codes can be found in following links






Wednesday, May 28, 2014

Arduino Code - 4017 Arduino Gateway

Add Analog In to the WebSocket protocol.

Digital Out

Turning on: "DOn=ON"
Turning off: "DOn=OFF"
n is channel number (0-7)
There is no reply.

Digital In

When you send "DI", it replies in "bbbbbbbb" format.
b is equal to each input channel, and it is either '0' or '1'.

Analog In

When you send "AIn", it replies in "+dd.ddd" format. 
n is channel number between 0 to 7. 
"+dd.ddd" is between -10.000 - +10.000.


The original is in posting: http://seosmartia.blogspot.kr/2014/05/arduino-code2-webarduinoadam.html

#include <SPI.h>
#include <Ethernet.h>
#include <WebSocket.h>
#include <SoftwareSerial.h>

byte mac[] = { 0x90, 0xA2, 0xDA, 0x0F, 0x25, 0xC4 };
byte ip[] = { 192,168,219,16 };

WebSocket wsServer;

int DE485 = 2;
SoftwareSerial Soft485(3, 4);
void ADMA4017_config(void);

char rx485Ln[8];
int rx485ix;
void AdamReceiver(void);

void setup() {
 Serial.begin(9600);

 Soft485.begin(9600);
 pinMode(DE485, OUTPUT);
 ADMA4017_config(); // Only need to be configured once.

 Ethernet.begin(mac, ip);
 wsServer.registerConnectCallback(&onConnect);
 wsServer.registerDataCallback(&onData);
 wsServer.registerDisconnectCallback(&onDisconnect);
 wsServer.begin();
 delay(100);

 rx485ix = -1;
}

void loop() {
 wsServer.listen();
 AdamReceiver();
}

void onConnect(WebSocket &socket) {
 Serial.println("onConnect called");
}

void onDisconnect( WebSocket &socket ) {
 Serial.println("onDisconnect called");
}

void ADMA4017_config(void) {
 digitalWrite(DE485, HIGH);
 // refer to ADAM - 4017 Arduino Gateway posting
 Soft485.write("02080600");
 Soft485.write(0x0D);
 digitalWrite(DE485, LOW);
}

void onData(WebSocket &socket, char* rxLn, byte rxSz) {
 // Analog In
 if((rxLn[0] == 'A') && (rxLn[1] == 'I')) {
  digitalWrite(DE485, HIGH);
  // Analog In cmd for 4017 is #AAN(cr)
  Soft485.write("#00");
  Soft485.write(rxLn[2]);
  Soft485.write(0x0D);
  digitalWrite(DE485, LOW);
 }
}

void AdamReceiver(void) {
 char rxChr;

 if(Soft485.available()) {
  rxChr = Soft485.read();
  if(rxChr == '>')
   rx485ix = 0;
  else if(rx485ix >= 0) {
   if(rxChr == 0x0d) {
    // 4017's replay is >+dd.ddd(cr)
    if(rx485ix == 7) {
     // The reply conatins decimal pts like
     // "+09.456"; however, JS handles it very
     // well.
     wsServer.send(rx485Ln, 7);
    }
    rx485ix = -1;
   }
   else {
    rx485Ln[rx485ix++] = rxChr;
    if(rx485ix > 7) 
     rx485ix = -1;
   }
  }
 }
}

Full codes can be found in following links







Tuesday, May 27, 2014

ADAM - 4017 Arduino Gateway

ADAM-4017

The manual "ADAM-4000_manual_V20.pdf" can be downloaded from the following site.
http://support.advantech.com.tw/support/DownloadSRDetail_New.aspx?SR_ID=1%2BGE%2B715&Doc_Source=Download

Use INIT* as GND and set RS-485 Address as "00". Unlike ADAM-4050, settings need to be modified.

Configuration Command

%AANNTTCCFF(cr) ( refer to manual page 150 )
  • AA: "00" current RS-485 address.
  • NN: "02" new address for RS-485. You can set it to anything. I used "02" because I used "01" for ADAM-4050
  • TT: "08" input range. ( refer to manual page 152 ). ±10V, for now I used maximum.
  • CC: "06" 9600 bps ( refer to manual page 152 ).
  • FF: "00" ( refer to manual page 152 ).
    • <b7> 0: 60 Hz
    • <b6> 0: Checksum Disable
    • <b5 b4 b3 b2> 0000
    • <b1 b0> 10: ( refer to manual page 387 ).
      Receives the 12 bit AD read value in binary format.
      It becomes 3 byte HEX-ASCII.
      However, ADAM-4050 does not support this, thus ?01(cr) will be returned.
    • <b1 b0> 00: If you set it to 00, you receive voltage value within the input range.
      If the input range is equal to ±10V, it returns values between -10.000 - +10.000, which is very nice.

Analog Input

#AAN(cr) ( refer to manual page 161 )
  • AA: "00" RS-485 address
  • N: "0" channel 0 - 7. We are using 0.
Response
>HHH(cr)
  • HHH: It returns 3 byte HEX-ASCII. "000" - "FFF"
    ADAM-4050 does not support this.
OR

>+dd.ddd(cr)
  • +dd.ddd: The following 7 bytes value gets returned. -10.000 - +10.000.