Posting data via HTTP Arduino Wifly library

Marcio Valenzuela on 1 Oct 2021
Latest activity Reply by Christopher Stapels on 20 Oct 2021

I am trying to migrate a project that used to post from an Arduino with a Tinysine Wifly shield (from Roving networks) to a php form/mysql db using this code snippet:

void reportToCloud() {
  data = "";
  Serial.println("Reporting to cloud...");
   if (wifly.available() > 0) {
  char ch = wifly.read();
  Serial.write(ch);
  if (ch == '\n') {
      /* add a carriage return */ 
      Serial.write('\r');
  }
   }
     if (wifly.open(site, 80)) {
         Serial.print("Connected to ");
    Serial.println(site);
      // Set data to send
      static char outstr1[15];
      static char outstr2[15];
      static char outstr3[15];
      static char outstr4[15];
      String dataString1 = dtostrf(uvindex, 8, 2, outstr1);
      String dataString2 = dtostrf(mq2ratio, 8, 2, outstr2);
      String dataString3 = dtostrf(CO2PPM, 8, 2, outstr3);
      String dataString4 = dtostrf(temperature, 8, 2, outstr4);
      data = String("uvindex=" + dataString1 + "&mq2=" + dataString2 + "&age=" + dataString3 + "&name=" + dataString4);    
      Serial.print(data); //name = temp && age = co2
      //Reset all values
      uvindex = 0;
      mq2ratio = 0;
      CO2PPM = 0;
      temperature = 0;
          /* Send the request */
    wifly.println("POST /arduino/data_post.php HTTP/1.0");
    wifly.println("Host: www.santiapps.com"); // SERVER ADDRESS HERE TOO
          wifly.println("Content-Type: application/x-www-form-urlencoded" );
          wifly.print("Content-Length: ");
          wifly.println(data.length());
    wifly.println();
          wifly.print(data);
          Serial.println("Posted successfully");
     } else {
         Serial.println(">>Failed to connect");
     }
     if (Serial.available() > 0) {
    wifly.write(Serial.read());
     }
     //Added Sat 14 Nov @820am
     wifly.close();
  }

Now Im trying to make it post to thingspeak for improved analysis capabilities and what not. I get a post successful but I get no data in my thingspeak channel. I've checked the string used and its fine:

void reportToCloud() {
  data = "";
  Serial.println("Reporting to cloud...");
  if (wifly.available() > 0) {
    char ch = wifly.read();
    Serial.write(ch);
    if (ch == '\n') {
      /* add a carriage return */ 
      Serial.write('\r');
    }
  }
    if (wifly.open(site, 80)) {
      Serial.print("Connected to ");
      Serial.println(site);
      // Set data to send
      static char outstr3[15];
      static char outstr4[15];
      String dataString3 = "33.33";//dtostrf(33.33, 8, 2, outstr3);//dtostrf(CO2PPM, 8, 2, outstr3);
      String dataString4 = "44.44";//dtostrf(44.44, 8, 2, outstr4);//dtostrf(temperature, 8, 2, outstr4);
      data = String("field3=" + dataString3 + "&field4=" + dataString4);    
      //Serial.print(data); //for debugging
      //Reset all individual values
      CO2PPM = 0;
      temperature = 0;
      /* Send the request */
      String postData = "POST /update?api_key=mykey&"+data+" HTTP/1.0";
      //Serial.println(postData);
      wifly.println("Host: api.thingspeak.com"); // SERVER ADDRESS HERE TOO
      wifly.println("Content-Type: application/x-www-form-urlencoded" );
      wifly.print("Content-Length: ");
      wifly.println(postData.length());
      wifly.println();
      wifly.print(postData);
      Serial.println("Posted successfully");
    } else {
      Serial.println(">>Failed to connect");
    }
    if (Serial.available() > 0) {
      wifly.write(Serial.read());
    }
    //Added Sat 14 Nov @820am
    wifly.close();
  }

postData is printed with this result:

printing co2...
87 105 102 108 121 45 87 101 98 
false returned
Free memory: 670
setPrompt hasnt been called
Already joined network
DeviceID: Wifly-WebClient2
Reporting to cloud...
open santiapps.com 80
Connected to santiapps.com
POST /update?api_key=mykey&field3=33.33&field4=44.44 HTTP/1.0
Posted successfully
close: failed, no *CLOS*
Reporting to cloud...
open santiapps.com 80
Connected to santiapps.com
POST /update?api_key=mykey=33.33&field4=44.44 HTTP/1.0
Posted successfully
close: failed, no *CLOS*

So basically the first snippet works fine and posts to the db via my php form and i can then read it off the db. The second snippet returns what seems to be a successful post but the data is not posted to thingspeak.

Christopher Stapels
Christopher Stapels on 14 Oct 2021 (Edited on 14 Oct 2021)

You might be disconnecting before the server acknowledges your post. Thins may result in your post or its data being rejected. I'm not clear which ones are snipper 1 and 2 above. Which one works and which one doesn't?

I noticed in one part you said

"POST /update?api_key=mykey=33.33&field4=44.44 HTTP/1.0"

This is missing a "&field..."

Marcio Valenzuela
Marcio Valenzuela on 14 Oct 2021

ok this is the snippet that didnt work and recently worked. I noticed it worked when the getData string had only field3. I got about 6 successful posts to TS even though quite a few failed because its posting every 100 or 300 and i got about 6 posts in 3 hours. I know that's a lot of fails, but at least I got some data on an hourly basis.

Then i added field4 to the string and it didnt post anything for 12 hours. So I disconnected it and let it sit overnight because I also think it might be an issue between the wifly and the router because sometimes it posts this to the SM [which i call Event 1's]:

Free memory: 263
setPrompt hasnt been called
Joining network
Alarms set!

which means it didnt post anything and other times it prints this to the SM [which i call Event 2's]:

Free memory: 263
Already joined network
DeviceID: Wifly-WebClient2
Reporting to cloud...
open api.thingspeak.com 80
Connected to api.thingspeak.com
Posted successfully
close: failed, no *CLOS*

Event 1's never result in TS registered data, but Event 2's sometimes do result in data posted to TS. So as I was saying, i let it rest for about 4 hours this morning because I was only getting Event 1's all through the nite, which I KNOW dont result in data posted. So after letting it rest, I started it back up and now Im getting Event 2's in the SM. Not data on TS yet but at least Im getting Event 2's.

#include "ThingSpeak.h"
#include <TimeAlarms.h>
#include <WiFlyHQ.h>
#include <SoftwareSerial.h>
#include <avr/wdt.h>
String AP = "WIFI_NAME";       // CHANGE ME
String PASS = "WIFI_PASSWORD"; // CHANGE ME
String API = "mykey";   // CHANGE ME
String HOST = "api.thingspeak.com";
String PORT = "80";
String field3 = "field3";
String field4 = "field4";
int countTrueCommand;
int countTimeCommand; 
boolean found = false; 
int valSensor = 1;
boolean initialiseSensorFlag;
unsigned long initialiseSensorTimestamp;
long previousMillis = 0;
long interval = 1000;
unsigned long duration;
unsigned long starttime;
unsigned long sampletime_ms = 30000;//sampe 30s ;
unsigned long lowpulseoccupancy = 0;
float ratio = 0;
float concentration = 0;
SoftwareSerial wifiSerial(2,3);
#define DEBUG 1                       
const int pinRx = 5;                  
const int pinTx = 4;                  
SoftwareSerial sensor(pinTx,pinRx);  
const unsigned char cmd_get_sensor[] = 
{
    0xff, 0x01, 0x86, 0x00, 0x00, 
    0x00, 0x00, 0x00, 0x79
};
unsigned char dataRevice[9];          //CO2 Sensor
int temperature;                      // Store Temp - CO2 sensor
int CO2PPM;                           // Store CO2 ppm - CO2 sensor
float uvindex;                        // Store UV - UV Sensor
float mq2ratio;                       // Store mq2 - MQ2 Sensor
WiFly wifly;                          //WIFI
String data;                          //WIFI
const char mySSID[] = "myssid";
const char myPassword[] = "mypwd";
const char site[] = "api.thingspeak.com";
void terminal();
void setup(){ 
  Serial.begin(115200);
  Alarm.timerRepeat(100, MainAlarm); //Every 30 mins
  Serial.println("Alarms set!");
  wdt_enable(WDTO_8S);
}
void loop(){
  Alarm.delay(10); // wait one second between clock display
  if (wifly.moduleCrashed==true){
    Serial.println("moduleCrashed");
    wdt_enable(WDTO_8S);
    while (true);
  }
    wdt_reset();
    if(initialiseSensorFlag == true && millis() - initialiseSensorTimestamp > 3000){
      initialiseSensorFlag = false;    
      //SampleUVAlarm(); // samples UV & MQ2
      wdt_reset();
      wifiStuff(); // connect to Wifi
      wdt_reset();
      reportToCloud(); // post to web
    }
  }
void wifiStuff(){
   char buf[32];
   Serial.print("Free memory: ");
   Serial.println(wifly.getFreeMemory(),DEC);
     wifiSerial.begin(9600);   
     if (!wifly.begin(&wifiSerial, &Serial)) {
         Serial.println("Failed to start wifly");
    terminal();
     }
     /* Join wifi network if not already associated */
     if (!wifly.isAssociated()) {
    /* Setup the WiFly to connect to a wifi network */
    Serial.println("Joining network");
    wifly.setSSID(mySSID);
          wifly.setPassphrase(myPassword);
    wifly.enableDHCP();
    if (wifly.join()) {
        Serial.println("Joined wifi network");
    } else {
        Serial.println("Failed to join wifi network");
        terminal();
    }
     } else {
         Serial.println("Already joined network");
     }
     wifly.setDeviceID("Wifly-WebClient2");
     Serial.print("DeviceID: ");
     Serial.println(wifly.getDeviceID(buf, sizeof(buf)));
     if (wifly.isConnected()) {
         Serial.println("Old connection active. Closing");
    wifly.close();
     }
  }
void initialiseSensor(){
  sensor.begin(9600);
  Serial.begin(115200);
  Serial.println("*********");
}
void MainAlarm(){
  Serial.println("Main Alarm...");
  Serial.println(wifly.moduleCrashed);
  initialiseSensor();
  initialiseSensorFlag = true;
  initialiseSensorTimestamp = millis();
}
void reportToCloud() {
  data = "";
  Serial.println("Reporting to cloud...");
   if (wifly.available() > 0) {
  char ch = wifly.read();
  Serial.write(ch);
  if (ch == '\n') {
      /* add a carriage return */ 
      Serial.write('\r');
  }
   }
     if (wifly.open(site, 80)) {
         Serial.print("Connected to ");
         Serial.println(site);
          // Set data to send
          static char outstr1[15];
          static char outstr2[15];
          static char outstr3[15];
          static char outstr4[15];
          String dataString1 = dtostrf(uvindex, 8, 2, outstr1);
          String dataString2 = dtostrf(mq2ratio, 8, 2, outstr2);
          String dataString3 = "23.45";//dtostrf(CO2PPM, 8, 2, outstr3);
          String dataString4 = "67.89";//dtostrf(temperature, 8, 2, outstr4);
          //Reset all values
          uvindex = 0;
          mq2ratio = 0;
          CO2PPM = 0;
          temperature = 0;
          /* Send the request */
          String getData = "POST /update?api_key=mykey&field3=56.78&field4=23.45 HTTP/1.1";
          wifly.println("Host: api.thingspeak.com"); 
          wifly.println("Content-Type: application/x-www-form-urlencoded" );
          wifly.print("Content-Length: ");
          wifly.println(data.length());
          wifly.println();
          wifly.println(getData);
          Serial.println("Posted successfully");
     } else {
         Serial.println(">>Failed to connect");
     }
     if (Serial.available() > 0) {
    wifly.write(Serial.read());
     }
     //Added Sat 14 Nov @820am
     wifly.close();
  }
/* Connect the WiFly serial to the serial monitor. */
void terminal(){
   while (1) {
  if (wifly.available() > 0) {
      Serial.write(wifly.read());
  }
  if (Serial.available() > 0) {
      wifly.write(Serial.read());
  }
   }
}
void SampleUVAlarm (){
  Serial.println("Sampling UV");
  int sensorValue = 0;
  long  sum = 0;
  uvindex = 0;
  for(int i=0;i<1024;i++)
   {  
      sensorValue=analogRead(A0);
      sum=sensorValue+sum;
      delay(2);
   }   
 sum = sum >> 10;
 Serial.print("The UV voltage value:");
 Serial.print(sum*4980.0/1023.0);
 Serial.println(" mV");
 delay(20);
 Serial.println("UVIndex is: ");
 uvindex = (307*(sum*4980.0/1023.0))/200; //mW/m2 -> W/m2
 Serial.println(uvindex);
 Serial.print("\n");
 SampleMQ2Alarm();
}
void SampleMQ2Alarm() {
  Serial.println("Sampling MQ2");
  float sensor_volt = 0;
  float RS_gas = 0; // Get value of RS in a GAS
  float ratio = 0; // Get ratio RS_GAS/RS_air
  mq2ratio = 0;
  int sensorValue = analogRead(A1);
  sensor_volt=(float)sensorValue/1024*5.0;
  RS_gas = (5.0-sensor_volt)/sensor_volt; // omit *RL
    Serial.print("MQ2 Sensor_volt = ");
    Serial.println(sensor_volt);
    Serial.print("RS_ratio = ");
    Serial.println(RS_gas);
    Serial.print("Rs/R0 = ");
    Serial.println(ratio);
    mq2ratio = ratio;
    Serial.print("\n\n");
    delay(1000);
    //mq2ratio = 333;
    printCO2(); // sample co2
  }
  void printCO2() {
    Serial.println("printing co2...");
    if(dataRecieve())
    {
      Serial.print("Temperature: ");
      Serial.print(temperature);
      Serial.print("  CO2: ");
      Serial.print(CO2PPM);
      Serial.println("");
    }
    delay(1000);
  }
bool dataRecieve(void) {
  byte data[9];
  int i = 0;
  CO2PPM = 0;
  temperature = 0;
    //transmit command data
    for(i=0; i<sizeof(cmd_get_sensor); i++)
    {
      sensor.write(cmd_get_sensor[i]);
    }
    delay(10);
    //begin reveiceing data
    if(sensor.available())
    {
      while(sensor.available())
      {
        for(int i=0;i<9; i++)
        {
          data[i] = sensor.read();
        }
      }
    }
    #if DEBUG
    for(int j=0; j<9; j++)
    {
      Serial.print(data[j]);
      Serial.print(" ");    
    }
    Serial.println("");
    #endif
    if((i != 9) || (1 + (0xFF ^ (byte)(data[1] + data[2] + data[3] 
      + data[4] + data[5] + data[6] + data[7]))) != data[8])
    {
      Serial.println("false returned");
      return false;
    }
    CO2PPM = (int)data[2] * 256 + (int)data[3];
    temperature = (int)data[4] - 40;  
    Serial.println(temperature);
    return true;
  }
Christopher Stapels
Christopher Stapels on 14 Oct 2021 (Edited on 14 Oct 2021)

you use getData to send but you are using the length of only data

    data = "";
  ....
  wifly.print("Content-Length: ");
  wifly.println(data.length());
  wifly.println();
  wifly.println(getData);
Marcio Valenzuela
Marcio Valenzuela on 16 Oct 2021 (Edited on 16 Oct 2021)

Thank you, I corrected that. Im getting successful posts according to the Serial Monitor but I dont see the new data in ThingSpeak...

Christopher Stapels
Christopher Stapels on 18 Oct 2021 (Edited on 18 Oct 2021)

I would suggest you try the GET request format (see the Write data page), its more simple, and then you can focus on other errors there might be with the wifly device or something else. Make sure to wait for a response from the server before disconnecting.

Marcio Valenzuela
Marcio Valenzuela on 18 Oct 2021

Ok so I did this, using GET but I get pretty much the same result. Serial Monitor output at the bottom:

#include <WiFlyHQ.h>
#include <SoftwareSerial.h>
String field3 = "field3";
SoftwareSerial wifiSerial(2,3);
WiFly wifly;                          //WIFI
const char mySSID[] = "myssid";
const char myPassword[] = "mypwd";
const char site[] = "api.thingspeak.com";
void terminal();
void setup() {
  // put your setup code here, to run once:
  Serial.begin(115200);
}
void loop() {
  // put your main code here, to run repeatedly:
  wifiStuff();
}
void wifiStuff(){
   char buf[32];
   Serial.print("Free memory: ");
   Serial.println(wifly.getFreeMemory(),DEC);
     wifiSerial.begin(9600);   
     if (!wifly.begin(&wifiSerial, &Serial)) {
         Serial.println("Failed to start wifly");
    terminal();
     }
     /* Join wifi network if not already associated */
     if (!wifly.isAssociated()) {
    /* Setup the WiFly to connect to a wifi network */
    Serial.println("Joining network");
    wifly.setSSID(mySSID);
          wifly.setPassphrase(myPassword);
    wifly.enableDHCP();
    if (wifly.join()) {
        Serial.println("Joined wifi network");
    } else {
        Serial.println("Failed to join wifi network");
        terminal();
    }
     } else {
         Serial.println("Already joined network");
     }
     wifly.setDeviceID("Wifly-WebClient2");
     Serial.print("DeviceID: ");
     Serial.println(wifly.getDeviceID(buf, sizeof(buf)));
     reportToCloud();
     if (wifly.isConnected()) {
         Serial.println("Old connection active. Closing");
    wifly.close();
     }
  }
void reportToCloud() {
  Serial.println("Reporting to cloud...");
   if (wifly.available() > 0) {
  char ch = wifly.read();
  Serial.write(ch);
  if (ch == '\n') {
      /* add a carriage return */ 
      Serial.write('\r');
  }
   }
    if (wifly.open(site, 80)) {
         Serial.print("Connected to ");
         Serial.println(site);
          /* Send the request */
          String getData = "GET /update?api_key=mykey&field3=56.78 HTTP/1.0";
          /* Send the request */
          wifly.println("Host: api.thingspeak.com"); // SERVER ADDRESS HERE TOO
          wifly.print("Content-Length: ");
          wifly.println(getData.length());
          wifly.println();
          wifly.println(getData);
          Serial.println("Posted successfully");
     } else {
         Serial.println(">>Failed to connect");
     }
     if (Serial.available() > 0) {
    wifly.write(Serial.read());
     }
     wifly.close();
  }
/* Connect the WiFly serial to the serial monitor. */
void terminal(){
   while (1) {
  if (wifly.available() > 0) {
      Serial.write(wifly.read());
  }
  if (Serial.available() > 0) {
      wifly.write(Serial.read());
  }
   }
}

Serial Monitor results:

Free memory: 898
setPrompt hasnt been called
Joining network
Joined wifi network
DeviceID: Wifly-WebClient2
Reporting to cloud...
open api.thingspeak.com 80
Connected to api.thingspeak.com
Posted successfully
close: failed, no *CLOS*
Free memory: 898
Already joined network
DeviceID: Wifly-WebClient2
Reporting to cloud...
open api.thingspeak.com 80
Connected to api.thingspeak.com
Posted successfully
Christopher Stapels
Christopher Stapels on 19 Oct 2021

Is there still no data in the channel from that update? I would suggest adding a short delay (50 ms?) before you query for the response, and for a GET, you don't need to send the length or a blank line either. That probably wouldn't make it fail though.

Marcio Valenzuela
Marcio Valenzuela on 19 Oct 2021

ok so I commented out the content length lines and the blank line. I also added a

delay(100); 

right after the

Serial.println("Posted Successfully");

and I still get not data in TS.

Marcio Valenzuela
Marcio Valenzuela on 19 Oct 2021

Is it possible the data is being posted but im just not seeing it in the channel stats page? Where else could i see the table perhaps instead of a chart?

Christopher Stapels
Christopher Stapels on 19 Oct 2021

If the number of entries does not increase, there wont be any more data in the channel. You can use the Read API or the export recent data button on your channel view to download the recent data.

Christopher Stapels
Christopher Stapels on 20 Oct 2021

At present, authentication is by write API key for the REST API.

Marcio Valenzuela
Marcio Valenzuela on 20 Oct 2021

Ok so I created a new channel and was about to modify the arduino code when I noticed I dont have a channelid for my first channel. How does my arduino know where to post? I mean all this time that the yun project has been posting, how did it know?