SIM7600, MQTT, Thingspeak, all well until Publish the message

Jitse Schaafsma on 14 Oct 2023 (Edited on 16 Oct 2023)
Latest activity Edit by Christopher Stapels on 18 Dec 2023

Hi there,
I hope someone can help me on this.
I have written a code to connect my SIM7600 to Thingspeak using MQTT. It is basically an aruidino code. All instructions are echo's with an OK message so assume those instrictions are correct. However, the final step, publishing the message to the MQTT server return an error : CMQTTCONNLOST : 0,1.
Any idea which instructions might be wron?
I included the commands send.
10:09:09.031 -> AT+CMQTTPUB=0,0,60,0,0
10:09:09.031 -> +CMQTTPUB: 0,0
10:09:09.031 ->
10:09:09.031 -> OK
10:09:09.031 ->
10:09:09.031 -> +CMQTTCONNLOST: 0,1
The program :
SerialAT.begin(115200, SERIAL_8N1, RXD2, TXD2);
delay(1000);
SerialAT.println("AT+CRESET"); // Reset the SIM7600 module
Read_Response(30000);
SerialAT.println("AT+IPR=115200"); // Set Baudrate of the SIM7600 module
Read_Response(30000);
SerialAT.println("AT+CSQ"); // Check signal level
Read_Response(10000);
SerialAT.println("AT+COPS?"); // Query network information
Read_Response(10000);
SerialAT.println("AT+CMQTTSTOP"); //Stop MQTT service
Read_Response(10000);
SerialAT.println("AT+CMQTTSTART"); //Start MQTT service
Read_Response(10000);
SerialAT.println("AT+CSSLCFG=\"sslversion\",0,3"); //Configure SSL contect ssl_ctxindex,sslversion <0,TLS1.2>
Read_Response(10000);
SerialAT.println("AT+CSSLCFG=\"authmode\",0,0"); //Establishing MQTT Connection <ssl_ctx_index,authmode<0,no autentification>
Read_Response(10000);
SerialAT.println("AT+CMQTTACCQ=0,\"Username\",0,4"); // Acquire a client <client_index,client_ID>
Read_Response(10000);
SerialAT.println("AT+CMQTTSSLCFG=0,0"); // Set the SSL context <session_id,ssl_ctx_index> ssl_ctx_index check AT+CSSLCFG command
Read_Response(10000);
SerialAT.println("AT+CMQTTCONNECT=0,\"tcp://mqtt3.thingspeak.com:1883\",90,1,\"Username\",\"password\""); //Connect to MQTT server <client_index,server_addr,keep_alive time,clean_session,username,password>
Read_Response_Ignore_OK(3000);
String payload="\"channels/"+ String(ChannelId)+"/publish\"";
int Payload_Length;
Payload_Length=payload.length();
SerialAT.println("AT+CMQTTTOPIC=0,"+String(Payload_Length)); // Input the publish message topic <client_index,req_length>
SerialAT.println(payload);
Read_Response_Ignore_OK(3000);
payload="field1=12&field2=13&status=MQTTPUBLISH";
Payload_Length=payload.length();
SerialAT.println("AT+CMQTTPAYLOAD=0,"+String(Payload_Length)); // input the publish message body <client_index,req_length>
SerialAT.println(payload);
Read_Response_Ignore_OK(1000);
SerialAT.println("AT+CMQTTPUB=0,0,60,0,0"); // Publish a message to the server <client_id,qos,pub_timeout,retained>
Read_Response_Ignore_OK(1000);
Serial.print(Receive_buffer);
Serial.println("And we are done");
Christopher Stapels
Christopher Stapels on 16 Oct 2023 (Edited on 15 Dec 2023)
There is a good MQTT troubleshoot page with a bunch of things to try. Ill assume you already have a device set up in ThingSpeak and that you have enabled publishing to that channel.
Your code above has some strange slashes for the topic i'd have alook at this line.
String payload="\"channels/"+ String(ChannelId)+"/publish\"";
Also you dont show us what is in the variable MQTTPUBLISH.
You also dont show us the Read_Response() function, but issume it involves some wait time since there is a large number in the argument. If possible, I suggest writing the functions to complete when there is no more data instead of with a fixed timeout if possible.
My suggestion is to start with the desktop client to make sure you can get all the connection and publish syntax correct before moving to the device.
Jitse Schaafsma
Jitse Schaafsma on 16 Oct 2023
Thank you for your response.
To be honest, i have tried so many variations of the payload to be submitted as part of the instruction : AT+CMQTTTOPIC. The one in my post is just one of many variations. Also, i see other posts including the status=MQTTPUBLISH in the message body. Again, i have also tried it without this included. The Read_Response() function makes sure that i read all echo's send from the SIM7600 so that i know if something is going wrong.
And yes, i have read many posts, troubleshooting, and it is the overload of information that makes me unsure about the syntax requirements.
So my specific question is :
What is the correct syntax for the topic (i assume something in the range of channels/12345/publish in which 12345 is the ClientId as stated in the MQTT information pane when constructing the MQTT app
What is the correct syntax of the message body ( I assume something like field1=1234&field2=6788).
Oh btw, I am pretty sure I have an mqtt connection because AT+CMQTTCONNECT echo's an OK statement. But then again, if that instruction is not correct please let me know.
Og btw 2, I have also an mQTT thingspeak on a
Any specific feedback is welkom because using a SIM7080, works fine. But the same syntax does not appear to work for the SIM7800.
Any concrete feedback is welcome,
Regards
Jitse Schaafsma
Jitse Schaafsma on 16 Oct 2023 (Edited on 16 Oct 2023)
To continue,
For another channel i created a complete new MQTT. I have made sure that channel and fields can be written. I also secured my client_ID and password.
My SIM7600 has a connection because:
20:34:11.926 -> AT+CMQTTCONNECT=0,"tcp://mqtt3.thingspeak.com:1883",90,1,"******","****" returns OK +CMQTTCONNECT: 0,0 meaning that the connection is established.
Then I upload channels/788628/publish as topic to the SIM7600 module I receive an OK echo.
Then I upload field1=1234 as a message to the SIM7600 module I receive an OK echo
To finally upload all to the MQTT server I need to use this command:
AT+CMQTTPUB=0,0,60,0,0\r\n
I do receive then this echo:
+CMQTTPUB: 0,0
but also
+CMQTTCONNLOST: 0,1
which means connection lost.
Documentation says :
When client disconnect passively, URC “+CMQTTCONNLOST” will be reported, then user need to connect MQTT server again.
So it appears releated to the SIM7600 module. But i do not understand why it "passively disconnects".
For completeness, these are my general settings:
const String URL="tcp://mqtt3.thingspeak.com";
const String port="1883"; //"1883" in case of no encryption , 8883 in case of TLS/SSL encryption;
const String client_index="0";
const String Keep_alive_time="90";
const String Clean_session="1";
const String server_type="0"; //0 - MQTT server with TCP 1 - MQTT server with SSL/TLS
const String Mqtt_version="4"; //3 - MQTT version 3.1 4 - MQTT version 3.1.1
const String ssl_ctx_index="0"; // The SSL context ID. The range is 0-9
const String authmode="0"; // 0 no authentification, 1 server authentication. It needs the root CA of the server. , 2 server and client authentication. It needs the root CA of the server,
// the cert and key of the client, 3 client authentication and no server authentication. It needs the cert and key of the client
const String session_id="0"; // A numeric parameter that identifies a client. The range of permitted values is 0 to 1
const String sslversion="3"; //0 – SSL3.0 1 – TLS1.0 2 – TLS1.1 3 – TLS1.2 4 – All
Christopher Stapels
Christopher Stapels on 15 Dec 2023
Jitse Schaafsma
Jitse Schaafsma on 17 Dec 2023 (Edited on 18 Dec 2023)
For those interested,
This is the basic code that finally works for me:
Error_Code=0;
while (1) {
Serial2.print("AT+CLTS=1\r\n"); // Get local timestamp
Read_Response_WaitFor("OK",SIMWaitTime);
//Serial.println(Receive_buffer);
delay(Delay_between_commands);
Serial2.print("AT+CCLK?\r\n");
Read_Response_WaitFor("OK",SIMWaitTime);
//Serial.println(Receive_buffer);
delay(Delay_between_commands);
if (Process_DateandTime(false)) {Error_Code=4;}
Serial2.print("AT+CSSLCFG=\"SSLVERSION\",0,3\r\n");
Read_Response_WaitFor("OK",SIMWaitTime);
Serial.println(Receive_buffer);
delay(Delay_between_commands);
Serial2.print("AT+CNACT=0,1\r\n");
Read_Response_WaitFor("OK",SIMWaitTime);
Serial.println(Receive_buffer);
delay(Delay_between_commands);
Serial2.print("AT+CNACT?\r\n");
Read_Response_WaitFor("OK",SIMWaitTime);
Serial.println(Receive_buffer);
delay(Delay_between_commands);
if (!GET_IP_Address ())
{
Serial.println("No IP address");
Serial.println("Program stalled");
Error_Code=2;
break;
}
Serial2.print("AT+SMCONF=\"URL\",\"mqtt3.thingspeak.com\",\"1883\"\r\n");
Read_Response_WaitFor("OK",SIMWaitTime);
Serial.println(Receive_buffer);
delay(Delay_between_commands);
Serial2.print("AT+SMCONF=\"KEEPTIME\",60\r\n");
Read_Response_WaitFor("OK",SIMWaitTime);
Serial.println(Receive_buffer);
delay(Delay_between_commands);
Serial2.print("AT+SMCONF=\"CLEANSS\",1\r\n");
Read_Response_WaitFor("OK",SIMWaitTime);
Serial.println(Receive_buffer);
delay(Delay_between_commands);
Serial2.print("AT+SMCONF=\"QOS\",0\r\n");
Read_Response_WaitFor("OK",SIMWaitTime);
Serial.println(Receive_buffer);
delay(Delay_between_commands);
Serial2.print("AT+SMCONF=\"RETAIN\",0\r\n");
Read_Response_WaitFor("OK",SIMWaitTime);
Serial.println(Receive_buffer);
delay(Delay_between_commands);
Serial2.print("AT+SMCONF=\"SUBHEX\",0\r\n");
Read_Response_WaitFor("OK",SIMWaitTime);
Serial.println(Receive_buffer);
delay(Delay_between_commands);
Serial.println("AT+SMCONF=\"CLIENTID\",\""+String(SPIFFS_Data.ClientId)+"\"\r\n");
Serial2.print("AT+SMCONF=\"CLIENTID\",\""+String(SPIFFS_Data.ClientId)+"\"\r\n");
Read_Response_WaitFor("OK",SIMWaitTime);
Serial.println(Receive_buffer);
delay(Delay_between_commands);
Serial.println("AT+SMCONF=\"USERNAME\",\""+String(SPIFFS_Data.Username)+"\"\r\n");
Serial2.print("AT+SMCONF=\"USERNAME\",\""+String(SPIFFS_Data.Username)+"\"\r\n");
Read_Response_WaitFor("OK",SIMWaitTime);
Serial.println(Receive_buffer);
delay(Delay_between_commands);
Serial.println("AT+SMCONF=\"PASSWORD\",\""+ String(SPIFFS_Data.Password)+ "\"\r\n");
Serial2.print("AT+SMCONF=\"PASSWORD\",\""+ String(SPIFFS_Data.Password)+ "\"\r\n");
Read_Response_WaitFor("OK",SIMWaitTime);
Serial.println(Receive_buffer);
delay(Delay_between_commands);
Serial2.print("AT+SMCONN\r\n"); // Check MQTT connection
Read_Response_WaitFor("OK",SIMWaitTime);
Serial.println(Receive_buffer);
delay(Delay_between_commands);
if(strstr(Receive_buffer,"ERROR") != NULL)
{
Serial.println("SMCONN ERROR " );
Error_Code=3;
break;
}
else
{
Serial.println("We have a MQTT connection");
String F1="field"+String(SPIFFS_Data.FieldNr)+"=";
String F2="&field"+String(SPIFFS_Data.FieldNr+1)+"=";
String F3="&field"+String(SPIFFS_Data.FieldNr+2)+"=";
String F4="&field"+String(SPIFFS_Data.FieldNr+3)+"=";
String payload; payload=F1+String(int(pressure))+F2+String(int(tmp))+F3+String(int(hum))+F4+String(int(BatVoltage*100))+"&"+Date_payload;
String Mqtt_Message;
int Payload_Length;
Payload_Length=payload.length();
Serial.print(payload);
Serial.print (" ");
Serial.println(Payload_Length);
//Mqtt_Message="AT+SMPUB=\"channels/"+ ChannelId+"/publish/fields/field1\","+String(Payload_Length)+",1,1\r\n"; //https://nl.mathworks.com/help/thingspeak/publishtoachannelfieldfeed.html
Mqtt_Message="AT+SMPUB=\"channels/"+ String(SPIFFS_Data.ChannelId)+"/publish\","+String(Payload_Length)+",1,1\r\n"; //https://nl.mathworks.com/help/thingspeak/publishtoachannelfeed.html
Serial.println(Mqtt_Message);
Serial2.print(Mqtt_Message);//Serial2.print("AT+SMPUB=\"channels/888617/publish/fields/field1\",1,1,1\r\n"); // lengte moet goed zijn
delay(Delay_between_commands);
Serial2.print(payload);
Read_Response_WaitFor("OK",20000); //SIMWaitTime);
Serial.println(Receive_buffer);
break;
}
} //end while
Serial.print("Done with error code : ");
Serial.println(Error_Code);
if (Error_Code==0)
{
Serial.print("Going to sleep for ");
Serial.print(SPIFFS_Data.Sleep_Period);
Serial.print(" seconds");
esp_sleep_enable_timer_wakeup(SPIFFS_Data.Sleep_Period * uS_TO_S_FACTOR);
esp_deep_sleep_start();
}
else
{
Serial.print("Going to sleep for ");
Serial.print(Sleep_Period_After_Error);
Serial.print(" seconds");
esp_sleep_enable_timer_wakeup(Sleep_Period_After_Error * uS_TO_S_FACTOR);
esp_deep_sleep_start();
}
bool GET_IP_Address () {
char *p0;
char *p1;
int span;
p0=strstr(Receive_buffer,"CNACT: ");
if (p0!=NULL)
{
p0=p0+strlen("CNACT: ");
p1=strstr(p0,"\n");
if (p1!=NULL)
{
span=min(19,p1-p0);
strncpy(IP_buffer, p0, span);
IP_buffer[span]='\0';
Serial.print("IP_buffer : ");
Serial.println(IP_buffer);
Serial.print(" : ");
Serial.println(strlen(IP_buffer));
if (strlen(IP_buffer) >15) {return true;} else {return false;} // 0,1,"10.236.109.95"
}
else
{
return (false);
Serial.println("No IP address");
}
}
else
{
return (false);
}
}
bool Process_DateandTime(bool PSUTZ) {
char *Process_ptr;
bool Date_Error;
Date_Error=false;
//Process_ptr=strstr(Receive_buffer,"CCLK:");
if (PSUTZ) {Process_ptr=strstr(Receive_buffer,"PSUTTZ:");}
else {Process_ptr=strstr(Receive_buffer,"CCLK:");}
if (Process_ptr!=NULL)
{
Serial.println(Process_ptr);
strcpy(date_time_char_array,Process_ptr);
Serial.println(date_time_char_array);
int k=0;
for (int i=0; i<strlen(date_time_char_array); i++)
{
if (strchr(allowed_char,date_time_char_array[i])!=NULL)
{
date_time_char_array_processed[k]=date_time_char_array[i];
k++;
}
}
Serial.println(date_time_char_array_processed);
Process_ptr=date_time_char_array_processed+1;
Serial.println(Process_ptr);
while (1)
{
Process_ptr=strtok(Process_ptr,"/,:"); // 23/04/14,11:56:48,+08,1
if (Process_ptr==NULL) {DateTime_Error=true;break;}
Thisday.Year=atoi(Process_ptr);
Process_ptr=strtok(NULL,"/,:"); // 23/04/14,11:56:48,+08,1
if (Process_ptr==NULL) {DateTime_Error=true;break;}
Thisday.Month=atoi(Process_ptr);
Process_ptr=strtok(NULL,"/,:"); // 23/04/14,11:56:48,+08,1
if (Process_ptr==NULL) {DateTime_Error=true;break;}
Thisday.Day=atoi(Process_ptr);
Process_ptr=strtok(NULL,"/,:"); // 23/04/14,11:56:48,+08,1
if (Process_ptr==NULL) {DateTime_Error=true;break;}
Thisday.Hour=atoi(Process_ptr);
Process_ptr=strtok(NULL,"/,:"); // 23/04/14,11:56:48,+08,1
if (Process_ptr==NULL) {DateTime_Error=true;break;}
Thisday.Minute=atoi(Process_ptr);
Process_ptr=strtok(NULL,"/,:"); // 23/04/14,11:56:48,+08,1
if (Process_ptr==NULL) {DateTime_Error=true;break;}
Thisday.Second=atoi(Process_ptr);
Process_ptr=strtok(NULL,"/,:"); // 23/04/14,11:56:48,+08,1
if (Process_ptr==NULL) {DateTime_Error=true;break;}
Thisday.Timezone=atoi(Process_ptr);
Thisday.Hour=Thisday.Hour+(Thisday.Timezone/4);
break;
}
Serial.println();
Serial.print(Thisday.Year);
Serial.print("/");
Serial.print( Thisday.Month);
Serial.print("/");
Serial.print( Thisday.Day);
Serial.print(",");
Serial.print(Thisday.Hour);
Serial.print(":");
Serial.print( Thisday.Minute);
Serial.print(":");
Serial.print( Thisday.Second);
Serial.print(",");
Serial.println( Thisday.Timezone);
Date_payload="created_at=20"+String(Thisday.Year)+"-"+Int_to_2_Digits(Thisday.Month)+"-"+Int_to_2_Digits(Thisday.Day)+" "+Int_to_2_Digits(Thisday.Hour)+":"+Int_to_2_Digits(Thisday.Minute)+":"+Int_to_2_Digits(Thisday.Second);
Serial.println(Date_payload);
//Serial.println("Done with it");
//while (1) {delay(300);}
if (Thisday.Year<23) {Date_Error=true; }
if (Thisday.Month<0 or Thisday.Month>12 ) {Date_Error=true; }
if (Thisday.Day<0 or Thisday.Month>31 ) {Date_Error=true; }
if (Thisday.Hour<0 or Thisday.Hour>24 ) {Date_Error=true; }
if (Thisday.Minute<0 or Thisday.Minute>60 ) {Date_Error=true; }
if (Thisday.Second<0 or Thisday.Second>60 ) {Date_Error=true; }
}
else
{
//strcpy(fnameJPG,"AA00000001012011.JPG");
Date_Error=true;
}
return Date_Error;
}
bool Read_Response_WaitFor(char* StopCharArray,long time_out) { // false als niet gevonden
int k=0;
long wait_until;
char ccc;
//while (!Serial2.available() and millis()<wait_until) {delay(10);}
time_taken=millis();
wait_until=time_taken+time_out;
while (millis()<wait_until)
{
while (Serial2.available())
{
ccc=Serial2.read();
//Serial.print(ccc);
if (k<buffer_size-1)
{
Receive_buffer[k]=ccc;
k++;
}
}
Receive_buffer[k]=NULL;
if (strstr(Receive_buffer,StopCharArray)!=NULL)
{
//Serial.println("OK found");
// SIM7600_Error=0;
Receive_buffer[k]=NULL;
return false;
}
}
//if (k>=buffer_size-1) {SIM7600_Error=1; strcpy(Receive_buffer,SIM7600_Error_list[SIM7600_Error].c_str());return;}
//if (millis()>=wait_until) {SIM7600_Error=2; strcpy(Receive_buffer,SIM7600_Error_list[SIM7600_Error].c_str()); return;}
Receive_buffer[k]=NULL;
time_taken=millis()-time_taken;
Serial.println();
Serial.print("1] This operation took ");
Serial.print(time_taken);
Serial.println(" millisec");
return true; //if (strstr(Receive_buffer,"ERROR")!=NULL) {return true;} else {return false;}
}