ThingSpeak Client Secure communication down since 19/07/2022 for ESP32
Latest activity Reply by Austin Jacobs
on 31 Jul 2023
Hi,
I have been using ThingSpeak upload on several CO2 detectors based on ESP32 Arduino board, for six mounth.
Some are using direct API HTTPS requests to write values on ThingSpeak channels. Those are still working (through a proxy)
Others with similar programmetion but using ThingSpeak secure Communication Library for Arduino, have stopped uploading values since 19/07/2022 around 16h.
Was there any non publiched modification on the library at this time ?
Did any body notice something similar ?
I have not investigateg the error yet...
Thanks for feed back,
TC
25 Comments
Time Descending
Hi,
I too have been using ThingSpeak upload on a number of detectors and haven't had new code releases for about a year, one is based on ESP32 and its last post was on Tuesday 25th July 2023 at 18:02:09 GMT+0100.
I am using a ThingSpeak secure Communication Library for Arduino too.
I've re-booted the device twice and the program is running fine but it is still failing to post. I suspect there is a certificate issue. If that's what it is, I don't understand why its expired so soon. Accoding to Digi Cert "Root certificates also typically have long periods of validity, compared to intermediate certificates. They will often last for 10 or 20 years...". My last post on this thread was on 30th July 2022 - uncanny - almost a year to the day.
Please do share any updates, my heating and hot water is now in local fallback mode which is much less efficient :-(
@Austin Jacobs I hope you were able to leave the fallback mode now that we have provided the new cert.
Our security team shared this quote from digicert.
"This year DigiCert started updating default root certificate (DigiCert TLS RSA SHA256 2020 CA1) to second generation public root certificates (DigiCert Global G2 TLS RSA SHA256 2020 CA1) starting from March 2023. One of the reasons is to align with Mozilla's distrust policy regarding old root certificate which will take effect in 2025. Thus, any newly requested certificate after March is by default attached with a second generation root certificate and that is why *.thingspeak.com certificate is updated too.
They added that due to this policy, if they didnt do it this year, they would need to request an exemtion and still do the update next year. The current root certificate for ThingSpeak is valid till Mar 29, 2031, but you can expect a similar roll out for the root certificate in the year 2029.
Well that explains it. Thanks for investigating.
I have applied the new cert and it works just fine now - all tickign along as before.
Thanks again for the support.
Hello Austin,
Yes, this is exactly that. I'd believe there had been a security update on the server side, like last year.
I'am looking for a working CA root certificat to roll back to secure upload, but in the mean time same code are woking switching to non secure upload.
TC
I happen to have the exact same problem since 25th of July 2023 at 19h (french time), that is about exactly one year, did you had server update by any chance ? Or Is there any kind of expiring date with certificats ? Could you eventually issue a new certificat here please, beacuse I can not get one working from thingspeak.com like last time?
Thank you in advance.
Regards,
TC
It does appear that the certificate has changed again, perhaps it is yearly. Ill talk to the devs about that
-----BEGIN CERTIFICATE-----
MIIEyDCCA7CgAwIBAgIQDPW9BitWAvR6uFAsI8zwZjANBgkqhkiG9w0BAQsFADBh
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3
d3cuZGlnaWNlcnQuY29tMSAwHgYDVQQDExdEaWdpQ2VydCBHbG9iYWwgUm9vdCBH
MjAeFw0yMTAzMzAwMDAwMDBaFw0zMTAzMjkyMzU5NTlaMFkxCzAJBgNVBAYTAlVT
MRUwEwYDVQQKEwxEaWdpQ2VydCBJbmMxMzAxBgNVBAMTKkRpZ2lDZXJ0IEdsb2Jh
bCBHMiBUTFMgUlNBIFNIQTI1NiAyMDIwIENBMTCCASIwDQYJKoZIhvcNAQEBBQAD
ggEPADCCAQoCggEBAMz3EGJPprtjb+2QUlbFbSd7ehJWivH0+dbn4Y+9lavyYEEV
cNsSAPonCrVXOFt9slGTcZUOakGUWzUb+nv6u8W+JDD+Vu/E832X4xT1FE3LpxDy
FuqrIvAxIhFhaZAmunjZlx/jfWardUSVc8is/+9dCopZQ+GssjoP80j812s3wWPc
3kbW20X+fSP9kOhRBx5Ro1/tSUZUfyyIxfQTnJcVPAPooTncaQwywa8WV0yUR0J8
osicfebUTVSvQpmowQTCd5zWSOTOEeAqgJnwQ3DPP3Zr0UxJqyRewg2C/Uaoq2yT
zGJSQnWS+Jr6Xl6ysGHlHx+5fwmY6D36g39HaaECAwEAAaOCAYIwggF+MBIGA1Ud
EwEB/wQIMAYBAf8CAQAwHQYDVR0OBBYEFHSFgMBmx9833s+9KTeqAx2+7c0XMB8G
A1UdIwQYMBaAFE4iVCAYlebjbuYP+vq5Eu0GF485MA4GA1UdDwEB/wQEAwIBhjAd
BgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwdgYIKwYBBQUHAQEEajBoMCQG
CCsGAQUFBzABhhhodHRwOi8vb2NzcC5kaWdpY2VydC5jb20wQAYIKwYBBQUHMAKG
NGh0dHA6Ly9jYWNlcnRzLmRpZ2ljZXJ0LmNvbS9EaWdpQ2VydEdsb2JhbFJvb3RH
Mi5jcnQwQgYDVR0fBDswOTA3oDWgM4YxaHR0cDovL2NybDMuZGlnaWNlcnQuY29t
L0RpZ2lDZXJ0R2xvYmFsUm9vdEcyLmNybDA9BgNVHSAENjA0MAsGCWCGSAGG/WwC
ATAHBgVngQwBATAIBgZngQwBAgEwCAYGZ4EMAQICMAgGBmeBDAECAzANBgkqhkiG
9w0BAQsFAAOCAQEAkPFwyyiXaZd8dP3A+iZ7U6utzWX9upwGnIrXWkOH7U1MVl+t
wcW1BSAuWdH/SvWgKtiwla3JLko716f2b4gp/DA/JIS7w7d7kwcsr4drdjPtAFVS
slme5LnQ89/nD/7d+MS5EHKBCQRfz5eeLjJ1js+aWNJXMX43AYGyZm0pGrFmCW3R
bpD0ufovARTFXFZkAdl9h6g4U5+LXUZtXMYnhIHUfoyMo5tS58aI7Dd8KvvwVVo4
chDYABPPTHPbqjc1qCmBaZx2vN4Ye5DUys/vZwP9BFohFrH/6j/f3IL16/RZkiMN
JCqVJUzKoZHm1Lesh3Sz8W2jmdv51b2EQJ8HmA==
-----END CERTIFICATE-----
Hello Christopher,
Thank you again. I could roll back to secure upload with this certificate. This is working fine anew.
Yes please give us feed back from the devs, if this had to be renewed yearly I'll probably come back next year...
The thing I cannot figure out, is why I am not able to get one working certificate myself with the given method :
I go to the lock of Chrome on thingspeak.com, click on "secure connection" ,then open "valid certificate" and export one from the small window as a .PEM file. I do get something looking like yours, but not working... I also tried from FireFox, but not better, though different .PEM file. Why do I get it wrong ?
Regards,
TC
I've found some confusion myself on the step before valid certificate. There may be two or more certificates to select (root or lower) and you might not be choosing the right one.
I'll try again. Thanks.
There are two certificates, looking like this in detail tab:
ESET SSL Filter CA
*.thingspeak.com
I think I want the highest; anyway I tried both, but none of them are working.
There is definitely something I don't get...
We are not yet aware of any issues with the library, but can you tell us what you are seeing from the device when the post fails?
Are you using the secure connection? ThingSpeak certificate may have been updated near that time, you might need to update the firmware.
Thanks for your patience. It seems some of the library examples for secure connections got jumbled in the last publish. We will plan to update the examples shortly. In the meantime, you can download the new cretificate.
In firefox
- Got to thingspeak.com. Click on the lock icon
- Click on Connection secure -> More information
- Click on view certificate
- Click on "DigiCert Global Root CA" in the tab
- Click on PEM(cert) under Miscellaneous
- Right click on the downloaded file. Open with any editor.
In Chrome,
- Got to thingspeak.com. Click on the lock icon
- Click on Connection secure -> certificate is valid
- Click Cert path tab
- select the top level Digicert
- View certificate
- Details tab
- Copy to file
- Choose base 64 encoded x 509
- View with text editor.
I was able to use the thingspeak library secure connection with esp32 with this cert, but you need to add these lines
in secrets.h:
client.setCACert(SECRET_TS_ROOT_CA);
#define SECRET_TS_ROOT_CA "-----BEGIN CERTIFICATE-----\n" \
... (cert you got above)
"-----END CERTIFICATE-----\n"
in main file:
const char* certificate = SECRET_TS_ROOT_CA;
% in setup fucntion
client.setCACert(certificate); // Set Root Certificate for authenticity check
I pulled out this simple case from WriteSingleFieldSecure GitHub's example with the adivesed modifications. It uploads values from 400 to 450 to my TS channel N°5 evry 30 seconds. It is not working I get -301 error code each time (similar code with non secure client do works). Please tel me if there is something obviously wrong, or if it is about CA certificat (I put it clearly the one got from thingspeak.com with the advise method).
TT
/*
WriteSingleField
Description: Writes a value to a channel on ThingSpeak every 20 seconds over secured HTTPS connection.
Hardware: ESP32 based boards
!!! IMPORTANT - Modify the secret.h file for this project with your network connection and ThingSpeak channel details. !!!
Note:
- Requires installation of EPS32 core and WiFiClientSecure library. See https://github.com/espressif/arduino-esp32/blob/master/docs/arduino-ide/boards_manager.md for details.
- Select the target hardware from the Tools->Board menu
- This example is written for a network using WPA encryption. For WEP or WPA, change the WiFi.begin() call accordingly.
ThingSpeak ( https://www.thingspeak.com ) is an analytic IoT platform service that allows you to aggregate, visualize, and
analyze live data streams in the cloud. Visit https://www.thingspeak.com to sign up for a free account and create a channel.
Documentation for the ThingSpeak Communication Library for Arduino is in the README.md folder where the library was installed.
See https://www.mathworks.com/help/thingspeak/index.html for the full ThingSpeak documentation.
For licensing information, see the accompanying license file.
Copyright 2020, The MathWorks, Inc.
*/
#define TS_ENABLE_SSL // For HTTPS SSL connection
#include <WiFiClientSecure.h>
#include "secret.h"
#include "ThingSpeak.h" // always include thingspeak header file after other header files and custom macros
char ssid[] = SECRET_SSID; // your network SSID (name)
char pass[] = SECRET_PASS; // your network password
WiFiClientSecure client;
unsigned long myChannelNumber = SECRET_CH_ID;
const char * myWriteAPIKey = SECRET_WRITE_APIKEY;
// const char* certificate = SECRET_TS_ROOT_CA;
const char* certificate = "-----BEGIN CERTIFICATE-----\n" \
"MIIDgDCCAmigAwIBAgIQNerH/sDRZ3n9Yvcpt60igTANBgkqhkiG9w0BAQsFADBI\n" \
"MRswGQYDVQQDDBJFU0VUIFNTTCBGaWx0ZXIgQ0ExHDAaBgNVBAoME0VTRVQsIHNw\n" \
"b2wuIHMgci4gby4xCzAJBgNVBAYTAlNLMB4XDTIyMDYzMDA5NDIxN1oXDTMyMDYy\n" \
"NzA5NDIxN1owSDEbMBkGA1UEAwwSRVNFVCBTU0wgRmlsdGVyIENBMRwwGgYDVQQK\n" \
"DBNFU0VULCBzcG9sLiBzIHIuIG8uMQswCQYDVQQGEwJTSzCCASIwDQYJKoZIhvcN\n" \
"AQEBBQADggEPADCCAQoCggEBAN3XvsuwGuJDYyciXcaGG5kyxCPTO9m55OvEu45X\n" \
"KCFiFXAEZPJYWeKptzQfv/RO5hPgIWDHfQPE5fb+61pV7w9dGue3mErg3Zw6DjDN\n" \
"PFk5YYzHlCRlSSocxuuFdKPMnuTPNkPSv8nmoYelhTpNfDRX5TVMml83bXB/ISYL\n" \
"O5SSpvhg0GSJtnJeZZOk5qXpImJr6/Im1BlQaMq8cbE944NPFfaDy3bSpdiTGv8Z\n" \
"WxRsuFiiJLjQ2Xs9HJYw8HUJeI9BBFZZZWpw7HGZaqlW1ADxl8qv3Kwduxb0CsV7\n" \
"R+5Utghg40c0i67bs5vE4IzdqrPrDtNNOx4d3/e7l58/4I8CAwEAAaNmMGQwDgYD\n" \
"VR0PAQH/BAQDAgIEMBIGA1UdEwEB/wQIMAYBAf8CAQAwHQYDVR0OBBYEFGgdMYoY\n" \
"0JSXdYADatVZUf/UZRpzMB8GA1UdIwQYMBaAFGgdMYoY0JSXdYADatVZUf/UZRpz\n" \
"MA0GCSqGSIb3DQEBCwUAA4IBAQBDGPrqHrG9OCSE8sVC2+yqijcuyGg7EZmEejPU\n" \
"Ev/+Elt6FoYBfSK9b961uVQ61KnDOIOHdm1GAthn2LwroAHCscOg/6Spb6XYybo6\n" \
"otwdt354rEeswh2rJWlwy8F1BUTZKT05XmgzrdSBJEVOcapv1VsmGqLz0aa8tnWw\n" \
"BlXrTXmvBSQRjiKhmv2X3BC/FT/ir8TWxQPyxC6cJTGvN/E3+dCHJsGYBZEzHT60\n" \
"H/PWuDp9B/7BsKHSba/frTqrjnMvGVgyAMy+4DFlqdb8O1tZUke5DBJJ7DmPRYRM\n" \
"AuD9BRS2uo03jjoCMB+8qW/0zyV531BRu8l6+PlNvQW8dZV3\n" \
"-----END CERTIFICATE-----\n";
int number = 400;
void setup() {
client.setCACert(certificate); // Set Root Certificate for authenticity check
Serial.begin(115200); //Initialize serial
while (!Serial) {
; // wait for serial port to connect. Needed for Leonardo native USB port only
}
Serial.println(certificate);
WiFi.mode(WIFI_STA);
ThingSpeak.begin(client); // Initialize ThingSpeak
}
void loop() {
// Connect or reconnect to WiFi
if(WiFi.status() != WL_CONNECTED){
Serial.print("Attempting to connect to SSID: ");
Serial.println(SECRET_SSID);
while(WiFi.status() != WL_CONNECTED){
WiFi.begin(ssid, pass); // Connect to WPA/WPA2 network. Change this line if using open or WEP network
Serial.print(".");
delay(5000);
}
Serial.println("\nConnected.");
}
// Write to ThingSpeak. There are up to 8 fields in a channel, allowing you to store up to 8 different
// pieces of information in a channel. Here, we write to field 1.
int x = ThingSpeak.writeField(myChannelNumber, 5, number, myWriteAPIKey);
if(x == 200){
Serial.println("Channel update successful.");
Serial.println( number);
}
else{
Serial.println("Problem updating channel. HTTP error code " + String(x));
}
// change the value
number++;
if(number > 450){
number = 400;
}
delay(30000); // Wait 30 seconds to update the channel again
}
Does the insecure mothod work for you? The get certificate method is very finiky, I may not have done a perfect job explaining how to get the cert. Please let me know about insecure, and try the get cert method once more.
Hello,
Yes, insecure method is working. In the same code, if I remove "#define TS_ENABLE_SSL" and replace "WiFiClientSecure client;" by "WiFiCleint client;",this is working.
Indeed, the described method to get the cert may not be up to date, but on the principle I think I got the method : getting the PEM(cert) of DigiCert Global Root CA from thingspeak.com web site either with FireFox or Chrome. Actually, this gives the same cert, so I thought it was just right, but if you can confirm that the posted cert is good, that would help. Maybe I got something wrong.
Thanks,
TT
This is the cert I used successfully
#define SECRET_TS_ROOT_CA "-----BEGIN CERTIFICATE-----\n" \
"MIIDrzCCApegAwIBAgIQCDvgVpBCRrGhdWrJWZHHSjANBgkqhkiG9w0BAQUFADBh\n" \
"MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\n" \
"d3cuZGlnaWNlcnQuY29tMSAwHgYDVQQDExdEaWdpQ2VydCBHbG9iYWwgUm9vdCBD\n" \
"QTAeFw0wNjExMTAwMDAwMDBaFw0zMTExMTAwMDAwMDBaMGExCzAJBgNVBAYTAlVT\n" \
"MRUwEwYDVQQKEwxEaWdpQ2VydCBJbmMxGTAXBgNVBAsTEHd3dy5kaWdpY2VydC5j\n" \
"b20xIDAeBgNVBAMTF0RpZ2lDZXJ0IEdsb2JhbCBSb290IENBMIIBIjANBgkqhkiG\n" \
"9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4jvhEXLeqKTTo1eqUKKPC3eQyaKl7hLOllsB\n" \
"CSDMAZOnTjC3U/dDxGkAV53ijSLdhwZAAIEJzs4bg7/fzTtxRuLWZscFs3YnFo97\n" \
"nh6Vfe63SKMI2tavegw5BmV/Sl0fvBf4q77uKNd0f3p4mVmFaG5cIzJLv07A6Fpt\n" \
"43C/dxC//AH2hdmoRBBYMql1GNXRor5H4idq9Joz+EkIYIvUX7Q6hL+hqkpMfT7P\n" \
"T19sdl6gSzeRntwi5m3OFBqOasv+zbMUZBfHWymeMr/y7vrTC0LUq7dBMtoM1O/4\n" \
"gdW7jVg/tRvoSSiicNoxBN33shbyTApOB6jtSj1etX+jkMOvJwIDAQABo2MwYTAO\n" \
"BgNVHQ8BAf8EBAMCAYYwDwYDVR0TAQH/BAUwAwEB/zAdBgNVHQ4EFgQUA95QNVbR\n" \
"TLtm8KPiGxvDl7I90VUwHwYDVR0jBBgwFoAUA95QNVbRTLtm8KPiGxvDl7I90VUw\n" \
"DQYJKoZIhvcNAQEFBQADggEBAMucN6pIExIK+t1EnE9SsPTfrgT1eXkIoyQY/Esr\n" \
"hMAtudXH/vTBH1jLuG2cenTnmCmrEbXjcKChzUyImZOMkXDiqw8cvpOp/2PV5Adg\n" \
"06O/nVsJ8dWO41P0jmP6P6fbtGbfYmbW0W5BjfIttep3Sp+dWOIrWcBAI+0tKIJF\n" \
"PnlUkiaY4IBIqDfv8NZ5YBberOgOzW6sRBc4L0na4UU+Krk2U886UAb3LujEV0ls\n" \
"YSEY1QSteDwsOoBrp+uvFRTp2InBuThs4pFsiv9kuXclVzDAGySj4dzp30d8tbQk\n" \
"CAUw7C29C79Fv1C5qfPrmAESrciIxpg0X40KPMbp1ZWVbd4=\n" \
"-----END CERTIFICATE-----\n"
Hello again,
YES, this is working with this cert, just like before. So I guess everything is about getting a good cert, and it seems I don't have the right method. I will work on that point, but if you could ever update the method, I could use it right. I am not figuring out yet what I do wrong, so I have my detectors all work with that one cert.
Thank you,
TT
Thank you, I am going to try that one.
Hello again,
So, I have tried to roll back to secure connection with esp32 using new downloaded certificate from thingspeak.com, but nothing to do, I am still getting the -301 error return code, either with thingspeak library or direct URL API GET. I also upgraded with new available card firmware v2.0.5, but the same results. I am stuck with non-secure connection which is working fine.
TT
NOTE : it is not a "DigiCert Global Root CA" but an "ESET SSL Filter CA" doest it mind ?
Hello,
OK, i'll try again in september as I am away for the month and post the results.
Thank.
TT
Hi,
Thanks for feedback.
Yes, indeed, I am using secure connections.
The reason why I suspected an eventual different return from the library request is that fails happened on two different networks at the same time. Thought the ones without library are making the https request through a proxy, so that might be a good hint. I'll be upgrading the ESP dev boar from v 2.0.2 to v2.0.4, and let you know.
My other main problem is to monitor the errors, on the actual networks where it failed. But I'll do that soon and let the community know.
TC
Hi,
So I am getting a -301 return code, where I used to get the 200. So this is a simple Failed to connect to ThingSpeak, which I don't understand yet.
Firmware update did not help. But I guess I am in the situation where Client is capable of doing SSL and TS_ENABLE_SSL macro is undefined (github page Case 3), still TS_ENABLE_SSL macro is defined.
So I am not understanding yet what changed leading to connection fails to ThingSpeak...
As I mentioned that happened on two different networks at the same time.
Https certificate seems to be valid.
I need to switch to non-secured connection to try...
I am open to any other suggestion.
Thanks.
TC
I can confirm the -301 for me as well, both using the old fingerprint and the new one. The non - secure connection works for the same device. We are looking into this, ill let you know when we find out what is happening
Hi,
Yes, I checked unsecured connection, it is working anew.
Actually, I retested the 4-case confidentiality versus authenticity:
Case 1: TS_ENABLE_SSL defined + Client secure --> normal case by not working with -301 return code !
Case 2: TS_ENABLE_SSL defined + Client unsecure --> get the warning return code 400 , value uploaded but need to change my code logic...
Case 3: TS_ENABLE_SSL undefined + Client secure --> expected error connecting with return code -301
Case 4: TS_ENABLE_SSL undefined + Client unsecure --> working with return code 200 (normal but not SSL)
I also try different root certificates for secure clients but full secured case 1 is still not working.
I am not sure of what to update on my side to make it work, so if you identify something on your side, that would be great.
Thank,
TC
Thanks TT, That explains why I have one working (ESP 01 unsecure I.e. case 4 above) and one stopped working (ESP32 secure I.e case 3 above)
Sign in to participate
