aws/aws-iot-device-sdk-cpp

SDK for connecting to AWS IoT from a device using C++

aws-iot-device-sdk-cpp's Issues

No information about aws-iot-device-sdk-cpp library in AWS IoT documentation

Hello,

I am planning to use AWS IoT PubSub functions for one of my projects and needed client SDK for the same. AWS Developer guide mentions about aws-iot-device-sdk-embedded-C but I could not find any documentation for this SDK.

If possible, I would like to use cpp sdk in our production code instead of the embedded c one, but lack of mention about this sdk in any official AWS documentations makes me wonder if this sdk is intended to be used for production yet.

Could you please clarify this?

issue about compile the source code

when I download the code ,and run "cmake ../.", there is configure error as below:

• cmake ../.
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /home/aws/aws-iot-device-sdk-cpp/build/third_party/rapidjson/download
  [100%] Built target rapidjson
  CMake Error at /usr/share/cmake-3.2/Modules/FindPackageHandleStandardArgs.cmake:138 (message):
  Could NOT find Threads (missing: Threads_FOUND)
  Call Stack (most recent call first):
  /usr/share/cmake-3.2/Modules/FindPackageHandleStandardArgs.cmake:374 (_FPHSA_FAILURE_MESSAGE)
  /usr/share/cmake-3.2/Modules/FindThreads.cmake:204 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  CMakeLists.txt:69 (find_package)

-- Configuring incomplete, errors occurred!

Creating libaws-iot-sdk-cpp for Android

Can I use this library for Android project ?
I have C++ project that will use this library.
Everything in Linux 14.04 desktop is OK
When I cross-compile this project for Android
it can not find -laws-iot-sdk-cpp
I also put library to lib folder of toolchain or /usr/local/lib and include all header file of library.
Questions are:

• Can I use this library for C++ project but cross-compile for Android ? (I'm using Qt)
• Can I build this library as shared library (file .so) with cross compiler ?

Note:

1. I added some texts to cmake files but failed

set(CMAKE_SYSTEM_NAME Android)

set(CMAKE_SYSTEM_VERSION 1)

set(CMAKE_C_COMPILER /opt/arm-linux-androideabi-4.8/bin/arm-linux-androideabi-gcc)
set(CMAKE_CXX_COMPILER /opt/arm-linux-androideabi-4.8/bin/arm-linux-androideabi-g++)

2. I also tried Cross-Compile with Cmake command (Removed text in cmake files of Note .1

/opt/cmake/bin/cmake ../. -DCMAKE_TOOLCHAIN_FILE=//home/nntchau/Android/Sdk/ndk-bundle/build/cmake/android.toolchain.cmake

Output is as below

-- Android: Selected GCC toolchain 'arm-linux-androideabi-4.9'
-- Check for working CXX compiler: //home/nntchau/Android/Sdk/ndk-bundle/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-g++
-- Check for working CXX compiler: //home/nntchau/Android/Sdk/ndk-bundle/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-g++ -- broken
CMake Error at /opt/cmake/share/cmake-3.8/Modules/CMakeTestCXXCompiler.cmake:44 (message):
The C++ compiler
"//home/nntchau/Android/Sdk/ndk-bundle/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-g++"
is not able to compile a simple test program.

It fails with the following output:

Change Dir: /home/nntchau/aws-iot-device-sdk-cpp-1.1.1/build_share/CMakeFiles/CMakeTmp

Run Build Command:"/usr/bin/make" "cmTC_2027d/fast"

/usr/bin/make -f CMakeFiles/cmTC_2027d.dir/build.make
CMakeFiles/cmTC_2027d.dir/build

make[1]: Entering directory
`/home/nntchau/aws-iot-device-sdk-cpp-1.1.1/build_share/CMakeFiles/CMakeTmp'

Building CXX object CMakeFiles/cmTC_2027d.dir/testCXXCompiler.cxx.o

//home/nntchau/Android/Sdk/ndk-bundle/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-g++
--target=armv7-none-linux-androideabi
--sysroot=//home/nntchau/Android/Sdk/ndk-bundle/platforms/android-9/arch-arm
-isystem
//home/nntchau/Android/Sdk/ndk-bundle/sources/cxx-stl/gnu-libstdc++/4.9/include
-isystem
//home/nntchau/Android/Sdk/ndk-bundle/sources/cxx-stl/gnu-libstdc++/4.9/libs/armeabi-v7a/include
-isystem
//home/nntchau/Android/Sdk/ndk-bundle/sources/cxx-stl/gnu-libstdc++/4.9/include/backward
-isystem
//home/nntchau/Android/Sdk/ndk-bundle/platforms/android-9/arch-arm/usr/include
-g -DANDROID -ffunction-sections -funwind-tables -fstack-protector-strong
-no-canonical-prefixes -march=armv7-a -mfloat-abi=softfp -mfpu=vfpv3-d16
-fno-integrated-as -mthumb -Wa,--noexecstack -Wformat
-Werror=format-security -O0 -fno-limit-debug-info -o
CMakeFiles/cmTC_2027d.dir/testCXXCompiler.cxx.o -c
/home/nntchau/aws-iot-device-sdk-cpp-1.1.1/build_share/CMakeFiles/CMakeTmp/testCXXCompiler.cxx

arm-linux-androideabi-g++: error: unrecognized command line option
'-fno-integrated-as'

arm-linux-androideabi-g++: error: unrecognized command line option
'-fno-limit-debug-info'

make[1]: *** [CMakeFiles/cmTC_2027d.dir/testCXXCompiler.cxx.o] Error 1

make[1]: Leaving directory
`/home/nntchau/aws-iot-device-sdk-cpp-1.1.1/build_share/CMakeFiles/CMakeTmp'

make: *** [cmTC_2027d/fast] Error 2

CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
CMakeLists.txt:2 (project)

Garbled text at compile in CMD, on Win10 zh_CN

as the pic below.
I tried common Chinese charset, such as UTF-8, GBK, GB2132. And doesn't work.

MAX_RW_BUF_LEN not large enough for WebSocketConnection::WssHandshake

The WSS HTTP GET request formed in WebSocketConnection::WssHandshake can exceed 2048 bytes, which is the size of MAX_RW_BUF_LEN. The error traced out is that the WSS handshake fails because there are no SSL packets to read. In fact the request has only been partially written because the buffer wasn't large enough to hold all of it in one go, so the IoT service hasn't sent a response.

Subscribing to an already subscribed topic will result in timeout (SDK error -703)

Hello,

I noticed that subscribing to an already subscribed topic results in -703 error.

This happens because SubscribeActionAsync::PerformAction filters out already subscribed topics here but does not check if the filtered topic list is empty or not before sending the SUB packet.

The timeout occurs because the client is waiting for an ACK which is never coming since nothing was sent to the server.

I would recommend checking for empty sub packet here before writing to network to resolve this issue. (I am assuming that the behavior is not by design. If so please let me know.)

AWS SDK Fails to build with OpenSSL1.1

When building against OpenSSL1.1, the following compile error is seen (this is without quieting deprecated declarations, obviously):

/home/kmiller/GitLab/data-management/AWS_SDK/aws-iot-device-sdk-cpp/network/OpenSSL/OpenSSLConnection.cpp: In destructor ‘awsiotsdk::network::OpenSSLInitializer::~OpenSSLInitializer()’: /home/kmiller/GitLab/data-management/AWS_SDK/aws-iot-device-sdk-cpp/network/OpenSSL/OpenSSLConnection.cpp:45:41: error: ‘void ERR_remove_thread_state(void*)’ is deprecated [-Werror=deprecated-declarations] ERR_remove_thread_state(NULL); ^ In file included from /usr/include/openssl/ct.h:13:0, from /usr/include/openssl/ssl.h:61, from /home/kmiller/GitLab/data-management/AWS_SDK/aws-iot-device-sdk-cpp/network/OpenSSL/OpenSSLConnection.hpp:40, from /home/kmiller/GitLab/data-management/AWS_SDK/aws-iot-device-sdk-cpp/network/OpenSSL/OpenSSLConnection.cpp:25: /usr/include/openssl/err.h:247:1: note: declared here DEPRECATEDIN_1_1_0(void ERR_remove_thread_state(void *))

Could this call to ERR_remove_thread_state just be simply wrapped in the following to compile with SSL 1.1.0f?

#if OPENSSL_VERSION_NUMBER >= 0x10000000L && OPENSSL_VERSION_NUMBER < 0x10100000L
...
#endif 

?

When subscribing to a topic with wildcards, callback is not called

When an application subscribes to a topic using wildcards '+' or '#', it receives Publish messages from the server, but the function ClientState::GetSubscription cannot find the proper subscription and therefore the application handler callback is not called.

The following patch seems to resolve the issue for me:

--- src/mqtt/ClientState.cpp 22 May 2017 15:59:16 -0000 1.1.1.1
+++ src/mqtt/ClientState.cpp 26 May 2017 16:49:56 -0000
@@ -54,11 +54,43 @@

            std::shared_ptr<Subscription> ClientState::GetSubscription(util::String p_topic_name) {
                    std::shared_ptr<Subscription> p_sub = nullptr;

•                   util::Map<util::String, std::shared_ptr<Subscription>>::const_iterator itr = subscription_map_.find(p_topic_name);
                    if(itr != subscription_map_.end()) {
                            p_sub = itr->second;
  

•            return p_sub;
                    }
  

•        //  Handle '+' or '#'
  

•        for (const auto &pair : subscription_map_) {
  

•            size_t index1, index2;
  

•            for (index1 = index2 = 0; index1 < p_topic_name.size() && index2 < pair.first.size(); ) {
  

•                if (pair.first[index2] == '#' && index2 + 1 >= pair.first.size()) {
  

•                    //  Terminating '#'
  

•                    p_sub = pair.second;
  

•                    return p_sub;
  

•                }
  

•                if (pair.first[index2] == '+' && (index2 == 0 || pair.first[index2 - 1] == '/') && (index2 + 1 >= pair.first.size() || pair.first[index2 + 1] == '/')) {
  

•                    //  A '+' that replaces a whole topic
  

•                    index2++;
  

•                    while (index1 < p_topic_name.size() && p_topic_name[index1] != '/') {
  

•                        index1++;
  

•                    }
  

•                } else if (p_topic_name[index1] == pair.first[index2]) {
  

•                    index1++;
  

•                    index2++;
  

•                } else {
  

•                    //  Mismatch found
  

•                    break;
  

•                }
  

•            }
  

•            if (index1 >= p_topic_name.size() && index2 >= pair.first.size()) {
  

•                //  Match found
  

•                p_sub = pair.second;
  

•                return p_sub;
  

•            }
  

•        }
                    return p_sub;
            }
  

JsonParserTester.RunTests unit test aborts

After a fresh pull and successful compile, ./bin/aws-iot-unit-tests aborts with the following message:

aws-iot-device-sdk-cpp/src/tests/unit/src/util/JsonParserTests.cpp:140: Failure
      Expected: ResponseCode::SUCCESS
      Which is: 4-byte object <00-00 00-00>
To be equal to: rc
      Which is: 4-byte object <9C-FF FF-FF>
aws-iot-unit-tests: aws-iot-device-sdk-cpp/bin/third_party/rapidjson/src/include/rapidjson/document.h:1128: rapidjson::GenericValue<Encoding, Allocator>::ConstMemberIterator rapidjson::GenericValue<Encoding, Allocator>::MemberEnd() const [with Encoding = rapidjson::UTF8<>; Allocator = rapidjson::MemoryPoolAllocator<>; rapidjson::GenericValue<Encoding, Allocator>::ConstMemberIterator = rapidjson::GenericMemberIterator<true, rapidjson::UTF8<>, rapidjson::MemoryPoolAllocator<> >]: Assertion `IsObject()' failed.
Aborted

Feature request: Reconnect and resubscribe callback

Hello,

It would be nice if SDK notifies about reconnect and resubscribe events to applications when the Keep-Alive runner thread reconnects and resubscribes.

Thanks.

Can not use class of library

Sorry for asking.
I compiled library and installed to PC (Ubuntu 14.04)
But when I use this library to a fresh C++ program (Code by Qt)
Classes can not be called.
After I trace down to header file
I got this in NetworkConnection.hpp
AWS_API_EXPORT class NetworkConnection {

But when I looked into macro "AWS_API_EXPORT", "__declspec(dllexport)" is blocked by defined macro.

#pragma once
#if defined (USE_WINDOWS_DLL_SEMANTICS) || defined (WIN32)
#ifdef _MSC_VER
#pragma warning(disable : 4251)
#endif // _MSC_VER

#ifdef USE_EXPORT
#define  AWS_API_EXPORT __declspec(dllexport)
#else // USE_EXPORT
#define AWS_API_EXPORT
#endif // USE_EXPORT
#else // defined (USE_WINDOWS_DLL_SEMANTICS) || defined (WIN32)
#define AWS_API_EXPORT
#endif // defined (USE_WINDOWS_DLL_SEMANTICS) || defined (WIN32)

What is this purpose ?

Note: I'm just try to build & use this library for native ubuntu C++ program

asan warning: alloc-dealloc-mismatch (operator new [] vs operator delete)

see https://github.com/aws/aws-iot-device-sdk-cpp/blob/master/src/util/logging/FormattedLogSystem.cpp#L72

when it goes out of scope at line 85 the wrong de-allocator is used

the fixed line is:
std::unique_ptr<char[]> outputBuff_uptr = std::unique_ptr<char[]>(new char[requiredLength]); (added array brackets to the types)

there are many other asan warnings when running integration and unit tests (see #15 for instructions). the rest seem to be leaks. generally, valgrind should catch most of these same errors, if you'd prefer to use that

shadow-delta-sample can be crashed with an existing shadow

add this line

                std::this_thread::sleep_for(std::chrono::seconds(10));

after this line in shadow-delta-sample: https://github.com/aws/aws-iot-device-sdk-cpp/blob/master/samples/ShadowDelta/ShadowDelta.cpp#L197

this causes the sample to download the existing shadow without trying to modify it first. set this shadow online (the "welcome" tag gets put in there by the gui shadow editor):

{
  "desired": {
    "welcome": "aws-iot",
    "nested": {
      "1": "1",
      "2": "2"
    }
  },
  "reported": {
    "welcome": "aws-iot"
  }
}

the sample will run and crash this way when it downloads the shadow a few seconds after adding the subscription (before the sleep() call):

shadow-delta-sample: /home/user/aws-iot-device-sdk-cpp/build/third_party/rapidjson/src/include/rapidjson/document.h:1111: rapidjson::GenericValue<Encoding, Allocator>& rapidjson::GenericValue<Encoding, Allocator>::operator[](const rapidjson::GenericValue<Encoding, SourceAllocator>&) [with SourceAllocator = rapidjson::MemoryPoolAllocator<>; Encoding = rapidjson::UTF8<>; Allocator = rapidjson::MemoryPoolAllocator<>]: Assertion `false' failed.
Aborted

calling AddShadowSubscription() after connect() disconnects client

with a client that is more or less copied from

https://github.com/aws/aws-iot-device-sdk-cpp/blob/master/samples/ShadowDelta/ShadowDelta.cpp

using openssl and the newest sdk

I might be making some simple error. Is there a reason this might be happening?

Subscribing to valid, but forbidden topics deadlocks lib

Trying to subscribe to e.g. "#" which at least according to awsiotsdk::mqtt::Subscription::IsValidTopicName() is a valid topic but according to the server policy my client is not authorized to subscribe to, results in no callbacks being called any more.
I could reproduce this with other topic ids too. I would have expected that the sdk would call my callback I provided on SubscribeAsync with an error code, or perhaps calls the disconnect callback or something, but it doesn't. In fact, subsequent [Publish|Subscribe]Async() calls callbacks are not called any more either until the sdk is shut down.

example log from my app with a allowed topic:

is connected true
topic to subscribe is "someallowed/topic/#" is valid topic: true
subscribe call queued with status: "Success : SDK Code 0." actionId: 2
publish call queued with status: "Success : SDK Code 0." actionId: 3
subscribeCallback for actionId 2 "Success : SDK Code 0."
publishCallback for actionId 3 "Success : SDK Code 0."
Received event from subscription: "someallowed/topic/MyNamespace.FakeMethod"

same app when trying to subscribe to something forbidden:

is connected true
topic to subscribe is "#" is valid topic: true
subscribe call queued with status: "Success : SDK Code 0." actionId: 2
publish call queued with status: "Success : SDK Code 0." actionId: 3

And that's it, I never get any callbacks called any more after this.

Undefine reference to anything in the ConfigCommon.hpp file

I'm trying to compile my version of the shadowDelta sample and after getting the cmake file set up properly. I run into a error during make: "undefined reference to awsiotsdk::ConfigCommon::" then anything that was called from that file. I know its not an issue with the SDK because nobody else I seen had any problems with it but I cannot get this to compile. Any help would be appreciated.

ShadowDelta sample code hangs in Connect call.

I have verified the certificates using the call:
openssl s_client -connect XXXXXXXXXX.us-XXXX-2.amazonaws.com:8883 -CAfile ./certs/rootCA.crt -cert ./certs/cert.pem -key ./certs/privkey.pem

However within the code I am seeing the error -407 , i.e SSL connection was closed. Any idea why this could be happening or what am I missing in my configuration below:

{
"endpoint": "xxxxxxxxxx.iot.us-xxxx-2.amazonaws.com",
"mqtt_port": 8883,
"https_port": 443,
"greengrass_discovery_port": 8443,
"root_ca_relative_path": "certs/rootCA.crt",
"device_certificate_relative_path": "certs/cert.pem",
"device_private_key_relative_path": "certs/privkey.pem",
"tls_handshake_timeout_msecs": 60000,
"tls_read_timeout_msecs": 2000,
"tls_write_timeout_msecs": 2000,
"aws_region": "",
"aws_access_key_id": "",
"aws_secret_access_key": "",
"aws_session_token": "",
"client_id": "test-device-01",
"thing_name": "test-device-01",
"is_clean_session": true,
"mqtt_command_timeout_msecs": 30000,
"keepalive_interval_secs": 600,
"minimum_reconnect_interval_secs": 1,
"maximum_reconnect_interval_secs": 128,
"maximum_acks_to_wait_for": 32,
"action_processing_rate_hz": 5,
"maximum_outgoing_action_queue_length": 32,
"discover_action_timeout_msecs": 300000
}

Subscribing to an already subscribed topic will result in crash

Hello,

In one of my tests, subscribing to an already subscribed topic results in crash inside the SDK. Here is the test sequence:

1. connect
2. subscribe to a topic, TopicA
3. disconnect
4. connect
5. subscribe again to TopicA ------> CRASH!!

I found the code causing the crash to be below in Subscribe.cpp:

ResponseCode SubscribeActionAsync::PerformAction(std::shared_ptr<NetworkConnection> p_network_connection, std::shared_ptr<ActionData> p_action_data) {

....

// Read running in separate thread, Insert before sending request to avoid situations where response arrives early
            util::Vector<std::shared_ptr<Subscription>>::iterator itr = p_subscribe_packet->subscription_list_.begin();
            while (itr != p_subscribe_packet->subscription_list_.end()) {
                util::String topic_name = (*itr)->GetTopicName()->ToStdString();
                auto existing_itr = p_client_state_->subscription_map_.find(topic_name);
                if (p_client_state_->subscription_map_.end() != existing_itr) {
                    if (existing_itr->second->IsActive()) {
                        itr = p_subscribe_packet->subscription_list_.erase(itr);
                    }
                } else {
                    p_client_state_->subscription_map_.insert(std::make_pair(topic_name, (*itr)));
                }

                itr++;
            }
...

I think when subscribing to an already subscribed topic, the code is erasing an element from p_subscribe_packet->subscription_list_ vector here while iterating the same and advancing the iterator afterwards here which causes the crash.

I would be happy to submit a fix.

Clarification regarding "Utf8cpp" third party license

Hello,

According to NOTICE file here, the SDK depends on some Utf8cpp library.

Could you please clarify the following things about this Utf8cpp library?

• Where is this library used inside the SDK.
• Utf8cpp library GitHub? link
• Utf8cpp library version used
• Utf8cpp library license link

Thanks

Feature request: PING api

Would be nice to have such an api to allow users to manually send a PING to AWS IoT broker.

segfault when reconnecting after disconnect

I get a segfault if I run my perpetual logging application and simulate a network disconnect followed by a reconnect. on reconnect, it sometimes segfaults with this valgrind output:

==66725== Process terminating with default action of signal 11 (SIGSEGV)
==66725== Access not within mapped region at address 0x0
==66725== at 0x758364: awsiotsdk::mqtt::KeepaliveActionRunner::PerformAction(std::shared_ptrawsiotsdk::NetworkConnection, std::shared_ptrawsiotsdk::ActionData) (Connect.cpp:369)

here's a link to the offending line:
https://github.com/aws/aws-iot-device-sdk-cpp/blob/master/src/mqtt/Connect.cpp#L369

I have no subscriptions so this function called on the previous line will return nullptr when given an empty subscription vector:
https://github.com/aws/aws-iot-device-sdk-cpp/blob/master/src/mqtt/Subscribe.cpp#L57

then nullptr is dereferenced in this bit:
p_subscribe_packet->ToString()

I think that's what's happening. I'm working on a fix and PR later today

Run "make" failed on Ubuntu 14.04

Hi dev team

I'm having problem when using aws-iot-device-sdk.
I do build process follow manual.
But when I run "make" command in "build" folder.
It always returns error belows:

[ 34%] Building CXX object CMakeFiles/aws-iot-sdk-cpp.dir/src/util/logging/Logging.cpp.o
[ 35%] Building CXX object CMakeFiles/aws-iot-sdk-cpp.dir/src/util/threading/ThreadTask.cpp.o
[ 37%] Linking CXX static library archive/libaws-iot-sdk-cpp.a
[ 37%] Built target aws-iot-sdk-cpp
Scanning dependencies of target aws-iot
[ 38%] Building CXX object cli/CMakeFiles/aws-iot.dir/cli.cpp.o
[ 40%] Building CXX object cli/CMakeFiles/aws-iot.dir/__/common/ConfigCommon.cpp.o
[ 41%] Building CXX object cli/CMakeFiles/aws-iot.dir/__/network/OpenSSL/OpenSSLConnection.cpp.o
[ 43%] Linking CXX executable ../bin/aws-iot
CMakeFiles/aws-iot.dir/__/network/OpenSSL/OpenSSLConnection.cpp.o: In function `awsiotsdk::network::OpenSSLConnection::ConnectInternal()':
OpenSSLConnection.cpp:(.text+0x1050): undefined reference to `SSL_get0_param'
OpenSSLConnection.cpp:(.text+0x1065): undefined reference to `X509_VERIFY_PARAM_set_hostflags'
OpenSSLConnection.cpp:(.text+0x10f0): undefined reference to `X509_VERIFY_PARAM_set1_ip_asc'
OpenSSLConnection.cpp:(.text+0x111e): undefined reference to `X509_VERIFY_PARAM_set1_host'
collect2: error: ld returned 1 exit status
make[2]: *** [bin/aws-iot] Error 1
make[1]: *** [cli/CMakeFiles/aws-iot.dir/all] Error 2
make: *** [all] Error 2

Note:

• I can build success with only "make aws-iot-sdk-cpp" (only .a file, no header files) but I need to run sample to get more understanding about aws-iot-sdk.
• I'm using openssl1.0.2g, cmake 3.8.2
• I created a normal c application and add "#include <openssl/ssl.h>", it is OK to build and run with these error method above.

shadow methods should be const-correct

see here for example:
https://github.com/aws/aws-iot-device-sdk-cpp/blob/master/src/shadow/Shadow.cpp#L64
mqtt_command_timeout, thing_name, client_token_prefix should be const. haven't looked at the shared_ptr, maybe that too. probably the whole codebase could use a once-over to fix this

What happens when network disconnects abruptly?

Hello,

Could you please explain how does the SDK behave on abrupt network disconnect?
When auto-reconnect is enabled, will the SDK be able to recover automatically as in be able to mqtt publish/subscribe/unsubscribe without connecting manually?

Will the disconnect callback be called in such abrupt network disconnect events?

Thanks

Support for "Authorizing Direct Calls to AWS Services"

https://docs.aws.amazon.com/iot/latest/developerguide/authorizing-direct-aws.html

Is support for this going to be added soon?

Disconnection-Callback while stopping broker gracefully

Hello,

I am just trying to get a disconnection callback when connected broker is stopped gracefully,
current sdk is working perfectly in the case of disconnection callback, when network connectivity goes,
but if the connected broker is stopped, in that case sdk immidiately terminates.

correct me if i am doing something wrong.

403 forbiden error on websocket connection

I'm testing the PubSub sample with the websocket configuration because opening the 8883 port isn't possible for my use-case.

Executing the sample, I'm facing a 403 Forbidden in the connection process

HTTP/1.1 403 Forbidden
content-type: application/json
content-length: 118
date: Wed, 13 Sep 2017 14:45:35 GMT
x-amzn-RequestId: XXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXX //hash replaced by XXX
connection: 

This error is leading to an out_of_range exception. Here is the call stack:

My config file looks like the following:

{
  "endpoint": "XXXXXXXXX.iot.XXXXXXX.amazonaws.com",
  "mqtt_port": 8883,
  "https_port": 443,
  "greengrass_discovery_port": 8443,
  "root_ca_relative_path": "certs/rootCA.crt",
  "device_certificate_relative_path": "certs/cert.pem",
  "device_private_key_relative_path": "certs/privkey.pem",
  "tls_handshake_timeout_msecs": 60000,
  "tls_read_timeout_msecs": 2000,
  "tls_write_timeout_msecs": 2000,
  "aws_region": "",
  "aws_access_key_id": "",
  "aws_secret_access_key": "",
  "aws_session_token": "",
  "client_id": "device_test",
  "thing_name": "device_test",
  "is_clean_session": true,
  "mqtt_command_timeout_msecs": 20000,
  "keepalive_interval_secs": 30,
  "minimum_reconnect_interval_secs": 1,
  "maximum_reconnect_interval_secs": 128,
  "maximum_acks_to_wait_for": 32,
  "action_processing_rate_hz": 5,
  "maximum_outgoing_action_queue_length": 32,
  "discover_action_timeout_msecs": 300000
}

And the certs folder is containing the mentioned files.

I didn't changed anything from the source code except solving two compilation issues where std::bind had to be replaced by std::bind<ResponseCode> here and here.

Do you have any idea on how to solve this 403 issue and protect the out_of_range?

Possible invalid read inside mqtt/Client

Mqtt client destruct sequence is as follows:

MqttClient::~MqttClient() {
        if (IsConnected()) {
            ResponseCode rc = Disconnect(p_client_state_->GetMqttCommandTimeout());
            if (ResponseCode::SUCCESS != rc) {
                AWS_LOG_ERROR(MQTT_CLIENT_LOG_TAG, "Disconnect returned error while destroying MQTT Client. %s",
                              ResponseHelper::ToString(rc).c_str());
            }
        }

        // p_client_state_.action_map_ and p_client_state_.outbound_action_queue_ retains p_client_state_
        // hence, calling p_client_state_->RegisterAction() or p_client_state_->EnqueueOutboundAction() introduces cyclic references inside p_client_state_
        // make sure that p_client_state_.action_map_ and p_client_state_.outbound_action_queue_ are cleared prior to p_client_state_ destructor
        // to break the cyclic references.
        p_client_state_->ClearRegisteredActions();
        p_client_state_->ClearOutboundActionQueue();
    }

To sum up, the destructor:

• disconnects client (if connected)
• clears registered actions and outbound action queue
• releases p_client_state_ and other retained instances

Note that p_client_state_ waits for all the running threads to finish.

Now consider the following test scenario:

- subscribe to a topic /foo
- wait for first message to arrive in topic /foo
- terminate the test once first message is received

Above test should reproduce the Invalid read when run under Valgrind.

Here's why:
NetworkRead class receives the message and calls the callback here. After calling the callback, it enqueues an outbound action for Puback if QoS is 1.

The test on the other hand does not wait for the Puback to finish (since it has no idea about it) and t terminates -> freeing the mqtt instance -> clearing registered actions and outbound queue -> thus releasing the PubackActionAsync created by NetworkRead for sending Puback to the server here.

Since Actions are only retained inside Action list or outbound queues, they should not be cleared until all running threads are gracefully shutdown.

I recommend changing the mqtt/Client's destruct sequence to as follows:

• disconnect the client (if connected)
• wait for all running threads to stop
  • Just need to clear the threads map as done here
• clear registered actions and outbound action queue
• release p_client_state_ and other retained instances

valgrind shows memory leaks inside SDK

Hello,

I ran the PubSub sample against valgrind and it looks like the SDK leaks memory (besides other issues like Mismatched free() etc). I am attaching the valgrind log for reference.

Please look into it.
log_memory_leak.txt

tsan warning: lock-order-inversion (potential deadlock)

found this when running llvm thread sanitizer on my program which has control flow similar to the pubsub sample:

https://github.com/aws/aws-iot-device-sdk-cpp/blob/master/samples/PubSub/PubSub.cpp

tsan warns about this mutex sequence:

first sync_action_response_lock_ here:
https://github.com/aws/aws-iot-device-sdk-cpp/blob/master/src/ClientCoreState.cpp#L98

ack_map_lock_ here:
https://github.com/aws/aws-iot-device-sdk-cpp/blob/master/src/ClientCoreState.cpp#L197

then sync_action_response_lock_ again here:
https://github.com/aws/aws-iot-device-sdk-cpp/blob/master/src/ClientCoreState.cpp#L112

before I investigate further, is this something you've seen in testing or maybe fixed in an upcoming release?

Unnecessary clutter in stdout

The following messages are unconditionally printed to stdout when a thread is killed:

aws-iot-device-sdk-cpp/src/util/threading/ThreadTask.cpp

Line 35 in c20c9e9

std::cout << "Exiting Thread " << thread_descriptor_ << "!!" << std::endl;

aws-iot-device-sdk-cpp/src/util/threading/ThreadTask.cpp

Line 43 in c20c9e9

std::cout << "Successfully Exited Thread " << thread_descriptor_ << "!!" << std::endl;

This seems a bit odd, and it probably makes more sense for these to be AWS_LOG_DEBUG or AWS_LOG_TRACE messages instead.

Memory leak from unfreed SSL_new

There is a case where SSL_free is not called on p_ssl_handle_. If the connection was never made then Disconnect() is never called (due to is_connected_ check) and p_ssl_handle_ is never freed. The fix that worked for me was to change destructor like so:

OpenSSLConnection::~OpenSSLConnection() {
            if (is_connected_) {
                Disconnect();
            }
            else {
                if (p_ssl_handle_) {
                    SSL_free(p_ssl_handle_);  // this is new
                }
            }
            SSL_CTX_free(p_ssl_context_);
#ifdef WIN32
            WSACleanup();
#endif
        }

There might also be another case if ConnectInternal() is called more then once before a Disconnect() is called. I can see this happening if there are reconnect attempts made externally. This can probably be fixed by just allocating p_ssl_handle_ once in ConnectInternal():

if (!p_ssl_handle_)
{
    p_ssl_handle_ = SSL_new(p_ssl_context_);
}

ERR_remove_state deprecated in 1.0.2

ERR_remove_state is deprecated in 1.0.2. Can we safely switch to ERR_remove_thread_state?

Resend Client Token

When I initialize the shadow of the device, the SDK creates a default client token, however, as this client token does not change with each request, it is not sent when I perform a "PerformUpdateAsync", how can I force the client token to be sent?

Thanks!

Subscribing a topic with QoS1 results in duplicate messages

Hello,

I am not sure if this is by design or a bug, but subscribing a topic with QoS1 always results in duplicate messages. However, these messages don't have the DUP flag set to 1. So it is as if the AWS MQTT broker is re-sending the same message over and over again or there is some issue inside the SDK.

I tried to debug the issue and found few interesting observations:

• The duplicate messages are received roughly every 30 seconds or so and are received 4-6 times before dying out.
  • However, 30 second thing is not always true.
• Using QoS 0 when subscribing, stops these duplicate messages for some reason.
• Other MQTT client like mosquitto does not seem to have this issue when tested with the same AWS MQTT broker. (No duplicate messages with either QoS 0 or QoS 1)

For now, as a workaround I have refrained from using QoS 1 when subscribing to AWS MQTT topics but this behavior is something not desirable and might have unknown consequences when used in production.

Could you please look into this issue?

Bug with UpdateDeviceShadow

Greetings :)!

I am building a project using this SDK and I have problem ... When I try to use the "UpdateDeviceShadow" method of the "Shadow" class the program responds with a "Segmentation fault" if you are not yet connected and subscribed to the device topics in the service.

I mean, if I first connect to the service and subscribe to the topics, I can use "UpdateDeviceShadow", however, if I do not this in advance the library dies.

This is a problem because sometimes you would like to work offline, that is, if my device starts and tries to connect to the internet and can not, it could load the last known state of the device (which can be stored in a file) and work with it, and once it manages to connect to the internet then it updates its status with the most recent.

That is exactly what is happening to me, sometimes I do not have internet when I start the device, so the device load the last known state using a file and call "UpdateDeviceShadow" to work in a kind of "disconnected mode", but currently this is not possible.

Thanks a lot!

3 sample projects does not show up in VS solution automatically

discovery-sample, robot-arm-sample, switch-sample does not show up in VS solution automatically.
I check the build/samples/Discovery, etc. The projects are already compile by Cmake.
It can be add to the solution manually.
So, I think this maybe a bug.

Where can I get the Greengrass core SDK?

Does the core SDK open to us ?

Just-in-Time registration of device certificates always fails

Hello,

My iot devices do Just-in-Time registration of device certificates when the first pub/sub mqtt command is issued. Having tried many times, I am unable to get it working with aws-iot-device-sdk-cpp. The SDK will always return NETWORK_SSL_CONNECT_ERROR.

Note that if I do Just-in-Time registration from some other MQTT client like, Mosquitto before using aws-iot-device-sdk-cpp, the pub/sub request succeeds.

Have you guys tested Just-in-Time registration with this SDK?
(I am using the default OpenSSLConnection provided with the SDK for my tests.)

JsonParser unit test fails

Error description

[----------] 3 tests from JsonParserTester
[ RUN      ] JsonParserTester.RunTests
/src/tests/unit/src/util/JsonParserTests.cpp:140: Failure
Expected equality of these values:
  ResponseCode::SUCCESS
    Which is: Success : SDK Code 0.
  rc
    Which is: Error occurred while trying to open the file : SDK Code -100.
aws-iot-unit-tests: /build/third_party/rapidjson/src/include/rapidjson/document.h:1233: rapidjson::GenericValue::MemberIterator rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >::FindMember(const GenericValue<Encoding, SourceAllocator> &) [Encoding = rapidjson::UTF8<char>, Allocator = rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>, SourceAllocator = rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>]: Assertion `IsObject()' failed.
Aborted

Can't open the file (SDK Code -100).

Debug information

• Failed in both Linux (Debian Stretch running in Docker) and mac OS 10.13.3.

Disconnect callback context data is always a nullptr

Using this code:
MqttClient::Create(m_networkConnection, std::chrono::milliseconds(30000), &onDisconnectedCallback, std::shared_ptr<DisconnectCallbackContextData>(this));

whenever the onDisconnectedCallback is called, the p_app_handler_data is null.

Missing disconnect callback?

Hello,

I am trying to understand how the SDK handles connection state. When auto-reconnect if set to false, SDK will not re-connect automatically after the keep-alive times out as set during connect and the mqtt client will become disconnected.

In the current SDK design, applications will not be able to receive information about this auto-disconnect since the SDK does not provide with a disconnect callback. aws-iot-device-sdk-embedded-C SDK provides us with such a callback here.

Having a disconnect callback might be useful in cases when applications want to control the re-connection to AWS IoT or for some other book-keeping or resource cleanups etc.

I also saw that ClientState caches the connection state and the mqtt clients return it when IsConnected api is called. Having a disconnect callback can help in making this connection state consistent as well.

Am I missing something here?

When is the disconnect callback called?

Hello,

Thank you for adding the disconnect callback in v1.1.0. Could you please explain when will applications receive this callback?

Build Error on RPI

Hi,

I have tried to build using cmake, I see following error

-- Configuring done
-- Generating done
-- Build files have been written to: /home/pi/surya/aws-iot-device-sdk-cpp/build/third_party/rapidjson/download
[100%] Built target rapidjson
CMake Error at /usr/local/share/cmake-3.2/Modules/FindPackageHandleStandardArgs.cmake:138 (message):
Could NOT find Threads (missing: Threads_FOUND)
Call Stack (most recent call first):
/usr/local/share/cmake-3.2/Modules/FindPackageHandleStandardArgs.cmake:374 (_FPHSA_FAILURE_MESSAGE)
/usr/local/share/cmake-3.2/Modules/FindThreads.cmake:204 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
CMakeLists.txt:69 (find_package)

-- Configuring incomplete, errors occurred!
See also "/home/pi/surya/aws-iot-device-sdk-cpp/build/CMakeFiles/CMakeOutput.log".
See also "/home/pi/surya/aws-iot-device-sdk-cpp/build/CMakeFiles/CMakeError.log".

Please let me know how to fix it, Thanks in advance.

AutoReconnect feature doesn't seem to work

having set client->SetAutoReconnectEnabled(true), I cannot see the library ever reconnecting to aws, if for instance the connection drops because ethernet goes down and back up again.

Unsubscribing a topic does not "deactivate" the corresponding subscription

Hello,

Just wanted to confirm the spec here. Consider the following situation:

- Create a subscription, say SubA with topic, say TopA
- Use this SubA to call subscribe
- My mqtt client is now subscribed to TopA
- SubA->isActive is now true
- Retain SubA
- Unsubscribe to TopA
- SubA->isActive is still true

After the above sequence, my expectation was that SubA->isActive should return false so as to make the specification more consistent.

What do you think?

src/mqtt/connect.cpp: comparison is always false due to limited range of data type

these comparisons are flagged by clang -wtype-limits. this causes the code after the comparison to never be executed:

https://github.com/aws/aws-iot-device-sdk-cpp/blob/master/src/mqtt/Connect.cpp#L137
https://github.com/aws/aws-iot-device-sdk-cpp/blob/master/src/mqtt/Connect.cpp#L163

this one is similar but just some extra code:

https://github.com/aws/aws-iot-device-sdk-cpp/blob/master/src/mqtt/Connect.cpp#L197

random crashes on disconnects

Every once in while our application crashes when the MBedTLS connection goes down. I managed to catch it in the debugger now, here's the stack trace:

The crash happens in Thread 13

`
Thread 13 (Thread 0x7fffd22e6700 (LWP 14027)):
#0 0x00007ffff65dbcc0 in __libc_write (fd=32, buf=0x7fffbc00c3c8, nbytes=31) at ../sysdeps/unix/sysv/linux/write.c:26
resultvar = 18446744073709551584
sc_cancel_oldtype = 0
sc_ret =
nbytes = 31
buf = 0x7fffbc00c3c8
fd = 32
#1 0x00007ffff72c4aff in mbedtls_net_send () from /usr/lib/x86_64-linux-gnu/libmbedtls.so.10
No symbol table info available.
#2 0x00007ffff72d165c in mbedtls_ssl_flush_output () from /usr/lib/x86_64-linux-gnu/libmbedtls.so.10
No symbol table info available.
#3 0x00007ffff72d400f in mbedtls_ssl_write_record () from /usr/lib/x86_64-linux-gnu/libmbedtls.so.10
No symbol table info available.
#4 0x00007ffff72d5d0f in mbedtls_ssl_send_alert_message () from /usr/lib/x86_64-linux-gnu/libmbedtls.so.10
No symbol table info available.
#5 0x00007ffff72d5e17 in mbedtls_ssl_close_notify () from /usr/lib/x86_64-linux-gnu/libmbedtls.so.10
No symbol table info available.
#6 0x00007ffff77819d7 in awsiotsdk::network::MbedTLSConnection::DisconnectInternal (this=0x555555b66f50) at ../../guh/libguh-core/MbedTLS/MbedTLSConnection.cpp:320
ret = 0
#7 0x0000555555589373 in awsiotsdk::NetworkConnection::Disconnect() ()
No symbol table info available.
#8 0x00005555555a9c0c in awsiotsdk::mqtt::DisconnectActionAsync::PerformAction(std::shared_ptrawsiotsdk::NetworkConnection, std::shared_ptrawsiotsdk::ActionData) ()
No symbol table info available.
#9 0x00005555555b904e in awsiotsdk::ClientCoreState::PerformAction(awsiotsdk::ActionType, std::shared_ptrawsiotsdk::ActionData, std::chrono::duration<long, std::ratio<1l, 1000l> >) ()
No symbol table info available.
#10 0x00005555555a6d24 in awsiotsdk::mqtt::KeepaliveActionRunner::PerformAction(std::shared_ptrawsiotsdk::NetworkConnection, std::shared_ptrawsiotsdk::ActionData) ()
No symbol table info available.
#11 0x00005555555b8a32 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<std::_Bind<awsiotsdk::ResponseCode (awsiotsdk::Action::*(std::unique_ptr<awsiotsdk::Action, std::default_deleteawsiotsdk::Action >, std::shared_ptrawsiotsdk::NetworkConnection, std::shared_ptrawsiotsdk::ActionData))(std::shared_ptrawsiotsdk::NetworkConnection, std::shared_ptrawsiotsdk::ActionData)> > > >::_M_run() ()
No symbol table info available.
#12 0x00007ffff63000ff in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
No symbol table info available.
#13 0x00007ffff65d17fc in start_thread (arg=0x7fffd22e6700) at pthread_create.c:465
pd = 0x7fffd22e6700
now =
unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140736719644416, 1081798041271840026, 140737106032670, 140737106032671, 140736719644416, 23, -1081719558347641574, -1081781811933565670}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
not_first_call =
pagesize_m1 =
sp =
freesize =
PRETTY_FUNCTION = "start_thread"
#14 0x00007ffff5d61b5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 12 (Thread 0x7fffe8b63700 (LWP 14026)):
#0 0x00007ffff65dc5f8 in __GI___nanosleep (requested_time=0x7fffe8b62b30, remaining=0x7fffe8b62b30) at ../sysdeps/unix/sysv/linux/nanosleep.c:27
resultvar = 18446744073709551100
sc_cancel_oldtype = 0
#1 0x00005555555b5cf5 in awsiotsdk::Action::ReadFromNetworkBuffer(std::shared_ptrawsiotsdk::NetworkConnection, std::vector<unsigned char, std::allocator >&, unsigned long) ()
No symbol table info available.
#2 0x00005555555ab639 in awsiotsdk::mqtt::NetworkReadActionRunner::ReadPacketFromNetwork(unsigned char&, std::vector<unsigned char, std::allocator >&) ()
No symbol table info available.
#3 0x00005555555ac19c in awsiotsdk::mqtt::NetworkReadActionRunner::PerformAction(std::shared_ptrawsiotsdk::NetworkConnection, std::shared_ptrawsiotsdk::ActionData) ()
No symbol table info available.
#4 0x00005555555b8a32 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<std::_Bind<awsiotsdk::ResponseCode (awsiotsdk::Action::*(std::unique_ptr<awsiotsdk::Action, std::default_deleteawsiotsdk::Action >, std::shared_ptrawsiotsdk::NetworkConnection, std::shared_ptrawsiotsdk::ActionData))(std::shared_ptrawsiotsdk::NetworkConnection, std::shared_ptrawsiotsdk::ActionData)> > > >::_M_run() ()
No symbol table info available.
#5 0x00007ffff63000ff in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
No symbol table info available.
#6 0x00007ffff65d17fc in start_thread (arg=0x7fffe8b63700) at pthread_create.c:465
pd = 0x7fffe8b63700
now =
unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737097643776, 1081798041271840026, 140737106032670, 140737106032671, 140737097643776, 21, -1081847998807757542, -1081781811933565670}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
not_first_call =
pagesize_m1 =
sp =
freesize =
PRETTY_FUNCTION = "start_thread"
#7 0x00007ffff5d61b5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 10 (Thread 0x7fffea366700 (LWP 14024)):
#0 0x00007ffff65dc5f8 in __GI___nanosleep (requested_time=0x7fffea365c60, remaining=0x7fffea365c60) at ../sysdeps/unix/sysv/linux/nanosleep.c:27
resultvar = 18446744073709551100
sc_cancel_oldtype = 0
#1 0x00005555555baa9d in awsiotsdk::ClientCoreState::ProcessOutboundActionQueue(std::shared_ptr<std::atomic >) ()
No symbol table info available.
#2 0x00005555555b88fa in std::thread::_State_impl<std::thread::_Invoker<std::tuple<std::_Bind<void (awsiotsdk::ClientCoreState::*(std::shared_ptrawsiotsdk::ClientCoreState, std::shared_ptr<std::atomic >))(std::shared_ptr<std::atomic >)> > > >::_M_run() ()
No symbol table info available.
#3 0x00007ffff63000ff in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
No symbol table info available.
#4 0x00007ffff65d17fc in start_thread (arg=0x7fffea366700) at pthread_create.c:465
pd = 0x7fffea366700
now =
unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737122821888, 1081798041271840026, 140737488339614, 140737488339615, 140737122821888, 21, -1081842497491522278, -1081781811933565670}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
not_first_call =
pagesize_m1 =
sp =
freesize =
PRETTY_FUNCTION = "start_thread"
#5 0x00007ffff5d61b5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 8 (Thread 0x7fffd2ae7700 (LWP 13637)):
#0 0x00007ffff5d55951 in __GI___poll (fds=0x7fffc40182f0, nfds=1, timeout=135119) at ../sysdeps/unix/sysv/linux/poll.c:29
resultvar = 18446744073709551100
sc_cancel_oldtype = 0
#1 0x00007ffff29d6169 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
No symbol table info available.
#2 0x00007ffff29d627c in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
No symbol table info available.
#3 0x00007ffff6acb47f in QEventDispatcherGlib::processEvents(QFlagsQEventLoop::ProcessEventsFlag) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
No symbol table info available.
#4 0x00007ffff6a70e3a in QEventLoop::exec(QFlagsQEventLoop::ProcessEventsFlag) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
No symbol table info available.
#5 0x00007ffff68903ca in QThread::exec() () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
No symbol table info available.
#6 0x00007ffff689529d in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
No symbol table info available.
#7 0x00007ffff65d17fc in start_thread (arg=0x7fffd2ae7700) at pthread_create.c:465
pd = 0x7fffd2ae7700
now =
unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140736728037120, 1081798041271840026, 140737488343278, 140737488343279, 140736728037120, 140736728037824, -1081720659469882086, -1081781811933565670}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
not_first_call =
pagesize_m1 =
sp =
freesize =
PRETTY_FUNCTION = "start_thread"
#8 0x00007ffff5d61b5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 3 (Thread 0x7fffeb46d700 (LWP 13632)):
#0 0x00007ffff5d55951 in __GI___poll (fds=0x7fffdc002de0, nfds=1, timeout=9997) at ../sysdeps/unix/sysv/linux/poll.c:29
resultvar = 18446744073709551100
sc_cancel_oldtype = 0
#1 0x00007ffff29d6169 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
No symbol table info available.
#2 0x00007ffff29d627c in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
No symbol table info available.
#3 0x00007ffff6acb47f in QEventDispatcherGlib::processEvents(QFlagsQEventLoop::ProcessEventsFlag) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
No symbol table info available.
#4 0x00007ffff6a70e3a in QEventLoop::exec(QFlagsQEventLoop::ProcessEventsFlag) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
No symbol table info available.
#5 0x00007ffff68903ca in QThread::exec() () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
No symbol table info available.
#6 0x00007ffff689529d in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
No symbol table info available.
#7 0x00007ffff65d17fc in start_thread (arg=0x7fffeb46d700) at pthread_create.c:465
pd = 0x7fffeb46d700
now =
unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737140676352, 1081798041271840026, 140737488344318, 140737488344319, 140737140676352, 140737140677056, -1081841256782844646, -1081781811933565670}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
not_first_call =
pagesize_m1 =
sp =
freesize =
PRETTY_FUNCTION = "start_thread"
#8 0x00007ffff5d61b5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 2 (Thread 0x7fffec5e3700 (LWP 13631)):
#0 0x00007ffff5d55951 in __GI___poll (fds=0x7fffe40053d0, nfds=2, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
resultvar = 18446744073709551100
sc_cancel_oldtype = 0
#1 0x00007ffff29d6169 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
No symbol table info available.
#2 0x00007ffff29d627c in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
No symbol table info available.
#3 0x00007ffff6acb47f in QEventDispatcherGlib::processEvents(QFlagsQEventLoop::ProcessEventsFlag) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
No symbol table info available.
#4 0x00007ffff6a70e3a in QEventLoop::exec(QFlagsQEventLoop::ProcessEventsFlag) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
No symbol table info available.
#5 0x00007ffff68903ca in QThread::exec() () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
No symbol table info available.
#6 0x00007ffff41b1e45 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5DBus.so.5
No symbol table info available.
#7 0x00007ffff689529d in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
No symbol table info available.
#8 0x00007ffff65d17fc in start_thread (arg=0x7fffec5e3700) at pthread_create.c:465
pd = 0x7fffec5e3700
now =
unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737158985472, 1081798041271840026, 140737488347166, 140737488347167, 140737158985472, 140737158986176, -1081838996556305126, -1081781811933565670}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
not_first_call =
pagesize_m1 =
sp =
freesize =
PRETTY_FUNCTION = "start_thread"
#9 0x00007ffff5d61b5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 1 (Thread 0x7ffff7fa9640 (LWP 13627)):
#0 0x00007ffff5d55951 in __GI___poll (fds=0x555555cc29b0, nfds=22, timeout=870) at ../sysdeps/unix/sysv/linux/poll.c:29
resultvar = 18446744073709551100
sc_cancel_oldtype = 0
#1 0x00007ffff29d6169 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
No symbol table info available.
#2 0x00007ffff29d627c in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
No symbol table info available.
#3 0x00007ffff6acb47f in QEventDispatcherGlib::processEvents(QFlagsQEventLoop::ProcessEventsFlag) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
No symbol table info available.
#4 0x00007ffff6a70e3a in QEventLoop::exec(QFlagsQEventLoop::ProcessEventsFlag) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
No symbol table info available.
#5 0x00007ffff6a79da4 in QCoreApplication::exec() () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
No symbol table info available.
#6 0x00005555555833a1 in main (argc=12, argv=0x7fffffffe2c8) at ../../guh/server/main.cpp:281
userId = 1000
ret = -1
application = { = {}, }
loggingFiltersPlugins = {{d = 0x55555584efc0, e = 0x55555584efc0}}
translator =
PRETTY_FUNCTION = "int main(int, char**)"
parser = {d = 0x55555584c490}
applicationDescription = {static null = {}, d = 0x555555850700}
foregroundOption = {d = {d = 0x55555584a870}}
debugDescription = {static null = {}, d = 0x555555871940}
sortedFilterList = {<QList> = {<QListSpecialMethods> = {}, {p = {static shared_null = {ref = {atomic = {_q_value = {<std::__atomic_base> = {static _S_alignment = 4, _M_i = -1}, }}}, alloc = 0, begin = 0, end = 0, array = {0x0}}, d = 0x55555584dc10}, d = 0x55555584dc10}}, }
sortedPluginList = {<QList> = {<QListSpecialMethods> = {}, {p = {static shared_null = {ref = {atomic = {_q_value = {<std::__atomic_base> = {static _S_alignment = 4, _M_i = -1}, }}}, alloc = 0, begin = 0, end = 0, array = {0x0}}, d = 0x555555850b10}, d = 0x555555850b10}}, }
allOption = {d = {d = 0x55555584d850}}
logOption = {d = {d = 0x55555584f3d0}}
dbusOption = {d = {d = 0x55555584b630}}
debugOption = {d = {d = 0x55555584f0f0}}
startForeground = true
service = {<QtService> = { = {_vptr.QtServiceBase = 0x10500000002, d_ptr = 0x5555555bdce9}, app = 0x5555555be4c0 main::__PRETTY_FUNCTION__}, }
ret = 0
`

The contents of "this" and "ssl_":
Locals ret 0 int this @0x555555b66f50 awsiotsdk::network::MbedTLSConnection [awsiotsdk::NetworkConnection] @0x555555b66f50 awsiotsdk::NetworkConnection cacert_ @0x555555b70950 mbedtls_x509_crt clicert_ @0x555555b70b78 mbedtls_x509_crt conf_ @0x555555b707d0 mbedtls_ssl_config ctr_drbg_ @0x555555b70490 mbedtls_ctr_drbg_context device_cert_location_ "/home/micha/Develop/cloudstuff/testserver/certificate.pem" awsiotsdk::util::String device_private_key_location_ "/home/micha/Develop/cloudstuff/testserver/private.key" awsiotsdk::util::String endpoint_ "a2addxakg5juii.iot.eu-west-1.amazonaws.com" awsiotsdk::util::String endpoint_port_ 8883 uint16_t entropy_ @0x555555b67050 mbedtls_entropy_context flags_ 0 uint32_t is_connected_ @0x555555b67009 std::atomic_bool pkey_ @0x555555b70da0 mbedtls_pk_context requires_free_ @0x555555b70db4 std::atomic_bool root_ca_location_ "/home/micha/Develop/cloudstuff/symantec.pem" awsiotsdk::util::String server_fd_ @0x555555b70db0 mbedtls_net_context server_verification_flag_ @0x555555b67008 std::atomic_bool ssl_ @0x555555b70618 mbedtls_ssl_context alpn_chosen 0x0 char* badmac_seen 0 unsigned int cli_id 0x0 unsigned char* cli_id_len 0 size_t client_auth 1 int conf @0x555555b707d0 mbedtls_ssl_config f_get_timer 0x0 mbedtls_ssl_get_timer_t* f_recv 0x0 mbedtls_ssl_recv_t* f_recv_timeout <not accessible> mbedtls_ssl_recv_timeout_t* f_send <not accessible> mbedtls_ssl_send_t* f_set_timer 0x0 mbedtls_ssl_set_timer_t* handshake 0x0 mbedtls_ssl_handshake_params* hostname "a2addxakg5juii.iot.eu-west-1.amazonaws.com" char* in_buf "" unsigned char* in_ctr "" unsigned char* in_epoch 0 uint16_t in_hdr "\027\003\003" unsigned char* in_hslen 0 size_t in_iv "" unsigned char* in_left 0 size_t in_len "" unsigned char* in_msg "�" unsigned char* in_msglen 0 size_t in_msgtype 23 int in_offt 0x0 unsigned char* in_window 0 uint64_t in_window_top 0 uint64_t keep_current_message 0 int major_ver 3 int minor_ver 3 int nb_zero 0 int next_record_offset 0 size_t out_buf "" unsigned char* out_ctr "" unsigned char* out_hdr "\025\003\003" unsigned char* out_iv "" unsigned char* out_left 31 size_t out_len "" unsigned char* out_msg "����H���XAe\r垴1W\022���C,���Q��vQt{o�Z�5-\t�$FVq����Z��\002[��\013\t�\031\005@u\023�����y�\001\006n��\025U��~Ν�{�m\t��$�Z\014\2776L͋�"... (unknown length) unsigned char* out_msglen 26 size_t out_msgtype 21 int own_verify_data "46�?�3q���\tQ" char[12] p_bio 0x555555b70db0 void* p_timer 0x0 void* peer_verify_data "�|j����ΞdyN" char[12] renego_records_seen 0 int renego_status 0 int secure_renegotiation 1 int session @0x7fffbc010520 mbedtls_ssl_session session_in @0x7fffbc010520 mbedtls_ssl_session session_negotiate 0x0 mbedtls_ssl_session* session_out @0x7fffbc010520 mbedtls_ssl_session split_done '\0' 0 0x00 signed char state 16 int transform @0x7fffbc007480 mbedtls_ssl_transform transform_in @0x7fffbc007480 mbedtls_ssl_transform transform_negotiate 0x0 mbedtls_ssl_transform* transform_out @0x7fffbc007480 mbedtls_ssl_transform verify_data_len 12 size_t tls_handshake_timeout_ @0x555555b67010 std::chrono::milliseconds tls_read_timeout_ @0x555555b67018 std::chrono::milliseconds tls_write_timeout_ @0x555555b67020 std::chrono::milliseconds Inspector Expressions Return Value Tooltip

openssl wrapper gets stuck if client is started before network is ready

in my client I set up a connection using code copied from the pubsub sample, which uses the openssl wrapper:

https://github.com/aws/aws-iot-device-sdk-cpp/tree/master/samples/PubSub

if I start my program before the system's network is available then it gets stuck forever on the iot_client->connect() call (see this line for example):

https://github.com/aws/aws-iot-device-sdk-cpp/blob/master/samples/PubSub/PubSub.cpp#L181

the error returned is -300, NETWORK_TCP_CONNECT_ERROR, with a log line "TCP Connection error" which means it came from this bit of code:

https://github.com/aws/aws-iot-device-sdk-cpp/blob/master/network/OpenSSL/OpenSSLConnection.cpp#L290

and this specific system call:

https://github.com/aws/aws-iot-device-sdk-cpp/blob/master/network/OpenSSL/OpenSSLConnection.cpp#L158

I'm going to instrument the system call to get its error code. What happens is the connection fails, my program waits N seconds and retries (with a fresh network::OpenSSLConnection object), and continues to get the same error even though my computer can ping and the network is now up. If I restart my client after the network is up it works fine.

Install doesn't install OpenSSLConnection under aws shared library

Hi,

A number of your samples ( and eventually user code ) rely on OpenSSLConnection.{cpp,hpp} . However, these two files don't make it into the final shared library or into the install include directory. This makes it difficult to build samples that look like Amazon's because the user then has to include the OpenSSLConnection files in their own project.

Just a thought that this could be installed and included in the shared library that is constructed.