octogonapus/awscrt.jl

A Julia client for some AWS services using the AWS Common Runtime (CRT).

awscrt.jl's Issues

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Segfault when running on_connection_complete from a sysimage

When attempting to connect to an endpoint for which the client is not authorized, libawscrt internally invokes on_connection_complete which can go one of two ways.

1. It ultimately prints an error to the user (this is the expected behavior):

      nested task error: Connection failed. AWS Error 5134: libaws-c-mqtt: AWS_ERROR_MQTT_UNEXPECTED_HANGUP, The connection was closed unexpectedly.
      Stacktrace:
       [1] macro expansion
         @ ~/.julia/packages/AWSCRT/XV7k2/src/AWSMQTT.jl:440 [inlined]
       [2] (::AWSCRT.var"#42#46"{AWSCRT.MQTTConnection, Base.RefValue{LibAWSCRT.aws_mqtt_connection_options}, Base.RefValue{ForeignCallbacks.ForeignToken}, ForeignCallbacks.ForeignCallback{AWSCRT.OnConnectionCompleteMsg}, Channel{Any}})()
         @ AWSCRT ./task.jl:429

2. It segfaults when running in a sysimage with a particular method precompiled. This is the precompile statement causing the problem.

I have created an MWE here (though it requires some real cloud resources): https://github.com/Octogonapus/awscrt_segfault_mwe

Implement websockets

Resubscribe segfaulting

on_resubscribe_complete causes a segfault when running ~/Documents/code/julia/usr/bin/julia-debug -g2 -O0 -t1 --project runtests.jl on 1.7.2

This looks like the same problem as we had to debug before. rr:

(rr) c
Continuing.
[New Thread 73262.73322]
[New Thread 73262.73320]
[New Thread 73262.73263]
[New Thread 73262.73318]

Thread 3 received signal SIGSEGV, Segmentation fault.
[Switching to Thread 73262.73320]
0x00007f06739addff in julia_on_resubscribe_complete_461 (connection=139665651500608, packet_id=0, topic_subacks=140733757279471, error_code=32767, userdata=139665651495056) at /home/salmon/Documents/code/AWSMQTT/src/AWSMQTT.jl:809
809     function on_resubscribe_complete(
(rr) bt
#0  0x00007f06739addff in julia_on_resubscribe_complete_461 (connection=139665651500608, packet_id=0, topic_subacks=140733757279471, error_code=32767, userdata=139665651495056)
    at /home/salmon/Documents/code/AWSMQTT/src/AWSMQTT.jl:809
#1  0x00007f06739b03f6 in jlcapi_on_resubscribe_complete_466 ()
#2  0x00007f067270aa0b in s_resubscribe_complete (connection=0x1c4db70, packet_id=2, error_code=0, userdata=0x1eb0b00) at /workspace/srcdir/aws-c-mqtt/source/client.c:2385
#3  0x00007f067270f958 in mqtt_request_complete (connection=0x1c4db70, error_code=0, packet_id=2) at /workspace/srcdir/aws-c-mqtt/source/client_channel_handler.c:963
#4  0x00007f067270e0ee in s_packet_handler_suback (connection=0x1c4db70, message_cursor=...) at /workspace/srcdir/aws-c-mqtt/source/client_channel_handler.c:313
#5  0x00007f067270e4fe in s_process_mqtt_packet (connection=0x1c4db70, packet_type=AWS_MQTT_PACKET_SUBACK, packet=...) at /workspace/srcdir/aws-c-mqtt/source/client_channel_handler.c:459
#6  0x00007f067270ea25 in s_process_read_message (handler=0x1c4db98, slot=0x7f06640309d0, message=0x7f066400fec0) at /workspace/srcdir/aws-c-mqtt/source/client_channel_handler.c:573
#7  0x00007f06726d88f8 in aws_channel_handler_process_read_message (handler=0x1c4db98, slot=0x7f06640309d0, message=0x7f066400fec0) at /workspace/srcdir/aws-c-io/source/channel.c:1000
#8  0x00007f06726d7c43 in aws_channel_slot_send_message (slot=0x7f066400bd60, message=0x7f066400fec0, dir=AWS_CHANNEL_DIR_READ) at /workspace/srcdir/aws-c-io/source/channel.c:760
#9  0x00007f06726f8912 in s_s2n_handler_process_read_message (handler=0x7f0664014d70, slot=0x7f066400bd60, message=0x7f066400be00) at /workspace/srcdir/aws-c-io/source/s2n/s2n_tls_channel_handler.c:538
#10 0x00007f06726d88f8 in aws_channel_handler_process_read_message (handler=0x7f0664014d70, slot=0x7f066400bd60, message=0x7f066400be00) at /workspace/srcdir/aws-c-io/source/channel.c:1000
#11 0x00007f06726d7c43 in aws_channel_slot_send_message (slot=0x7f066400bdb0, message=0x7f066400be00, dir=AWS_CHANNEL_DIR_READ) at /workspace/srcdir/aws-c-io/source/channel.c:760
#12 0x00007f06726fc4a7 in s_do_read (socket_handler=0x7f0664014770) at /workspace/srcdir/aws-c-io/source/socket_channel_handler.c:164
#13 0x00007f06726fc7a6 in s_on_readable_notification (socket=0x7f0664001be0, error_code=0, user_data=0x7f0664014770) at /workspace/srcdir/aws-c-io/source/socket_channel_handler.c:221
#14 0x00007f06726f54e6 in s_on_socket_io_event (event_loop=0x1ba6d40, handle=0x7f0664001cd8, events=3, user_data=0x7f0664001be0) at /workspace/srcdir/aws-c-io/source/posix/socket.c:1618
#15 0x00007f06726e82a9 in s_main_loop (args=0x1ba6d40) at /workspace/srcdir/aws-c-io/source/linux/epoll_event_loop.c:624
#16 0x00007f067266e31a in thread_fn (arg=0x11086a0) at /workspace/srcdir/aws-c-common/source/posix/thread.c:137
#17 0x00007f069f03f2a5 in start_thread (arg=0x7f06718b8640) at pthread_create.c:481
#18 0x00007f069ef67323 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(rr) disas
Dump of assembler code for function julia_on_resubscribe_complete_461:
   0x00007f06739add80 <+0>:     sub    $0x228,%rsp
   0x00007f06739add87 <+7>:     mov    %esi,0x8c(%rsp)
   0x00007f06739add8e <+14>:    mov    %rdx,0x90(%rsp)
   0x00007f06739add96 <+22>:    mov    %ecx,0x9c(%rsp)
   0x00007f06739add9d <+29>:    movabs $0x7f069e8e2060,%rax
   0x00007f06739adda7 <+39>:    mov    (%rax),%rax
   0x00007f06739addaa <+42>:    mov    %rax,0x220(%rsp)
   0x00007f06739addb2 <+50>:    lea    0x110(%rsp),%rax
   0x00007f06739addba <+58>:    vxorps %xmm0,%xmm0,%xmm0
   0x00007f06739addbe <+62>:    vmovdqa %xmm0,(%rax)
   0x00007f06739addc2 <+66>:    movq   $0x0,0x10(%rax)
   0x00007f06739addca <+74>:    mov    %fs:0x0,%rax
   0x00007f06739addd3 <+83>:    mov    -0x8(%rax),%rax
   0x00007f06739addd7 <+87>:    mov    %rax,0xa0(%rsp)
   0x00007f06739adddf <+95>:    movq   $0x4,0x110(%rsp)
   0x00007f06739addeb <+107>:   lea    0x110(%rsp),%r9
   0x00007f06739addf3 <+115>:   add    $0x8,%r9
   0x00007f06739addf7 <+119>:   mov    %r9,0xa8(%rsp)
=> 0x00007f06739addff <+127>:   mov    (%rax),%r9

The problem is, I've gone through the debugging steps I've learned so far and that hasn't changed the fact that mov %fs:0x0,%rax is getting generated. I'm not sure how else to debug this. @IanButterworth or @vchuravy do you have any ideas? Code is here https://github.com/Octogonapus/AWSCRT.jl/blob/main/src/AWSMQTT.jl#L809

Implement library cleanup

Most of the CRT libraries have init and cleanup routines. I don't think it is strictly necessary to invoke the cleanup routines (the kernel will clean up after us), but it would be best practice to do so. This can be tricky to implement as the cleanup routines must be called after all AWS resources are freed (so after all finalizers). We can try to implement something based on https://github.com/JuliaLang/julia/pull/20124/files

Implement multi-topic subscriptions

aws-c-mqtt has aws_mqtt_client_connection_subscribe_multiple to subscribe to multiple topics with one subscribe packet.

Add SecretBuffer support to TLSContextOptions

Internally, secrets such as private keys should be stored in SecretBuffer not String.
Continue accepting String in the relevant methods, but convert them to SecretBuffer.
Add methods accepting SecretBuffer.

Finish TLS context options

• pkcs11
• pkcs12

Not going to do windows cert store until the JLL builds on windows.

Crash after connection interruption due to an authorization error

I experienced a crash after a connection interruption due to an authorization error (due to publishing to a topic I didn't have permissions for). I'm not sure if this is a bug in my wrapper or in the C libraries.

AWSCRT v0.1.0. Julia v1.7.2.

mqtt_client = MQTTClient(
    ClientTLSContext(
        create_client_with_mtls(
            certificate_pem,
            private_key;
            ca_filepath = joinpath(@__DIR__, "AmazonRootCA1.pem"),
        ),
    ),
)
mqtt_connection = MQTTConnection(mqtt_client)
t = connect(
    mqtt_connection,
    terraform_output(env, "endpoint_url"),
    8883,
    mqtt_client_id;
    on_connection_interrupted = (conn, error_code) -> begin
        @warn "MQTT connection interrupted: $(AWSCRT.aws_err_string(error_code))"
    end,
    on_connection_resumed = (conn, return_code, session_present) -> begin
        @info "MQTT connection resumed" return_code session_present
        if !session_present
            # the server forgot all our subscriptions, so we have to resubscribe from our end
            resub_task = resubscribe_existing_topics(conn)
            @async begin
                resub_task_result = fetch(resub_task)
                @info "Resubscribed existing topics" resub_task_result
            end
        end
    end,
)

┌ Warning: MQTT connection interrupted: AWS Error 5134: libaws-c-mqtt: AWS_ERROR_MQTT_UNEXPECTED_HANGUP, The connection was closed unexpectedly.
└ @ Main REPL[174]:7
┌ Info: MQTT connection resumed
│   return_code = AWS_MQTT_CONNECT_ACCEPTED::aws_mqtt_connect_return_code = 0x00000000
└   session_present = false

Fatal error condition occurred in /workspace/x86_64-linux-gnu-libgfortran3-cxx11/destdir/include/aws/common/array_list.inl:338: !list->length || list->data
Exiting Application
################################################################################
Stack trace:
################################################################################
/home/salmon/.julia/artifacts/92d49a7c9ad8f335cef3f3863856a100fdb18f04/lib/libawscrt.so(aws_backtrace_print+0x41) [0x7ff664b188b1]
/home/salmon/.julia/artifacts/92d49a7c9ad8f335cef3f3863856a100fdb18f04/lib/libawscrt.so(aws_fatal_assert+0x43) [0x7ff664b087d3]
/home/salmon/.julia/artifacts/92d49a7c9ad8f335cef3f3863856a100fdb18f04/lib/libawscrt.so(+0x67532) [0x7ff664ae7532]
/home/salmon/.julia/artifacts/92d49a7c9ad8f335cef3f3863856a100fdb18f04/lib/libawscrt.so(+0xf75cb) [0x7ff664b775cb]
/home/salmon/.julia/artifacts/92d49a7c9ad8f335cef3f3863856a100fdb18f04/lib/libawscrt.so(+0xfd0e5) [0x7ff664b7d0e5]
/home/salmon/.julia/artifacts/92d49a7c9ad8f335cef3f3863856a100fdb18f04/lib/libawscrt.so(+0xd93b6) [0x7ff664b593b6]
/home/salmon/.julia/artifacts/92d49a7c9ad8f335cef3f3863856a100fdb18f04/lib/libawscrt.so(+0x9b4de) [0x7ff664b1b4de]
/home/salmon/.julia/artifacts/92d49a7c9ad8f335cef3f3863856a100fdb18f04/lib/libawscrt.so(+0xe4338) [0x7ff664b64338]
/home/salmon/.julia/artifacts/92d49a7c9ad8f335cef3f3863856a100fdb18f04/lib/libawscrt.so(+0x99133) [0x7ff664b19133]
/lib64/libpthread.so.0(+0x92a5) [0x7ff723a3a2a5]
/lib64/libc.so.6(clone+0x43) [0x7ff723962323]

signal (6): Aborted

Implement proxy options