Any way to copy a string into an element atomically?

[cjappl]

Any way to copy a string into an element atomically?

[cjappl]

Just had some rubber duck debugging here explaining the problem to you.

This seems to be working:

Setup:

struct StringBuffer
{
    char mBuffer[1024];
};

moodycamel::ReaderWriterQueue<StringBuffer> mStringBufferQueue {100};

Producer thread:

void ProduceResults()
{
    StringBuffer stringBuffer;
    const char *myString = "hello world!";
    strncpy(stringBuffer.mBuffer, myString, 1024);

    const bool success = mPimpl->mStringBufferQueue.try_enqueue(std::move(stringBuffer));
    assert(success); // If this fails, the queue may be too small
}

consuming thread

void Consume()
{
    StringBuffer stringBuffer;
    const bool foundAnItem = mPimpl->mStringBufferQueue.try_dequeue(stringBuffer);
    if (foundAnItem)
    {
        printf("CAPPLE StringBuffer: %s\n", stringBuffer.mBuffer);
    }
}

Going to leave this up over the weekend while I chew it over a bit more. Anyone and everyone feel free to poke holes in my plans if you feel the urge 😄

Thanks for such a great library, it's coming in extremely handy

[cameron314]

Yeah, was about to post a similar proof of concept:

#include <cstring>
#include <cassert>

struct FixedString {
    char s[128];
};

int main() {
    moodycamel::ReaderWriterQueue<FixedString> q{100};
    q.enqueue(FixedString{ "test123" });
    
    FixedString fs;
    q.try_dequeue(fs);
    assert(strcmp(fs.s, "test123") == 0);
}

[cameron314]

Note that this approach works but involves several copies along the way. Probably better would be to have an efficient allocator of 1024-byte blocks, and pass pointers to them around instead.

For logging specifically, there's usually some formatting (sprintf or ostream) going on when the message is being created, which implies there's already a temporary buffer. Allocating that memory from a pool then moving ownership of it into the queue will result in a minimal number of copies.

Depending on your needs, the "allocator" can be as simple as another queue in the other direction (log producer dequeues pointer to string buffer, writes to it, enqueues pointer to log queue; log consumer dequeues pointer to string buffer, writes it to file, then enqueues pointer to "free" block back onto the first queue).

[cjappl]

Depending on your needs, the "allocator" can be as simple as another queue in the other direction (log producer dequeues pointer to string buffer, writes to it, enqueues pointer to log queue; log consumer dequeues pointer to string buffer, writes it to file, then enqueues pointer to "free" block back onto the first queue).

Wow! Very clever idea....

I think that just may work for what I'm trying to do.

I really appreciate you weighing in, I was feeling very stuck with the problem! I am going to close this, but I will update my progress in this ticket in case someone else has this same question in the future. There is remarkably little content on the web about realtime safe loggers out there!

Thanks again