[BUG]Memtable test hangs

[noobpwnftw]

[BUG]Memtable test hangs

[royguo]

Thanks for reporting this bug.

@yapple and @ustcwelcome Please have a look on this.

[yapple]

@noobpwnftw good question. But that's not a bug.
The Seqreadwrite bench has a least one thread to do write operation, others thread do read operation . Read thread create a read iterator to read all data, and all read threads repeat that seqread operation num_operations times. You can know that by reading the bench code.
so, the faster the write rate, the more write data, the slower the bench rate.
Terarkdb does some little engineering optimizes on Skiplist implementation, so the write Skiplist speed is faster than Rocksdb. You can test that use -num_threads 1.
In conclusion, the bench is mystified, we will refine it in the future.
Thank you for your attention.

[yapple]

On my machine, Terarkdb's write rate is faster than RocksDB 8.5 times(12:1.4).

[noobpwnftw]

For concurrent tests, you need at least 2 threads(one thread is always writer in memtablerep_bench, if -num_threads is 1, seqreadwrite test is the same as fillseq.

[yapple]

For concurrent tests, you need at least 2 threads(one thread is always writer in memtablerep_bench, if -num_threads is 1, seqreadwrite test is the same as fillseq.

Yes, we did the concurrent tests, and reproduce the hangs behavior.
But what I meaning is that the bench logic leads to the behavior.
the faster the write rate, the more write data, the slower the bench rate.
because TerarkDB writes so fast, so on this bench, we hang.

[noobpwnftw]

If it is so fast, how long does it take to write and scan 1000 record 10 times(as per given command)?

[yapple]

1. The command's param num_operations means that scan all data num_operations times.
2. the fill thread finish until the read thread finish, you can see from that code
   ConcurrentFillBenchmarkThread
   because the Skiplist supports multi versions, so the number of records will be far greater than 1000.
   so, you can't use the benchmark to test the case like that write and scan 1000 records 10 times.

[yapple]

@noobpwnftw
./memtablerep_bench -benchmarks fillrandom,readseq
maybe you need this command.

[noobpwnftw]

This issue can be fixed like this:

class ConcurrentFillBenchmarkThread : public FillBenchmarkThread {
 public:
  ConcurrentFillBenchmarkThread(MemTableRep* table, KeyGenerator* key_gen,
                                uint64_t* bytes_written, uint64_t* bytes_read,
                                uint64_t* sequence, uint64_t num_ops,
                                uint64_t* read_hits,
                                std::atomic_int* threads_done)
      : FillBenchmarkThread(table, key_gen, bytes_written, bytes_read, sequence,
                            num_ops, read_hits) {
    threads_done_ = threads_done;
  }

  void operator()() override {
-   // # of read threads will be total threads - write threads (always 1). Loop
-   // while all reads complete.
-   while ((*threads_done_).load() < (FLAGS_num_threads - 1)) {
+   for (unsigned int i = 0; i < num_ops_; ++i) {
      FillOne();
    }
  }