sfu-dis/ermia

ERMIA: Memory-Optimized OLTP engine for Heterogeneous Workloads (SIGMOD 2016)

ermia's Issues

Build problems

There seems to be a problem with the cmake configuration. The files under /src/dbcore should probably be put into /dbcore; right now cmake complains about not being able to find these files.

It would be nice to have libgflags-dev and libgoogle-glog-dev added as cmake dependencies.

Also, some deprecated casts don't build with g++-7:

In file included from /home/yihehuang/git/ermia/benchmarks/../dbcore/sm-common.h:8:0,
                 from /home/yihehuang/git/ermia/benchmarks/../dbcore/sm-index.h:5,
                 from /home/yihehuang/git/ermia/benchmarks/ndb_wrapper.h:3,
                 from /home/yihehuang/git/ermia/benchmarks/bench.h:12,
                 from /home/yihehuang/git/ermia/benchmarks/bench.cc:15:
/home/yihehuang/git/ermia/benchmarks/../dbcore/sm-oid-alloc-impl.h: In static member function ‘static constexpr size_t sm_allocator::l1_alloc_size()’:
/home/yihehuang/git/ermia/benchmarks/../dbcore/sm-defs.h:231:30: error: reinterpret_cast from integer to pointer
 #define OFFSETOF(tp, expr) (((char *)&((tp *)0)->expr) - ((char *)((tp *)0)))
                             ~^~~~~~~~~~~~~~~~~~~~~~~~~
/home/yihehuang/git/ermia/benchmarks/../dbcore/sm-oid-alloc-impl.h:311:12: note: in expansion of macro ‘OFFSETOF’
     return OFFSETOF(sm_allocator, l1[L1_CAPACITY]);
            ^~~~~~~~

Should be easy to fix. Builds fine with g++-5.

Questions about the central_log_buffer allocation

Hi, Mr Wang.
In window-buffer.cpp:130, you allocate the central_log_buffer in such a strange way.
U allocate an anonymous, private and map_sized virtual address space first. (In a multi-threads program, why MAP_PRIVATE?)
And you mmap an temporal in-memory file twice. (why not ftruncate(fd, map_size) and just mmap once?)
I replaced this code snippet with an single malloc, and ran correctly. (so why not just malloc?)
Appreciate your time for reading this.

Can not get TPC-C to work

TPC-C doesn't seem to work... Am I doing anything wrong here?

./ermia_SI -verbose -benchmark tpcc -threads 8 -scale_factor 8 -seconds 30 -log_data_dir /dev/shm/yihehuang/ermia-log -log_buffer_mb=128 -log_segment_mb=16384 -parallel_loading
WARNING: benchmark built in DEBUG mode!!!
PID: 19882
CC: SI
  phantom-protection: 0
Settings and properties
  node-memory       : 12GB
  num-threads       : 8
  numa-nodes        : 1
  benchmark         : tpcc
  var-encode        : yes
  print-cpu-util    : 0
  log-dir           : /dev/shm/yihehuang/ermia-log
  tmpfs-dir         : /dev/shm
  log-buffer-mb     : 128
  log-ship-by-rdma  : 0
  logbuf-partitions : 64
  worker-threads    : 8
  total-threads     : 8
  persist-policy    : sync
  command-log       : 0
  command-logbuf    : 16MB
  btree_internal_node_size: 264
  btree_leaf_node_size    : 312
  read_view_stat_interval : 0ms
  read_view_stat_file     : /dev/shm/ermia_read_view_stat
  log_ship_offset_replay  : 0
  parallel-loading: 1
  retry-txns        : 0
  backoff-txns      : 0
  scale-factor      : 8
  group-commit      : 0
  commit-queue      : 25000
  group-commit-size : 4KB
  recovery-warm-up  : none
  log-key-for-update: 0
  enable-chkpt      : 0
  enable-gc         : 0
  null-log-device   : 0
  truncate-at-bench-start : 0
  num-backups       : 0
  wait-for-backups  : 1
tpcc settings:
  random home warehouse (%)    : 0
  cross_partition_transactions : 1
  separate_tree_per_partition  : 0
  new_order_remote_item_pct    : 1
  new_order_fast_id_gen        : 0
  uniform_item_dist            : 0
  order_status_scan_hack       : 0
  microbench rows            : 100000
  microbench wr ratio (%)    : 0
  microbench wr rows         : 0
  number of suppliers : 10000
  workload_mix                 : [45, 43, 0, 4, 4, 4, 0, 0]
timed region log_recovery took 801.636 ms
table customer_0 size 0
table customer_name_idx_0 size 0
table district_0 size 0
table history_0 size 0
table item_0 size 38
table nation_0 size 62
table new_order_0 size 0
table oorder_0 size 0
table oorder_c_id_idx_0 size 0
table order_line_0 size 0
table region_0 size 5
table stock_0 size 69
table stock_data_0 size 69
table supplier_0 size 43
table warehouse_0 size 16
starting benchmark...
Sec,Commits,Aborts
F0208 23:23:00.089686 19887 tpcc.cc:1333] Wrong return value 2
*** Check failure stack trace: ***
F0208 23:23:00.089695 19890 Fp1333] Wrong return value 20208 23:23:00.089699 19889 tpcc.cc:1333] Wrong return value 2F0208 23:23:00.089704 19883 tpcc.cc:1333] Wrong return value 2F020F 23:23:00.089721 19888 00002cc:18 5] 23W23:0002F89722 19885 tpcc.cc:1825] 2Wrong return value 208
23:23:F0.89721 19886 tpcc.cc:2825]Wrong return value 208 23:23:00.089721 19884 tpcc.cc:1825] Wrong return value 2
*** Check failure stack trace: ***
F0208 23:23:00.089695 19890 Fp1333] Wrong return value 20208 23:23:00.089699 19889 tpcc.cc:1333] Wrong return value 2F0208 23:23:00.089704 19883 tpcc.cc:1333] Wrong return value 2F020F 23:23:00.089721 19888 00002cc:18 5] 23W23:0002F89722 19885 tpcc.cc:1825] 2Wrong return value 208
23:23:F0.89721 19886 tpcc.cc:2825]Wrong return value 208 23:23:00.089721 19884 tpcc.cc:1825] Wrong return value 2
Aborted (core dumped)

Adding a table to TPC-C

What steps should I follow if I want to add a table to the TPC-C benchmark?

Right now I'm adding a table by changing the TPCC_TABLE_LIST marco, as well as adding an additional RegisterTable call in the constructor of class tpcc_bench_runner. It compiles, loads, and the output appears to show the correct numbers/types of tables being initialized. However it will then reliably crash upon calling OrderedIndex::remove() (in Delivery transactions). It crashes on base_txn_btree.cc:209, for a completely different table (not the new table I added). And the only thing I changed was adding a table...

It's confusing isn't it... Any help/insight is greatly appreciated.

Assertion failures in debug mode

It appears that when building in debug mode, the assertion at base_txn_btree.cc:209 always fails.

The problem doesn't exist when building in release mode (which I believe turn these assertions off). Is it still correct though?

GC performance

Frequent (supposedly infrequent) object pool access:
The GC thread is calling put_object_list too frequently, which needs to acquire a mutex protecting a centralized pool of reclaimed objects.