Comments (6)
You can create a breakpoint on TerarkZipTable creation. Or you may use sst_dump tool to get information.
from terarkdb.
just to confirm. does this mean it is using terarkzip correctly? coz zstd is stated
./sst_dump --file=005967.sst --show_properties
from [] to []
Process 005967.sst
Sst file format: block-based
Table Properties:is:issue is:open
------------------------------
# data blocks: 19124
# entries: 870547
# deletions: 0
# merge operands: 0
# range deletions: 0
raw key size: 49783841
raw average key size: 57.186850
raw value size: 36562974
raw average value size: 42.000000
data block size: 60045443
index block size (user-key? 0, delta-value? 0): 718092
filter block size: 2315629is:issue is:open
(estimated) table size: 63079164
filter policy name: rocksdb.BuiltinBloomFilter
prefix extractor name: nullptr
column family ID: 6
column family name: f
comparator name: leveldb.BytewiseComparator
merge operator name: nullptr
value meta extractor name: nullptr
property collectors names: []
SST file compression algo: ZSTD
creation time: 1628963688
time stamp of earliest key: 0
Raw user collected properties
------------------------------
# rocksdb.block.based.table.index.type: 0x00000000
# rocksdb.block.based.table.prefix.filtering: 0x30
# rocksdb.block.based.table.whole.key.filtering: 0x31
# rocksdb.deleted.keys: 0x00
# rocksdb.merge.operands: 0x00
from terarkdb.
Sst file format: block-based
Seems to be a Block-based Table.
from terarkdb.
can u show what it means to be terarkzipped? how will it look like? what special setting i need to put to use terarkzip?
from terarkdb.
by the way, your documentation is very old. it doesnt seem to work even with the example compilation (doesnt work at all)
from terarkdb.
You may use db_bench
to fill a TerarkDB database with TerarkZipTable. Here is an example command:
./db_bench \
--benchmarks=fillrandom \
--use_existing_db=0 \
--statistics=0 \
--stats_per_interval=1 \
--stats_interval_seconds=60 \
--max_background_flushes=6 \
--max_background_compactions=15 \
--enable_lazy_compaction=0 \
--level0_file_num_compaction_trigger=4 \
--sync=1 \
--allow_concurrent_memtable_write=1 \
--bytes_per_sync=32768 \
--wal_bytes_per_sync=32768 \
--delayed_write_rate=419430400 \
--enable_write_thread_adaptive_yield=1 \
--threads=1 \
--num_levels=7 \
--key_size=36 \
--value_size=8192 \
--level_compaction_dynamic_level_bytes=true \
--mmap_read=false \
--compression_type=zstd \
--memtablerep=patricia_trie \
--blob_size=1024 \
--blob_gc_ratio=0.0625 \
--write_buffer_size=268435456 \
--max_write_buffer_number=10 \
--target_file_size_base=134217728 \
--target_blob_file_size=134217728 \
--blob_file_defragment_size=33554432 \
--max_dependence_blob_overlap=128 \
--optimize_filters_for_hits=true \
--optimize_range_deletion=true \
--num=60000000 \
--db=test_db_1 \
--benchmark_write_rate_limit=20971520 \
--use_terark_table=1
--use_terark_table=1
enables TerarkZipTable on disk, and --memtablerep=patricia_trie
enables in-memory trie format.
sst_dump
doesn't support reading a TerarkZipTable. We have created a separate issue #108 for that.
from terarkdb.
Related Issues (20)
- what is the largest db size of terarkdb? HOT 1
- compilation issue WITH_TESTS=OFF WITH_ZNS=OFF ./build.sh HOT 1
- customized terarkzip default options/options.cc modification. problem as mentioned HOT 4
- terarksql possible to opensource the storage engine? HOT 2
- can someone help make terarkzip default by checking what's wrong with this options.cc file?
- can someone pls provide the build.sh instructions for ubuntu 22.04 docker environment?
- another server with old installation of libterarkzip.a
- trying to use golang binding from grocksdb but realised that terarkzip option not available
- Is there memory growing issue like rocksdb do
- question about delete column family. HOT 7
- No Content in LOG file where using terarkdb in JNI HOT 2
- Is it possible to add switch to schedule TTL HOT 8
- for time based key, which is better? little endian or big endian for this patricia trie?
- May I know if TerarkFS support is done and can be tested?
- Ubuntu 22.04 compilation error
- how stable is terarkdb for production use? HOT 1
- About max subcompaction and blob size HOT 2
- is it possible to separately store l1 and l2 or l3 blob in different disks? HOT 1
- [BUG] example/simple_example.cc is out of date
- different prefix indexing strategies per ColumnFamilyOptions
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from terarkdb.