twmht/python-rocksdb

Python bindings for RocksDB

Note

The original pyrocksdb (https://pypi.python.org/pypi/pyrocksdb/0.4) has not been updated for long time. I update pyrocksdb to support the latest rocksdb. Please open issues in github if you have any problem.

News (2019/04/18)

Currently I am refactoring the code, and more features like TTL are coming soon. And the installation with cmake will be much more easily.

News (2019/04/19)

I have created a new branch(https://github.com/twmht/python-rocksdb/tree/pybind11) which provides the basic functions (put, get and delete) now. And the installtion is much more easily! you can try it if you encounter any installtion issues in the current version of python-rocksdb.

The branch is under development and will be released to PypI after I migrate most of the existing features.

pyrocksdb

Python bindings for RocksDB. See http://python-rocksdb.readthedocs.io/en/latest/ for a more comprehensive install and usage description.

Quick Install

Quick install for debian/ubuntu like linux distributions.

$ apt-get install build-essential libsnappy-dev zlib1g-dev libbz2-dev libgflags-dev liblz4-dev
$ git clone https://github.com/facebook/rocksdb.git
$ cd rocksdb
$ mkdir build && cd build
$ cmake ..
$ make
$ cd ..
$ export CPLUS_INCLUDE_PATH=${CPLUS_INCLUDE_PATH}${CPLUS_INCLUDE_PATH:+:}`pwd`/include/
$ export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}${LD_LIBRARY_PATH:+:}`pwd`/build/
$ export LIBRARY_PATH=${LIBRARY_PATH}${LIBRARY_PATH:+:}`pwd`/build/

$ apt-get install python-virtualenv python-dev
$ virtualenv pyrocks_test
$ cd pyrocks_test
$ . bin/active
$ pip install python-rocksdb

Quick Usage Guide

>>> import rocksdb
>>> db = rocksdb.DB("test.db", rocksdb.Options(create_if_missing=True))
>>> db.put(b'a', b'data')
>>> print db.get(b'a')
b'data'

python-rocksdb's Issues

librocksdb.so.5.13 cannot open shared object file

Hi,

After following the instructions it looks like rocksdb cannot be resolved. From Visual Code, when importing rocksdb I get 'Unresolved import rocksb', from the command line, when running python and doing an import I get: 'ImportError: librocksdb.so.5.13: cannot open shared object file: No such file or directory'.

It is happening with and without VirtualEnv.

Any ideas?

Thanks.

Env:
Python 2.7.15rc1.
Ubuntu 18.04.
Rocksdb 5.12.4.

how can i update on ubuntu

i want to update, how can i?

Add support for openAsSecondary

RocksDB 6.1 has added support for secondary instances which allow read only connections to continuously follow changes from a primary instance (not possible with the current readonly support).

I believe we need to add support for the new OpenAsSecondary method to expose that feature to python-rocksdb.

This comment has more background.

how to set a merge_operator default?

hi,twmht,I read pyrocksdb doc,only one case to set merge_operator AssocCounter,rocks have default mergeOperator,such "uint64add",in java code options.setMergeOperatorName("uint64add");

how to do the same thing in python-rocksdb?

class AssocCounter(rocksdb.interfaces.AssociativeMergeOperator):
def merge(self, key, existing_value, value):
if existing_value:
s = int(existing_value) + int(value)
return (True, str(s).encode('ascii'))
return (True, value)

def name(self):
    return b'AssocCounter'

opts = rocksdb.Options()
opts.create_if_missing = True
opts.merge_operator = AssocCounter()

Import Error - undefined symbol: _ZN7rocksdb11NewLRUCacheEmibd

Debian 9 with Python 3.7.2 built from source. Application I use is suggested to update to python3.7 so trying to get rocksdb (python-rocksdb) working. In 3.6 was using pyrocksdb without issue.

Subject line describes the error

ImportError: /usr/local/lib/python3.7/site-packages/rocksdb/_rocksdb.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN7rocksdb11NewLRUCacheEmibd

python-rocksdb have some problem in time-to-live db

when a rockdb opened by with a time-to-live option,in java is

Options options = new Options().setCreateIfMissing(true);

            options.setKeepLogFileNum(10);
            rawDatadb = TtlDB.open(options, "xxx", 43200, false);

when open using python-rocksdb,the value will print some odd character,it maybe the inner data of time-to-live,when open by python-rocksdb: rocks_db = rocksdb.DB("xxx", opts, read_only=True),even add a ttl to the DB is the same

such as:
[{"name":"haha"}]�E�X

Quick Install instructions in README.rst are out of date

$ git clone https://github.com/facebook/rocksdb.git
Cloning into 'rocksdb'...
remote: Counting objects: 53785, done.
remote: Compressing objects: 100% (10/10), done.
remote: Total 53785 (delta 0), reused 2 (delta 0), pack-reused 53775
Receiving objects: 100% (53785/53785), 100.79 MiB | 7.72 MiB/s, done.
Resolving deltas: 100% (40041/40041), done.
$ cd rocksdb/
$ make build
Makefile:105: Warning: Compiling in debug mode. Don't use the resulting binary in production
  GEN      util/build_version.cc
Makefile:105: Warning: Compiling in debug mode. Don't use the resulting binary in production
  GEN      util/build_version.cc
make: *** No rule to make target 'build'.  Stop.

As you can see, following them leads to failed build. Additionally, I don't think it should be done in debug mode.

install problems in mac os

Hi,

I have tried to install python-rocksdb on mac os. Here is the steps:

1. brew install rocksdb
2. pip install python-rocksdb

After then, python -c 'import rocksdb'

Symbol not found: __ZNK7rocksdb13MergeOperator17PartialMergeMultiERKNS_5SliceERKSt5dequeIS1_SaIS1_EEPSsPNS_6LoggerE
Referenced from: /Users/liu/Library/Caches/Python-Eggs/python_rocksdb-0.6.6-py2.7-macosx-10.7-x86_64.egg-tmp/rocksdb/_rocksdb.so
Expected in: flat namespace

I also try to install from source code of python-rocksdb, however , the same problems apprear.

So, Is there any instructions for install python-rocksdb on mac os? Thx

Installation error on python 3.6.2

Followed steps in rocksdb installation on OSX and its installed perfectly. But pip install python-rocksdb is having issues, its failing with below error. Please advise any work around.

$ brew install rocksdb
Warning: rocksdb 5.7.3 is already installed

$python --version
Python 3.6.2

$pip install python-rocksdb
Collecting python-rocksdb
Using cached python-rocksdb-0.6.7.tar.gz
Requirement already satisfied: setuptools in /Users/ /virtualenvs/test_grpc/lib/python3.6/site-packages (from python-rocksdb)
Building wheels for collected packages: python-rocksdb
Running setup.py bdist_wheel for python-rocksdb ... error
Complete output from command /Users/ /virtualenvs/test_grpc/bin/python3.6 -u -c "import setuptools, tokenize;file='/private/var/folders/k0/x72z1czs6wz8pxbw0lw7055c0000gn/T/pip-build-3q9ryllh/python-rocksdb/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" bdist_wheel -d /var/folders/k0/x72z1czs6wz8pxbw0lw7055c0000gn/T/tmphnyp9bqkpip-wheel- --python-tag cp36:
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-10.6-intel-3.6
creating build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/init.py -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/errors.py -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/interfaces.py -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/merge_operators.py -> build/lib.macosx-10.6-intel-3.6/rocksdb
creating build/lib.macosx-10.6-intel-3.6/rocksdb/tests
copying rocksdb/tests/init.py -> build/lib.macosx-10.6-intel-3.6/rocksdb/tests
copying rocksdb/tests/test_db.py -> build/lib.macosx-10.6-intel-3.6/rocksdb/tests
copying rocksdb/tests/test_memtable.py -> build/lib.macosx-10.6-intel-3.6/rocksdb/tests
copying rocksdb/tests/test_options.py -> build/lib.macosx-10.6-intel-3.6/rocksdb/tests
running egg_info
writing python_rocksdb.egg-info/PKG-INFO
writing dependency_links to python_rocksdb.egg-info/dependency_links.txt
writing requirements to python_rocksdb.egg-info/requires.txt
writing top-level names to python_rocksdb.egg-info/top_level.txt
reading manifest file 'python_rocksdb.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'python_rocksdb.egg-info/SOURCES.txt'
copying rocksdb/_rocksdb.cpp -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/rocksdb.pyx -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/backup.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/cache.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/comparator.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/db.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/env.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/filter_policy.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/iterator.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/logger.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/memtablerep.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/merge_operator.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/options.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/slice.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/slice_transform.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/snapshot.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/status.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/std_memory.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/table_factory.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/universal_compaction.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
creating build/lib.macosx-10.6-intel-3.6/rocksdb/cpp
copying rocksdb/cpp/comparator_wrapper.hpp -> build/lib.macosx-10.6-intel-3.6/rocksdb/cpp
copying rocksdb/cpp/filter_policy_wrapper.hpp -> build/lib.macosx-10.6-intel-3.6/rocksdb/cpp
copying rocksdb/cpp/memtable_factories.hpp -> build/lib.macosx-10.6-intel-3.6/rocksdb/cpp
copying rocksdb/cpp/merge_operator_wrapper.hpp -> build/lib.macosx-10.6-intel-3.6/rocksdb/cpp
copying rocksdb/cpp/slice_transform_wrapper.hpp -> build/lib.macosx-10.6-intel-3.6/rocksdb/cpp
copying rocksdb/cpp/utils.hpp -> build/lib.macosx-10.6-intel-3.6/rocksdb/cpp
copying rocksdb/cpp/write_batch_iter_helper.hpp -> build/lib.macosx-10.6-intel-3.6/rocksdb/cpp
running build_ext
building 'rocksdb.rocksdb' extension
creating build/temp.macosx-10.6-intel-3.6
creating build/temp.macosx-10.6-intel-3.6/rocksdb
/usr/bin/clang -fno-strict-aliasing -Wsign-compare -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.6/include/python3.6m -c rocksdb/rocksdb.cpp -o build/temp.macosx-10.6-intel-3.6/rocksdb/rocksdb.o -std=c++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing
In file included from rocksdb/rocksdb.cpp:491:
In file included from /usr/local/include/rocksdb/env.h:27:
/usr/local/include/rocksdb/status.h:286:16: error: no member named 'move' in namespace 'std'
*this = std::move(s);
~~~~~^
/usr/local/include/rocksdb/status.h:295:18: error: no member named 'move' in namespace 'std'
code = std::move(s.code);
~~~~~^
/usr/local/include/rocksdb/status.h:297:21: error: no member named 'move' in namespace 'std'
subcode = std::move(s.subcode);
~~~~~^
In file included from rocksdb/_rocksdb.cpp:491:
/usr/local/include/rocksdb/env.h:52:12: error: no member named 'unique_ptr' in namespace 'std'
using std::unique_ptr;
~~~~~^
/usr/local/include/rocksdb/env.h:53:12: error: no member named 'shared_ptr' in namespace 'std'
using std::shared_ptr;
~~~~~^
/usr/local/include/rocksdb/env.h:139:36: error: unknown type name 'unique_ptr'
unique_ptr* result,
^
/usr/local/include/rocksdb/env.h:139:46: error: expected ')'
unique_ptr* result,
^
/usr/local/include/rocksdb/env.h:138:35: note: to match this '('
virtual Status NewSequentialFile(const std::string& fname,
^
/usr/local/include/rocksdb/env.h:151:38: error: unknown type name 'unique_ptr'
unique_ptr* result,
^
/usr/local/include/rocksdb/env.h:151:48: error: expected ')'
unique_ptr* result,
^
/usr/local/include/rocksdb/env.h:150:37: note: to match this '('
virtual Status NewRandomAccessFile(const std::string& fname,
^
/usr/local/include/rocksdb/env.h:163:34: error: unknown type name 'unique_ptr'
unique_ptr* result,
^
/usr/local/include/rocksdb/env.h:163:44: error: expected ')'
unique_ptr* result,
^
/usr/local/include/rocksdb/env.h:162:33: note: to match this '('
virtual Status NewWritableFile(const std::string& fname,
^
/usr/local/include/rocksdb/env.h:174:37: error: unknown type name 'unique_ptr'
unique_ptr* result,
^
/usr/local/include/rocksdb/env.h:174:47: error: expected ')'
unique_ptr* result,
^
/usr/local/include/rocksdb/env.h:173:36: note: to match this '('
virtual Status ReopenWritableFile(const std::string& fname,
^
/usr/local/include/rocksdb/env.h:182:36: error: unknown type name 'unique_ptr'
unique_ptr* result,
^
/usr/local/include/rocksdb/env.h:182:46: error: expected ')'
unique_ptr* result,
^
/usr/local/include/rocksdb/env.h:180:35: note: to match this '('
virtual Status ReuseWritableFile(const std::string& fname,
^
/usr/local/include/rocksdb/env.h:191:34: error: unknown type name 'unique_ptr'
unique_ptr* result,
^
/usr/local/include/rocksdb/env.h:191:44: error: expected ')'
unique_ptr* result,
^
/usr/local/include/rocksdb/env.h:190:33: note: to match this '('
virtual Status NewRandomRWFile(const std::string& fname,
^
/usr/local/include/rocksdb/env.h:204:31: error: unknown type name 'unique_ptr'
unique_ptr* result) = 0;
^
/usr/local/include/rocksdb/env.h:204:41: error: expected ')'
unique_ptr* result) = 0;
^
/usr/local/include/rocksdb/env.h:203:30: note: to match this '('
virtual Status NewDirectory(const std::string& name,
^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
error: command '/usr/bin/clang' failed with exit status 1

---

Failed building wheel for python-rocksdb
Running setup.py clean for python-rocksdb
Failed to build python-rocksdb
Installing collected packages: python-rocksdb
Running setup.py install for python-rocksdb ... error
Complete output from command /Users/ /virtualenvs/test_grpc/bin/python3.6 -u -c "import setuptools, tokenize;file='/private/var/folders/k0/x72z1czs6wz8pxbw0lw7055c0000gn/T/pip-build-3q9ryllh/python-rocksdb/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /var/folders/k0/x72z1czs6wz8pxbw0lw7055c0000gn/T/pip-kmhr4f9j-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/ /virtualenvs/test_grpc/bin/../include/site/python3.6/python-rocksdb:
running install
running build
running build_py
creating build
creating build/lib.macosx-10.6-intel-3.6
creating build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/init.py -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/errors.py -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/interfaces.py -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/merge_operators.py -> build/lib.macosx-10.6-intel-3.6/rocksdb
creating build/lib.macosx-10.6-intel-3.6/rocksdb/tests
copying rocksdb/tests/init.py -> build/lib.macosx-10.6-intel-3.6/rocksdb/tests
copying rocksdb/tests/test_db.py -> build/lib.macosx-10.6-intel-3.6/rocksdb/tests
copying rocksdb/tests/test_memtable.py -> build/lib.macosx-10.6-intel-3.6/rocksdb/tests
copying rocksdb/tests/test_options.py -> build/lib.macosx-10.6-intel-3.6/rocksdb/tests
running egg_info
writing python_rocksdb.egg-info/PKG-INFO
writing dependency_links to python_rocksdb.egg-info/dependency_links.txt
writing requirements to python_rocksdb.egg-info/requires.txt
writing top-level names to python_rocksdb.egg-info/top_level.txt
reading manifest file 'python_rocksdb.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'python_rocksdb.egg-info/SOURCES.txt'
copying rocksdb/_rocksdb.cpp -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/rocksdb.pyx -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/backup.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/cache.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/comparator.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/db.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/env.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/filter_policy.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/iterator.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/logger.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/memtablerep.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/merge_operator.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/options.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/slice.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/slice_transform.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/snapshot.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/status.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/std_memory.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/table_factory.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
copying rocksdb/universal_compaction.pxd -> build/lib.macosx-10.6-intel-3.6/rocksdb
creating build/lib.macosx-10.6-intel-3.6/rocksdb/cpp
copying rocksdb/cpp/comparator_wrapper.hpp -> build/lib.macosx-10.6-intel-3.6/rocksdb/cpp
copying rocksdb/cpp/filter_policy_wrapper.hpp -> build/lib.macosx-10.6-intel-3.6/rocksdb/cpp
copying rocksdb/cpp/memtable_factories.hpp -> build/lib.macosx-10.6-intel-3.6/rocksdb/cpp
copying rocksdb/cpp/merge_operator_wrapper.hpp -> build/lib.macosx-10.6-intel-3.6/rocksdb/cpp
copying rocksdb/cpp/slice_transform_wrapper.hpp -> build/lib.macosx-10.6-intel-3.6/rocksdb/cpp
copying rocksdb/cpp/utils.hpp -> build/lib.macosx-10.6-intel-3.6/rocksdb/cpp
copying rocksdb/cpp/write_batch_iter_helper.hpp -> build/lib.macosx-10.6-intel-3.6/rocksdb/cpp
running build_ext
building 'rocksdb.rocksdb' extension
creating build/temp.macosx-10.6-intel-3.6
creating build/temp.macosx-10.6-intel-3.6/rocksdb
/usr/bin/clang -fno-strict-aliasing -Wsign-compare -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.6/include/python3.6m -c rocksdb/rocksdb.cpp -o build/temp.macosx-10.6-intel-3.6/rocksdb/rocksdb.o -std=c++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing
In file included from rocksdb/rocksdb.cpp:491:
In file included from /usr/local/include/rocksdb/env.h:27:
/usr/local/include/rocksdb/status.h:286:16: error: no member named 'move' in namespace 'std'
*this = std::move(s);
~~~~~^
/usr/local/include/rocksdb/status.h:295:18: error: no member named 'move' in namespace 'std'
code = std::move(s.code);
~~~~~^
/usr/local/include/rocksdb/status.h:297:21: error: no member named 'move' in namespace 'std'
subcode = std::move(s.subcode);
~~~~~^
In file included from rocksdb/_rocksdb.cpp:491:
/usr/local/include/rocksdb/env.h:52:12: error: no member named 'unique_ptr' in namespace 'std'
using std::unique_ptr;
~~~~~^
/usr/local/include/rocksdb/env.h:53:12: error: no member named 'shared_ptr' in namespace 'std'
using std::shared_ptr;
~~~~~^
/usr/local/include/rocksdb/env.h:139:36: error: unknown type name 'unique_ptr'
unique_ptr* result,
^
/usr/local/include/rocksdb/env.h:139:46: error: expected ')'
unique_ptr* result,
^
/usr/local/include/rocksdb/env.h:138:35: note: to match this '('
virtual Status NewSequentialFile(const std::string& fname,
^
/usr/local/include/rocksdb/env.h:151:38: error: unknown type name 'unique_ptr'
unique_ptr* result,
^
/usr/local/include/rocksdb/env.h:151:48: error: expected ')'
unique_ptr* result,
^
/usr/local/include/rocksdb/env.h:150:37: note: to match this '('
virtual Status NewRandomAccessFile(const std::string& fname,
^
/usr/local/include/rocksdb/env.h:163:34: error: unknown type name 'unique_ptr'
unique_ptr* result,
^
/usr/local/include/rocksdb/env.h:163:44: error: expected ')'
unique_ptr* result,
^
/usr/local/include/rocksdb/env.h:162:33: note: to match this '('
virtual Status NewWritableFile(const std::string& fname,
^
/usr/local/include/rocksdb/env.h:174:37: error: unknown type name 'unique_ptr'
unique_ptr* result,
^
/usr/local/include/rocksdb/env.h:174:47: error: expected ')'
unique_ptr* result,
^
/usr/local/include/rocksdb/env.h:173:36: note: to match this '('
virtual Status ReopenWritableFile(const std::string& fname,
^
/usr/local/include/rocksdb/env.h:182:36: error: unknown type name 'unique_ptr'
unique_ptr* result,
^
/usr/local/include/rocksdb/env.h:182:46: error: expected ')'
unique_ptr* result,
^
/usr/local/include/rocksdb/env.h:180:35: note: to match this '('
virtual Status ReuseWritableFile(const std::string& fname,
^
/usr/local/include/rocksdb/env.h:191:34: error: unknown type name 'unique_ptr'
unique_ptr* result,
^
/usr/local/include/rocksdb/env.h:191:44: error: expected ')'
unique_ptr* result,
^
/usr/local/include/rocksdb/env.h:190:33: note: to match this '('
virtual Status NewRandomRWFile(const std::string& fname,
^
/usr/local/include/rocksdb/env.h:204:31: error: unknown type name 'unique_ptr'
unique_ptr* result) = 0;
^
/usr/local/include/rocksdb/env.h:204:41: error: expected ')'
unique_ptr* result) = 0;
^
/usr/local/include/rocksdb/env.h:203:30: note: to match this '('
virtual Status NewDirectory(const std::string& name,
^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
error: command '/usr/bin/clang' failed with exit status 1

----------------------------------------

Command "/Users/ /virtualenvs/test_grpc/bin/python3.6 -u -c "import setuptools, tokenize;file='/private/var/folders/k0/x72z1czs6wz8pxbw0lw7055c0000gn/T/pip-build-3q9ryllh/python-rocksdb/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /var/folders/k0/x72z1czs6wz8pxbw0lw7055c0000gn/T/pip-kmhr4f9j-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/~/virtualenvs/test_grpc/bin/../include/site/python3.6/python-rocksdb" failed with error code 1 in /private/var/folders/k0/x72z1czs6wz8pxbw0lw7055c0000gn/T/pip-build-3q9ryllh/python-rocksdb/

Test failure in module

Hi!

Thanks for the development work. I'm attempting to add a feature (which I will PR if I am successful), but after installing rocksdb 5.4.6 on Sierra and compiling your cython wrapper, I am getting a test failure in TestOptions.test_simple:

>       self.assertEqual(rocksdb.CompressionType.no_compression, opts.compression)
E       AssertionError: u'no_compression' != u'snappy_compression'
E       - no_compression
E       ?  ^
E       + snappy_compression
E       ? + ^^^^

From what I can tell, it looks like the default compression type is not set in your wrapper, so must be set in the base library -- indeed this seems to be the case, at least for the java base library?

I am assuming this is safe to ignore but wanted to bring it to your attention.

EDIT It looks like it defaults to Snappy if it is supported, otherwise to no compression:

https://github.com/facebook/rocksdb/blob/master/options/cf_options.h#L182
Thank you!

pip install rocksdb fails in windows 10

Hi.
I tried to download rocksdb via pip and its fails.
I have:
Windows 10
Python 3.6.7

C:\Windows\System32>pip install python-rocksdb
Collecting python-rocksdb
Using cached https://files.pythonhosted.org/packages/94/81/9a8e498a26ec34da9e1ae87e831468aedb4a6ba42aa9ea779b97896cfea7/python-rocksdb-0.7.0.tar.gz
Requirement already satisfied: setuptools>=25 in c:\users\user\appdata\local\programs\python\python36\lib\site-packages (from python-rocksdb) (39.0.1)
Building wheels for collected packages: python-rocksdb
Building wheel for python-rocksdb (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: 'c:\users\user\appdata\local\programs\python\python36\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\user\AppData\Local\Temp\pip-install-y3yr_tqw\python-rocksdb\setup.py'"'"'; file='"'"'C:\Users\user\AppData\Local\Temp\pip-install-y3yr_tqw\python-rocksdb\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\user\AppData\Local\Temp\pip-wheel-igp6foi2' --python-tag cp36
cwd: C:\Users\user\AppData\Local\Temp\pip-install-y3yr_tqw\python-rocksdb
Complete output (60 lines):
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-3.6
creating build\lib.win-amd64-3.6\rocksdb
copying rocksdb\errors.py -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\interfaces.py -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\merge_operators.py -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb_init_.py -> build\lib.win-amd64-3.6\rocksdb
creating build\lib.win-amd64-3.6\rocksdb\tests
copying rocksdb\tests\test_db.py -> build\lib.win-amd64-3.6\rocksdb\tests
copying rocksdb\tests\test_memtable.py -> build\lib.win-amd64-3.6\rocksdb\tests
copying rocksdb\tests\test_options.py -> build\lib.win-amd64-3.6\rocksdb\tests
copying rocksdb\tests_init_.py -> build\lib.win-amd64-3.6\rocksdb\tests
running egg_info
writing python_rocksdb.egg-info\PKG-INFO
writing dependency_links to python_rocksdb.egg-info\dependency_links.txt
writing requirements to python_rocksdb.egg-info\requires.txt
writing top-level names to python_rocksdb.egg-info\top_level.txt
reading manifest file 'python_rocksdb.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'python_rocksdb.egg-info\SOURCES.txt'
copying rocksdb_rocksdb.cpp -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb_rocksdb.pyx -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\backup.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\cache.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\comparator.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\db.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\env.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\filter_policy.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\iterator.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\logger.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\memtablerep.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\merge_operator.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\options.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\slice_.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\slice_transform.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\snapshot.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\status.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\std_memory.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\table_factory.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\universal_compaction.pxd -> build\lib.win-amd64-3.6\rocksdb
creating build\lib.win-amd64-3.6\rocksdb\cpp
copying rocksdb\cpp\comparator_wrapper.hpp -> build\lib.win-amd64-3.6\rocksdb\cpp
copying rocksdb\cpp\filter_policy_wrapper.hpp -> build\lib.win-amd64-3.6\rocksdb\cpp
copying rocksdb\cpp\memtable_factories.hpp -> build\lib.win-amd64-3.6\rocksdb\cpp
copying rocksdb\cpp\merge_operator_wrapper.hpp -> build\lib.win-amd64-3.6\rocksdb\cpp
copying rocksdb\cpp\slice_transform_wrapper.hpp -> build\lib.win-amd64-3.6\rocksdb\cpp
copying rocksdb\cpp\utils.hpp -> build\lib.win-amd64-3.6\rocksdb\cpp
copying rocksdb\cpp\write_batch_iter_helper.hpp -> build\lib.win-amd64-3.6\rocksdb\cpp
running build_ext
cythoning rocksdb/_rocksdb.pyx to rocksdb_rocksdb.cpp
building 'rocksdb._rocksdb' extension
creating build\temp.win-amd64-3.6
creating build\temp.win-amd64-3.6\Release
creating build\temp.win-amd64-3.6\Release\rocksdb
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Ic:\users\user\appdata\local\programs\python\python36\include -Ic:\users\user\appdata\local\programs\python\python36\include "-IC:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\cppwinrt" /EHsc /Tprocksdb_rocksdb.cpp /Fobuild\temp.win-amd64-3.6\Release\rocksdb_rocksdb.obj -std=c++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing -fno-rtti
cl : Command line error D8021 : invalid numeric argument '/Wextra'
error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\bin\HostX86\x64\cl.exe' failed with exit status 2

ERROR: Failed building wheel for python-rocksdb
Running setup.py clean for python-rocksdb
Failed to build python-rocksdb
Installing collected packages: python-rocksdb
Running setup.py install for python-rocksdb ... error
ERROR: Command errored out with exit status 1:
command: 'c:\users\user\appdata\local\programs\python\python36\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\user\AppData\Local\Temp\pip-install-y3yr_tqw\python-rocksdb\setup.py'"'"'; file='"'"'C:\Users\user\AppData\Local\Temp\pip-install-y3yr_tqw\python-rocksdb\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record 'C:\Users\user\AppData\Local\Temp\pip-record-b5iqi1cs\install-record.txt' --single-version-externally-managed --compile
cwd: C:\Users\user\AppData\Local\Temp\pip-install-y3yr_tqw\python-rocksdb
Complete output (60 lines):
running install
running build
running build_py
creating build
creating build\lib.win-amd64-3.6
creating build\lib.win-amd64-3.6\rocksdb
copying rocksdb\errors.py -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\interfaces.py -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\merge_operators.py -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb_init_.py -> build\lib.win-amd64-3.6\rocksdb
creating build\lib.win-amd64-3.6\rocksdb\tests
copying rocksdb\tests\test_db.py -> build\lib.win-amd64-3.6\rocksdb\tests
copying rocksdb\tests\test_memtable.py -> build\lib.win-amd64-3.6\rocksdb\tests
copying rocksdb\tests\test_options.py -> build\lib.win-amd64-3.6\rocksdb\tests
copying rocksdb\tests_init_.py -> build\lib.win-amd64-3.6\rocksdb\tests
running egg_info
writing python_rocksdb.egg-info\PKG-INFO
writing dependency_links to python_rocksdb.egg-info\dependency_links.txt
writing requirements to python_rocksdb.egg-info\requires.txt
writing top-level names to python_rocksdb.egg-info\top_level.txt
reading manifest file 'python_rocksdb.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'python_rocksdb.egg-info\SOURCES.txt'
copying rocksdb_rocksdb.cpp -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb_rocksdb.pyx -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\backup.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\cache.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\comparator.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\db.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\env.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\filter_policy.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\iterator.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\logger.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\memtablerep.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\merge_operator.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\options.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\slice_.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\slice_transform.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\snapshot.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\status.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\std_memory.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\table_factory.pxd -> build\lib.win-amd64-3.6\rocksdb
copying rocksdb\universal_compaction.pxd -> build\lib.win-amd64-3.6\rocksdb
creating build\lib.win-amd64-3.6\rocksdb\cpp
copying rocksdb\cpp\comparator_wrapper.hpp -> build\lib.win-amd64-3.6\rocksdb\cpp
copying rocksdb\cpp\filter_policy_wrapper.hpp -> build\lib.win-amd64-3.6\rocksdb\cpp
copying rocksdb\cpp\memtable_factories.hpp -> build\lib.win-amd64-3.6\rocksdb\cpp
copying rocksdb\cpp\merge_operator_wrapper.hpp -> build\lib.win-amd64-3.6\rocksdb\cpp
copying rocksdb\cpp\slice_transform_wrapper.hpp -> build\lib.win-amd64-3.6\rocksdb\cpp
copying rocksdb\cpp\utils.hpp -> build\lib.win-amd64-3.6\rocksdb\cpp
copying rocksdb\cpp\write_batch_iter_helper.hpp -> build\lib.win-amd64-3.6\rocksdb\cpp
running build_ext
skipping 'rocksdb_rocksdb.cpp' Cython extension (up-to-date)
building 'rocksdb._rocksdb' extension
creating build\temp.win-amd64-3.6
creating build\temp.win-amd64-3.6\Release
creating build\temp.win-amd64-3.6\Release\rocksdb
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Ic:\users\user\appdata\local\programs\python\python36\include -Ic:\users\user\appdata\local\programs\python\python36\include "-IC:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\cppwinrt" /EHsc /Tprocksdb_rocksdb.cpp /Fobuild\temp.win-amd64-3.6\Release\rocksdb_rocksdb.obj -std=c++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing -fno-rtti
cl : Command line error D8021 : invalid numeric argument '/Wextra'
error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\bin\HostX86\x64\cl.exe' failed with exit status 2
----------------------------------------
ERROR: Command errored out with exit status 1: 'c:\users\user\appdata\local\programs\python\python36\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\user\AppData\Local\Temp\pip-install-y3yr_tqw\python-rocksdb\setup.py'"'"'; file='"'"'C:\Users\user\AppData\Local\Temp\pip-install-y3yr_tqw\python-rocksdb\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record 'C:\Users\user\AppData\Local\Temp\pip-record-b5iqi1cs\install-record.txt' --single-version-externally-managed --compile Check the logs for full command output.

No attributes in rocksdb module

Hi guys,
I just tried to install the python-rocksdb lib and facing this problem. It seemed like everything worked just fine but when I try to run a basic script I get the following error:

python3 rocksdb.py 
Traceback (most recent call last):
  File "rocksdb.py", line 1, in <module>
    import rocksdb
  File "/Users/eeepmb/workspace/rocksdb.py", line 3, in <module>
    db = rocksdb.DB("test.db", rocksdb.Options(create_if_missing=True))
AttributeError: module 'rocksdb' has no attribute 'DB'

Unfortunately I don't have the original logging anymore but when I rerun the commands I get the following output.

brew install rocksdb
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 3 taps (homebrew/core, homebrew/cask and caskroom/cask).
==> New Formulae
anime-downloader
==> Updated Formulae
corectl vamp-plugin-sdk
Warning: rocksdb 6.1.2 is already installed and up-to-date
To reinstall 6.1.2, run brew reinstall rocksdb

pip3 install python-rocksdb
Requirement already satisfied: python-rocksdb in /usr/local/lib/python3.7/site-packages (0.7.0)
Requirement already satisfied: setuptools>=25 in /usr/local/lib/python3.7/site-packages (from python-rocksdb) (41.0.1)

I'm using Python 3.7.3 (.. but had the same issue with Python 3.6.5) on macOS Mojave v.10.14.5.
I already googled a lot but couldn't find much on this. It's also weird because in my IDE (Visual Studio Code) the modules are listed when I type rocksdb..

Any help on this would be highly appreciated.

Publish binary wheel with RocksDB included

Hey,

since it's not trivial to find and build a rocksdb version that works with this library, it would make sense to publish binary wheels on PyPi that include rocksdb.

Tools like https://github.com/joerick/cibuildwheel make creating such wheels easy and automated.

Adding a close() method

This a request that came up many times on the old pyrocksdb repo, but was never implemented. The reasoning seemed off to me. Without a close() method, there really is no way to guarantee a lock is released as Python's garbage collection can't fully be relied on for this.

If you're consistently recreating databases from scratch (as I am in my use-case), this really complicates things. Any chance such a method can be added?

Error on Ubuntu - Snappy not supported or corrupted Snappy compressed block contents'

Installation of RocksDB and Python-RocksDB went smoothly.

Then I tried to access a db I got this error.

Traceback (most recent call last):
  File "main.py", line 57, in <module>
    name_v = vectorize(name_w)
  File "main.py", line 39, in vectorize
    x = db.get(k.encode())
  File "rocksdb/_rocksdb.pyx", line 1796, in rocksdb._rocksdb.DB.get
  File "rocksdb/_rocksdb.pyx", line 76, in rocksdb._rocksdb.check_status
rocksdb.errors.Corruption: b'Corruption: Snappy not supported or corrupted Snappy compressed block contents'

segmentation fault

Something went wrong when i delete the db pointer :) For me it is not a critical, because i never close the database without ending my program.

(fsPython) root@srv:/own/services/back/src/database/new# python
Python 3.5.3 (default, Sep 27 2018, 17:25:39) 
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import rocksdb
>>> opt = rocksdb.Options()
>>> db = rocksdb.DB("/tmp/2323", rocksdb.Options(create_if_missing=True))
>>> db.close()
>>> del db
Segmentation fault
(fsPython) root@srv:/own/services/back/src/database/new# 

Maybe it is not good when i got this during exit:

(fsPython) root@srv:/own/services/back/src/database/new# python
Python 3.5.3 (default, Sep 27 2018, 17:25:39) 
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import rocksdb
>>> opt = rocksdb.Options()
>>> db = rocksdb.DB("/tmp/2323", rocksdb.Options(create_if_missing=True))
>>> db.close()
>>> exit()
Segmentation fault
(fsPython) root@srv:/own/services/back/src/database/new# 

Linux srv 4.9.0-8-amd64 #1 SMP Debian 4.9.144-3.1 (2019-02-19) x86_64 GNU/Linux

Update

Debug:
https://gist.github.com/iFA88/1e688ea42a2e58d0f8ca88f16657acfa

Install fail in py3.7 success in py3.6

python-rocksdb-0.7.0
OSX 10.14.4
py3.7.3

Has installed rocksdb in OSX

➜  ~ brew reinstall rocksdb
==> Reinstalling rocksdb
==> Downloading https://homebrew.bintray.com/bottles/rocksdb-5.18.3.mojave.bottle.tar.gz
Already downloaded: /Users/wyx/Library/Caches/Homebrew/downloads/7361dd525c184cda02a61c7cf325294a92e67c8a5cc88840aa6e77ad50dcc154--rocksdb-5.18.3.mojave.bottle.tar.gz
==> Pouring rocksdb-5.18.3.mojave.bottle.tar.gz
🍺  /usr/local/Cellar/rocksdb/5.18.3: 107 files, 58.8MB

ERROR report

Collecting python-rocksdb
  Downloading https://files.pythonhosted.org/packages/94/81/9a8e498a26ec34da9e1ae87e831468aedb4a6ba42aa9ea779b97896cfea7/python-rocksdb-0.7.0.tar.gz (219kB)
     |████████████████████████████████| 225kB 83kB/s 
Requirement already satisfied: setuptools>=25 in ./.env/lib/python3.7/site-packages (from python-rocksdb) (41.0.1)
Building wheels for collected packages: python-rocksdb
  Building wheel for python-rocksdb (setup.py) ... error
  ERROR: Complete output from command /Users/wyx/loh_workspace/loh-web/.env/bin/python3.7 -u -c 'import setuptools, tokenize;__file__='"'"'/private/var/folders/pr/y105tp4s1w761lxysjs2pg0m0000gn/T/pip-install-s9tb5kpr/python-rocksdb/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/pr/y105tp4s1w761lxysjs2pg0m0000gn/T/pip-wheel-5o0g2po0 --python-tag cp37:
  ERROR: running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.macosx-10.14-x86_64-3.7
  creating build/lib.macosx-10.14-x86_64-3.7/rocksdb
  copying rocksdb/interfaces.py -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
  copying rocksdb/merge_operators.py -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
  copying rocksdb/__init__.py -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
  copying rocksdb/errors.py -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
  creating build/lib.macosx-10.14-x86_64-3.7/rocksdb/tests
  copying rocksdb/tests/test_options.py -> build/lib.macosx-10.14-x86_64-3.7/rocksdb/tests
  copying rocksdb/tests/test_db.py -> build/lib.macosx-10.14-x86_64-3.7/rocksdb/tests
  copying rocksdb/tests/__init__.py -> build/lib.macosx-10.14-x86_64-3.7/rocksdb/tests
  copying rocksdb/tests/test_memtable.py -> build/lib.macosx-10.14-x86_64-3.7/rocksdb/tests
  running egg_info
  writing python_rocksdb.egg-info/PKG-INFO
  writing dependency_links to python_rocksdb.egg-info/dependency_links.txt
  writing requirements to python_rocksdb.egg-info/requires.txt
  writing top-level names to python_rocksdb.egg-info/top_level.txt
  reading manifest file 'python_rocksdb.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  writing manifest file 'python_rocksdb.egg-info/SOURCES.txt'
  copying rocksdb/_rocksdb.cpp -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
  copying rocksdb/_rocksdb.pyx -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
  copying rocksdb/backup.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
  copying rocksdb/cache.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
  copying rocksdb/comparator.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
  copying rocksdb/db.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
  copying rocksdb/env.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
  copying rocksdb/filter_policy.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
  copying rocksdb/iterator.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
  copying rocksdb/logger.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
  copying rocksdb/memtablerep.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
  copying rocksdb/merge_operator.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
  copying rocksdb/options.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
  copying rocksdb/slice_.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
  copying rocksdb/slice_transform.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
  copying rocksdb/snapshot.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
  copying rocksdb/status.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
  copying rocksdb/std_memory.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
  copying rocksdb/table_factory.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
  copying rocksdb/universal_compaction.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
  creating build/lib.macosx-10.14-x86_64-3.7/rocksdb/cpp
  copying rocksdb/cpp/comparator_wrapper.hpp -> build/lib.macosx-10.14-x86_64-3.7/rocksdb/cpp
  copying rocksdb/cpp/filter_policy_wrapper.hpp -> build/lib.macosx-10.14-x86_64-3.7/rocksdb/cpp
  copying rocksdb/cpp/memtable_factories.hpp -> build/lib.macosx-10.14-x86_64-3.7/rocksdb/cpp
  copying rocksdb/cpp/merge_operator_wrapper.hpp -> build/lib.macosx-10.14-x86_64-3.7/rocksdb/cpp
  copying rocksdb/cpp/slice_transform_wrapper.hpp -> build/lib.macosx-10.14-x86_64-3.7/rocksdb/cpp
  copying rocksdb/cpp/utils.hpp -> build/lib.macosx-10.14-x86_64-3.7/rocksdb/cpp
  copying rocksdb/cpp/write_batch_iter_helper.hpp -> build/lib.macosx-10.14-x86_64-3.7/rocksdb/cpp
  running build_ext
  cythoning rocksdb/_rocksdb.pyx to rocksdb/_rocksdb.cpp
  /Users/wyx/loh_workspace/loh-web/.env/lib/python3.7/site-packages/Cython/Compiler/Main.py:367: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /private/var/folders/pr/y105tp4s1w761lxysjs2pg0m0000gn/T/pip-install-s9tb5kpr/python-rocksdb/rocksdb/_rocksdb.pyx
    tree = Parsing.p_module(s, pxd, full_module_name)
  building 'rocksdb._rocksdb' extension
  creating build/temp.macosx-10.14-x86_64-3.7
  creating build/temp.macosx-10.14-x86_64-3.7/rocksdb
  clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -I/usr/local/opt/zlib/include -I/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/include/python3.7m -c rocksdb/_rocksdb.cpp -o build/temp.macosx-10.14-x86_64-3.7/rocksdb/_rocksdb.o -std=c++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing -fno-rtti -mmacosx-version-min=10.7 -stdlib=libc++
  rocksdb/_rocksdb.cpp:617:10: fatal error: 'rocksdb/slice.h' file not found
  #include "rocksdb/slice.h"
           ^~~~~~~~~~~~~~~~~
  1 error generated.
  error: command 'clang' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for python-rocksdb
  Running setup.py clean for python-rocksdb
Failed to build python-rocksdb
Installing collected packages: python-rocksdb
  Running setup.py install for python-rocksdb ... error
    ERROR: Complete output from command /Users/wyx/loh_workspace/loh-web/.env/bin/python3.7 -u -c 'import setuptools, tokenize;__file__='"'"'/private/var/folders/pr/y105tp4s1w761lxysjs2pg0m0000gn/T/pip-install-s9tb5kpr/python-rocksdb/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/pr/y105tp4s1w761lxysjs2pg0m0000gn/T/pip-record-mp_x0il7/install-record.txt --single-version-externally-managed --compile --install-headers /Users/wyx/loh_workspace/loh-web/.env/bin/../include/site/python3.7/python-rocksdb:
    ERROR: running install
    running build
    running build_py
    creating build
    creating build/lib.macosx-10.14-x86_64-3.7
    creating build/lib.macosx-10.14-x86_64-3.7/rocksdb
    copying rocksdb/interfaces.py -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
    copying rocksdb/merge_operators.py -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
    copying rocksdb/__init__.py -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
    copying rocksdb/errors.py -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
    creating build/lib.macosx-10.14-x86_64-3.7/rocksdb/tests
    copying rocksdb/tests/test_options.py -> build/lib.macosx-10.14-x86_64-3.7/rocksdb/tests
    copying rocksdb/tests/test_db.py -> build/lib.macosx-10.14-x86_64-3.7/rocksdb/tests
    copying rocksdb/tests/__init__.py -> build/lib.macosx-10.14-x86_64-3.7/rocksdb/tests
    copying rocksdb/tests/test_memtable.py -> build/lib.macosx-10.14-x86_64-3.7/rocksdb/tests
    running egg_info
    writing python_rocksdb.egg-info/PKG-INFO
    writing dependency_links to python_rocksdb.egg-info/dependency_links.txt
    writing requirements to python_rocksdb.egg-info/requires.txt
    writing top-level names to python_rocksdb.egg-info/top_level.txt
    reading manifest file 'python_rocksdb.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'python_rocksdb.egg-info/SOURCES.txt'
    copying rocksdb/_rocksdb.cpp -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
    copying rocksdb/_rocksdb.pyx -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
    copying rocksdb/backup.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
    copying rocksdb/cache.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
    copying rocksdb/comparator.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
    copying rocksdb/db.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
    copying rocksdb/env.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
    copying rocksdb/filter_policy.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
    copying rocksdb/iterator.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
    copying rocksdb/logger.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
    copying rocksdb/memtablerep.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
    copying rocksdb/merge_operator.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
    copying rocksdb/options.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
    copying rocksdb/slice_.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
    copying rocksdb/slice_transform.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
    copying rocksdb/snapshot.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
    copying rocksdb/status.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
    copying rocksdb/std_memory.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
    copying rocksdb/table_factory.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
    copying rocksdb/universal_compaction.pxd -> build/lib.macosx-10.14-x86_64-3.7/rocksdb
    creating build/lib.macosx-10.14-x86_64-3.7/rocksdb/cpp
    copying rocksdb/cpp/comparator_wrapper.hpp -> build/lib.macosx-10.14-x86_64-3.7/rocksdb/cpp
    copying rocksdb/cpp/filter_policy_wrapper.hpp -> build/lib.macosx-10.14-x86_64-3.7/rocksdb/cpp
    copying rocksdb/cpp/memtable_factories.hpp -> build/lib.macosx-10.14-x86_64-3.7/rocksdb/cpp
    copying rocksdb/cpp/merge_operator_wrapper.hpp -> build/lib.macosx-10.14-x86_64-3.7/rocksdb/cpp
    copying rocksdb/cpp/slice_transform_wrapper.hpp -> build/lib.macosx-10.14-x86_64-3.7/rocksdb/cpp
    copying rocksdb/cpp/utils.hpp -> build/lib.macosx-10.14-x86_64-3.7/rocksdb/cpp
    copying rocksdb/cpp/write_batch_iter_helper.hpp -> build/lib.macosx-10.14-x86_64-3.7/rocksdb/cpp
    running build_ext
    skipping 'rocksdb/_rocksdb.cpp' Cython extension (up-to-date)
    building 'rocksdb._rocksdb' extension
    creating build/temp.macosx-10.14-x86_64-3.7
    creating build/temp.macosx-10.14-x86_64-3.7/rocksdb
    clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -I/usr/local/opt/zlib/include -I/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/include/python3.7m -c rocksdb/_rocksdb.cpp -o build/temp.macosx-10.14-x86_64-3.7/rocksdb/_rocksdb.o -std=c++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing -fno-rtti -mmacosx-version-min=10.7 -stdlib=libc++
    rocksdb/_rocksdb.cpp:617:10: fatal error: 'rocksdb/slice.h' file not found
    #include "rocksdb/slice.h"
             ^~~~~~~~~~~~~~~~~
    1 error generated.
    error: command 'clang' failed with exit status 1
    ----------------------------------------
ERROR: Command "/Users/wyx/loh_workspace/loh-web/.env/bin/python3.7 -u -c 'import setuptools, tokenize;__file__='"'"'/private/var/folders/pr/y105tp4s1w761lxysjs2pg0m0000gn/T/pip-install-s9tb5kpr/python-rocksdb/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/pr/y105tp4s1w761lxysjs2pg0m0000gn/T/pip-record-mp_x0il7/install-record.txt --single-version-externally-managed --compile --install-headers /Users/wyx/loh_workspace/loh-web/.env/bin/../include/site/python3.7/python-rocksdb" failed with error code 1 in /private/var/folders/pr/y105tp4s1w761lxysjs2pg0m0000gn/T/pip-install-s9tb5kpr/python-rocksdb/

rocksdb dealloc error

Greetings!
With the latest master of python-rocksdb and with v5.18.3 of RocksDB I get this error on exit even when I use column families or not:

AttributeError: 'list' object has no attribute 'clear'
Exception AttributeError: "'list' object has no attribute 'clear'" in 'rocksdb._rocksdb.DB.__dealloc__' ignored

No support for 5.4.6?

Installing the latest version of rocksdb (5.4.6) and attempting to use python-rocksdb, I get:

ImportError: dlopen(/Users/Eli/.virtualenvs/ac/lib/python2.7/site-packages/rocksdb/_rocksdb.so, 2): Library not loaded: /usr/local/opt/rocksdb/lib/librocksdb.5.3.dylib
  Referenced from: /Users/Eli/.virtualenvs/ac/lib/python2.7/site-packages/rocksdb/_rocksdb.so
  Reason: image not found

Is this somehow pegged to 5.3?

Compression type XXX is not linked with the binary

in Mac OS and ubuntu 18.04, I have install rocksdb(v5.18.3) as README include snappy/lz4, and python-rocksdb-0.7.0, Mac OS works fine, but in ubuntu 18.04, error happen, code as follow:

Python 3.6.7 (default, Oct 22 2018, 11:32:17)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import rocksdb
>>> options = rocksdb.Options()
>>> options.create_if_missing = True
>>> options.compression = rocksdb.CompressionType.snappy_compression
>>> db = rocksdb.DB("test.db", options)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "rocksdb/_rocksdb.pyx", line 1636, in rocksdb._rocksdb.DB.__cinit__
  File "rocksdb/_rocksdb.pyx", line 82, in rocksdb._rocksdb.check_status
rocksdb.errors.InvalidArgument: b'Invalid argument: Compression type Snappy is not linked with the binary.'
>>>

illegal instruction

root@vmi210639:/home/rocksdb/pyrocks_test# python3.6 -c 'import rocksdb'
Illegal instruction

Default target_file_size_multiplier=1 is not recommended

This is a major issue, just something I wanted to bring attention to.

target_file_size_multiplier is set to 1 by default, however, the official RocksDB Tuning Guide mentions that it is 10 by default, and it is recommended not to change it.

Indeed, it is causing me issues with a large DB. The multiplier is 1 which means that all levels have the same size, so the number of SST files grows linearly. This means that I hit the open files limit when running my job at large-scale.

Is there any reason why the python-rocksdb defaults are not aligned with the official defaults?

crash on rocksdb load

gcc 8.3.1 on rocksdb 6.1.7

Starting program: /usr/bin/python3 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Python 3.7.3 (default, Apr  4 2019, 08:55:10) 
[GCC 8.3.1 20190329] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import rocksdb

Program received signal SIGILL, Illegal instruction.
0x00007ffff24684ca in std::__detail::_Prime_rehash_policy::_M_bkt_for_elements
    (this=0x7ffff2813260 <rocksdb::sanity_level_cf_options+32>, 
    __n=<optimized out>) at /usr/include/c++/8.3.1/bits/stl_algobase.h:219
219	    max(const _Tp& __a, const _Tp& __b)
(gdb) 

In this file

25	static const std::unordered_map<std::string, OptionsSanityCheckLevel>
26	    sanity_level_db_options {};
27	
28	// The sanity check level for column-family options
29	static const std::unordered_map<std::string, OptionsSanityCheckLevel>
30	    sanity_level_cf_options = {
31	        {"comparator", kSanityLevelLooselyCompatible},
32	        {"table_factory", kSanityLevelLooselyCompatible},
33	        {"merge_operator", kSanityLevelLooselyCompatible}};

doesn't compile after recent rocksdb update

Remove PlainTable's feature store_index_in_file (#4914) introduced yesterday breaks compilation of python-rocksdb:

rocksdb/_rocksdb.cpp:13542:25: error: 'struct rocksdb::PlainTableOptions' has no member named 'store_index_in_file'
__pyx_v_table_options.store_index_in_file = __pyx_t_8;

~ColumnFamilySet(): Assertion `last_ref' failed

I'm having trouble getting python-rocksdb running. I followed https://github.com/twmht/python-rocksdb#quick-install with the small change of git checkout v6.4.6 to get a stable rocksdb release. When a run a script that db.puts about 700k entries with a total key+value bytes of < 1GB it exits with

python3: /code/rocksdb/db/column_family.cc:1284: rocksdb::ColumnFamilySet::~ColumnFamilySet(): Assertion `last_ref' failed.
Aborted

I first ran into this problem on my laptop running gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516 and Python 3.6.8. I reproduced it in a container on a server running gcc (Ubuntu 4.8.4-2ubuntu1~14.04.4) 4.8.4 and Python 3.6.8.

Compilation fails

This is using Python 3.6 and RocksDB 5.2.1.

    writing dependency_links to python_rocksdb.egg-info/dependency_links.txt
    writing requirements to python_rocksdb.egg-info/requires.txt
    writing top-level names to python_rocksdb.egg-info/top_level.txt
    warning: manifest_maker: standard file '-c' not found
    
    reading manifest file 'python_rocksdb.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'python_rocksdb.egg-info/SOURCES.txt'
    copying rocksdb/_rocksdb.cpp -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/_rocksdb.pyx -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/backup.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/cache.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/comparator.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/db.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/env.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/filter_policy.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/iterator.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/logger.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/memtablerep.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/merge_operator.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/options.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/slice_.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/slice_transform.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/snapshot.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/status.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/std_memory.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/table_factory.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/universal_compaction.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    creating build/lib.linux-x86_64-3.6/rocksdb/cpp
    copying rocksdb/cpp/comparator_wrapper.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
    copying rocksdb/cpp/filter_policy_wrapper.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
    copying rocksdb/cpp/memtable_factories.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
    copying rocksdb/cpp/merge_operator_wrapper.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
    copying rocksdb/cpp/slice_transform_wrapper.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
    copying rocksdb/cpp/utils.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
    copying rocksdb/cpp/write_batch_iter_helper.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
    running build_ext
    building 'rocksdb._rocksdb' extension
    creating build/temp.linux-x86_64-3.6
    creating build/temp.linux-x86_64-3.6/rocksdb
    gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/python/include/python3.6m -c rocksdb/_rocksdb.cpp -o build/temp.linux-x86_64-3.6/rocksdb/_rocksdb.o -std=c++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing
    cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++
    rocksdb/_rocksdb.cpp:495:39: fatal error: utilities/merge_operators.h: No such file or directory
     #include "utilities/merge_operators.h"
                                           ^
    compilation terminated.
    error: command 'gcc' failed with exit status 1
    
    ----------------------------------------
Command "/usr/local/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-vhcbg9kt/python-rocksdb/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-htioghan-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-vhcbg9kt/python-rocksdb/

RocksDB is installed using:

apt-get install -y libsnappy-dev zlib1g-dev libbz2-dev libgflags-dev
git clone https://github.com/facebook/rocksdb.git /tmp/rocksdb
cd /tmp/rocksdb
git checkout v5.2.1
make shared_lib -j 2
make install-shared INSTALL_PATH=/usr

This used to work with pyrocksdb. Is there anything I need to change to use it with python-rocksdb?

Know item exists, but db.get(item, cfname) returns None

I am reading from a colleagues DB directory. I know his column family names, and I can successfully read the DB into Python.

db.column_families

returns the expected names of the column families.

Also, iterating over the list returned by,

db.get_live_files_metadata()

returns some expected keys for my SST files.

However, when I use one of these keys with,

db.get(key, cf_name)

the return value is None.

Any pointers / tips for what I might be doing wrong?

My end goal is to iterate over all of the keys and store them in a text file.

Windows

Great to see this project picked up and updated. Thanks!
Will it install for all platforms including Windows?

Build Fails on MacOS

Building python-rocksdb fails on recent version of MacOS. The final error you get is when linking with libstdc++:

clang: warning: libstdc++ is deprecated; move to libc++ with a minimum deployment target of OS X 10.9 [-Wdeprecated]
ld: library not found for -lstdc++
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: command 'g++' failed with exit status 1

Summary of how to work around right now:

brew install rocksdb
export MACOSX_DEPLOYMENT_TARGET=10.9
pip install python-rocksdb

The pandas folks ran into this too; here is a discussion on how to resolve the issue by passing additional args to Clang: pandas-dev/pandas#23424

No status codes on simple operations

The rocksdb cpp library returns a Status object for each operation. Unfortunately, operations such as Put or Delete don't seem to return anything when the operation fails. For example, if I do the following:

import rocksdb
# Create an entirely new database
db = rocksdb.DB("/tmp/foo", rocksdb.Options(create_if_missing=True))
result = db.delete("blah")
print(result)

result is None. No exceptions are raised.

This makes it hard to build an API that responds to failures or unexpected events. For example, I may want to take a different path if I try to delete an object which does not exist. Of course, I could try to get it first, and if that fails, then error out, but that's not very efficient.

Would it be possible to extend the API somehow to allow this?

AttributeError: module 'pyrocksdb' has no attribute 'DB'

I've tried the installation 3 times - wiped/cleaned everything related to rocksdb and started from scratch each time, but still have the same result.
This is using the instructions from here, on Ubuntu 18.04.1 LTS:
https://github.com/twmht/python-rocksdb/tree/pybind11

After a successful installation into my Python 3.6.7 virtual environment, I get this when trying to follow through the example:
`

import pyrocksdb
db = pyrocksdb.DB()
Traceback (most recent call last):

File "", line 1, in
AttributeError: module 'pyrocksdb' has no attribute 'DB'
`

I've also tried installing outside my virtual environment and with python2.7 - issues each time. Any guidance you can provide would be much appreciated.

target_file_size_multiplier on pybind11

Since @twmht said that he would welcome requests for surfacing specific options in pybind11, here's one.

In my experience, it is important to increase the default target_file_size_multiplier = 1, I usually set it to 10, which is what the official Tuning Guide recommends. If target_file_size_multiplier = 1, all files regardless of the level have the same size. This means that the number of files increases linearly, and it is rather easy to hit the default Linux open file limit when operating at large scale. Setting it to 10 ensures that the number of files only grows logarithmically.

On a separate note, I am a bit confused about the options API, particularly about the difference between Options and DBOptions, can they be used in the same context? Btw, Options is shown in duplicate in the navigation bar of the docs.

using rocksdb api open existing/old leveldb errors out

In [3]: db = rocksdb.DB("/.../Chains/SC234", rocksdb.Options(create_if_missing=True))
L0 files seqno 5173423 5182061 vs. 72057594037927935 0
Aborted (core dumped)

Did there anyone hit this error? The path was used by leveldb, I'm trying to open an existing leveldb directory with rocksdb api.

Sequence Number methods marked TODO

Hi,

I'm wondering for the reason of the following methods are TODO, any specific issue that you found with making the python bindings? I'm wondering since my project would benefit from having at least the GetLastSequenceNumber method, and if not too much of an problem I could add it.

    # TODO: SequenceNumber GetLatestSequenceNumber()
    # TODO: Status GetUpdatesSince(
              # SequenceNumber seq_number,
              # unique_ptr[TransactionLogIterator]*)

PrefixExtractor does not use readoptions

As per rocksDB API the ReadOptions.prefix_same_as_start=true has to be provided to take advantage of reduces IO scans. I don't see a way to set the options . These are the only options supported

        cdef options.ReadOptions opts
        opts.verify_checksums = py_opts['verify_checksums']
        opts.fill_cache = py_opts['fill_cache']
        if py_opts['snapshot'] is not None:
            opts.snapshot = (<Snapshot?>(py_opts['snapshot'])).ptr

        if py_opts['read_tier'] == "all":
            opts.read_tier = options.kReadAllTier
        elif py_opts['read_tier'] == 'cache':
            opts.read_tier = options.kBlockCacheTier
        else:
            raise ValueError("Invalid read_tier")

        return opts

Error installing in Ubuntu

I can not install this in Ubuntu. This is the error log.
Thanks for help.

ubuntu@machine:/$ sudo -H pip install pyrocksdb
Collecting pyrocksdb
Using cached pyrocksdb-0.4.tar.gz
Requirement already satisfied: setuptools in /usr/local/lib/python2.7/dist-packages (from pyrocksdb)
Building wheels for collected packages: pyrocksdb
Running setup.py bdist_wheel for pyrocksdb ... error
Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;file='/tmp/pip-build-RPak5Y/pyrocksdb/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" bdist_wheel -d /tmp/tmpuvfsYVpip-wheel- --python-tag cp27:
missing cimport in module 'pair': /usr/local/lib/python2.7/dist-packages/Cython/Includes/libcpp/deque.pxd
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/errors.py -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/init.py -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/interfaces.py -> build/lib.linux-x86_64-2.7/rocksdb
creating build/lib.linux-x86_64-2.7/rocksdb/tests
copying rocksdb/tests/test_options.py -> build/lib.linux-x86_64-2.7/rocksdb/tests
copying rocksdb/tests/init.py -> build/lib.linux-x86_64-2.7/rocksdb/tests
copying rocksdb/tests/test_db.py -> build/lib.linux-x86_64-2.7/rocksdb/tests
running egg_info
writing requirements to pyrocksdb.egg-info/requires.txt
writing pyrocksdb.egg-info/PKG-INFO
writing top-level names to pyrocksdb.egg-info/top_level.txt
writing dependency_links to pyrocksdb.egg-info/dependency_links.txt
reading manifest file 'pyrocksdb.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'pyrocksdb.egg-info/SOURCES.txt'
copying rocksdb/_rocksdb.cpp -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/rocksdb.pyx -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/backup.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/cache.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/comparator.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/db.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/env.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/filter_policy.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/iterator.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/logger.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/memtablerep.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/merge_operator.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/options.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/slice.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/slice_transform.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/snapshot.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/status.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/std_memory.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/table_factory.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/universal_compaction.pxd -> build/lib.linux-x86_64-2.7/rocksdb
creating build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/comparator_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/filter_policy_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/memtable_factories.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/merge_operator_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/slice_transform_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/utils.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/write_batch_iter_helper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
running build_ext
building 'rocksdb._rocksdb' extension
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/rocksdb
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c rocksdb/_rocksdb.cpp -o build/temp.linux-x86_64-2.7/rocksdb/_rocksdb.o -std=c++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
rocksdb/_rocksdb.cpp:350:27: fatal error: rocksdb/slice.h: No such file or directory
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

---

Failed building wheel for pyrocksdb
Running setup.py clean for pyrocksdb
Failed to build pyrocksdb
Installing collected packages: pyrocksdb
Running setup.py install for pyrocksdb ... error
Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;file='/tmp/pip-build-RPak5Y/pyrocksdb/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-te8FPn-record/install-record.txt --single-version-externally-managed --compile:
missing cimport in module 'pair': /usr/local/lib/python2.7/dist-packages/Cython/Includes/libcpp/deque.pxd
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/errors.py -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/init.py -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/interfaces.py -> build/lib.linux-x86_64-2.7/rocksdb
creating build/lib.linux-x86_64-2.7/rocksdb/tests
copying rocksdb/tests/test_options.py -> build/lib.linux-x86_64-2.7/rocksdb/tests
copying rocksdb/tests/init.py -> build/lib.linux-x86_64-2.7/rocksdb/tests
copying rocksdb/tests/test_db.py -> build/lib.linux-x86_64-2.7/rocksdb/tests
running egg_info
writing requirements to pyrocksdb.egg-info/requires.txt
writing pyrocksdb.egg-info/PKG-INFO
writing top-level names to pyrocksdb.egg-info/top_level.txt
writing dependency_links to pyrocksdb.egg-info/dependency_links.txt
reading manifest file 'pyrocksdb.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'pyrocksdb.egg-info/SOURCES.txt'
copying rocksdb/_rocksdb.cpp -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/rocksdb.pyx -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/backup.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/cache.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/comparator.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/db.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/env.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/filter_policy.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/iterator.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/logger.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/memtablerep.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/merge_operator.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/options.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/slice.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/slice_transform.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/snapshot.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/status.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/std_memory.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/table_factory.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/universal_compaction.pxd -> build/lib.linux-x86_64-2.7/rocksdb
creating build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/comparator_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/filter_policy_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/memtable_factories.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/merge_operator_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/slice_transform_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/utils.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/write_batch_iter_helper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
running build_ext
building 'rocksdb._rocksdb' extension
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/rocksdb
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c rocksdb/_rocksdb.cpp -o build/temp.linux-x86_64-2.7/rocksdb/_rocksdb.o -std=c++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
rocksdb/_rocksdb.cpp:350:27: fatal error: rocksdb/slice.h: No such file or directory
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

----------------------------------------

Command "/usr/bin/python -u -c "import setuptools, tokenize;file='/tmp/pip-build-RPak5Y/pyrocksdb/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-te8FPn-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-RPak5Y/pyrocksdb/

Support for transactions

It seems the bindings are missing transaction support.

python-rocksdb do not support SeekForPrev api

rocksdb support SeekForPrev from 4.13, it's a great feature, it can be a lot easier to do some date-related work, https://github.com/facebook/rocksdb/wiki/SeekForPrev .python-rocksdb need to support it

Question: Highest rocksDB version

Greetings!

I would like to know which is the highest version of rocksDB [ https://github.com/facebook/rocksdb/releases ] are compatible with the python rocksDB lib. I currently use version 5.8 which runs perfectly.

Thanks.

Compact Range in pybind11 branch

My use case is the following: create a RocksDB index as a batch process, then let multiple processes open it as read-only and use the index repeatedly.

It is a rather basic use case, but I am currently unable to implement it in the latest python-rocksdb master release because there is no reliable way to close a DB (see #58) and not all data is written to disk when the index creation is done.

Following the advice from #58, I ported this section of my code to the pybind11 branch. Now, this is far from ideal, I assume pybind11 to be rather unstable and many features are missing, but at least DB closing works.

However, I would like the index to be fast for read-only reads. As far as I understand, compaction can't happen in read-only mode and closing a DB doesn't necessarily force compaction. So, if the DB is not compacted during creation, it will be relatively slow for reads. And since such few options are exposed in the API, I am also unable to make background compaction more aggressive.

So, in essence, I would be very grateful if compact_range were exposed to the API in pybind11.

Please let me know if you have any alternative suggestions that might help me in my use case.

Missing options

Some options seem to be missing in python-rocksdb.

For example, https://github.com/facebook/rocksdb/wiki/Setup-Options-and-Basic-Tuning supports setting 'bottonmost_compression' (the compression type used for the nth level). But python-rocksdb only supports setting 'compression' (the compression type used for the first n-1 levels).

Is there any way to set these missing options?

Compiling issues

I ran into a few hiccups while trying to install and had a few questions/comments -

1. It only seems to work when compiling rocksdb using the cmake + make approach (as per your instructions.) If I try make shared_lib as the rocksdb INSTALL file suggests, and make that my lib directory, it doesn't work. Any idea what the difference is between the two methods of compiling?

2. I ended up setting the path to the rocksdb.so by including CFLAGS="-Wl,-rpath,$ROCKSDB_LIB" before pip installing python-rocksdb. That will add the path to the rocksdb lib into the RPATH of your shared library. Then you don't need to constantly set LD_LIBRARY_PATH. Even better would be a way to statically include the rocksdb.a file but I didn't have luck there.

3. It seems if I compile with gcc 4.9 it will fail with undefined symbol errors, such as undefined symbol: _ZTIN7rocksdb10ComparatorE. When I upgraded my gcc (via linuxbrew) to gcc 5.4 I was able to get rid of those types of errors. Maybe something changed with name mangling from 4 to 5 but I really don't know.

Wish I could help further. If it turns out gcc 5+ is truly needed, would be a good idea to add that to the docs.

Thanks for your work in supporting this package!

pip3 install python-rocksdb fails

elementary OS 5.0
Python 3.6.7

platon@platon-N750JV:~$ sudo pip3 install python-rocksdb
[sudo] пароль для platon:           
The directory '/home/platon/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/platon/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting python-rocksdb
  Downloading https://files.pythonhosted.org/packages/94/81/9a8e498a26ec34da9e1ae87e831468aedb4a6ba42aa9ea779b97896cfea7/python-rocksdb-0.7.0.tar.gz (219kB)
    100% |████████████████████████████████| 225kB 2.4MB/s 
Requirement already satisfied: setuptools>=25 in ./.local/lib/python3.6/site-packages (from python-rocksdb)
Installing collected packages: python-rocksdb
  Running setup.py install for python-rocksdb ... error
    Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-x3_bbfl_/python-rocksdb/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-qg74bhu7-record/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.6
    creating build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/errors.py -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/interfaces.py -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/__init__.py -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/merge_operators.py -> build/lib.linux-x86_64-3.6/rocksdb
    creating build/lib.linux-x86_64-3.6/rocksdb/tests
    copying rocksdb/tests/test_memtable.py -> build/lib.linux-x86_64-3.6/rocksdb/tests
    copying rocksdb/tests/__init__.py -> build/lib.linux-x86_64-3.6/rocksdb/tests
    copying rocksdb/tests/test_options.py -> build/lib.linux-x86_64-3.6/rocksdb/tests
    copying rocksdb/tests/test_db.py -> build/lib.linux-x86_64-3.6/rocksdb/tests
    running egg_info
    writing python_rocksdb.egg-info/PKG-INFO
    writing dependency_links to python_rocksdb.egg-info/dependency_links.txt
    writing requirements to python_rocksdb.egg-info/requires.txt
    writing top-level names to python_rocksdb.egg-info/top_level.txt
    reading manifest file 'python_rocksdb.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'python_rocksdb.egg-info/SOURCES.txt'
    copying rocksdb/_rocksdb.cpp -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/_rocksdb.pyx -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/backup.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/cache.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/comparator.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/db.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/env.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/filter_policy.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/iterator.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/logger.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/memtablerep.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/merge_operator.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/options.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/slice_.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/slice_transform.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/snapshot.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/status.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/std_memory.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/table_factory.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/universal_compaction.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    creating build/lib.linux-x86_64-3.6/rocksdb/cpp
    copying rocksdb/cpp/comparator_wrapper.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
    copying rocksdb/cpp/filter_policy_wrapper.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
    copying rocksdb/cpp/memtable_factories.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
    copying rocksdb/cpp/merge_operator_wrapper.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
    copying rocksdb/cpp/slice_transform_wrapper.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
    copying rocksdb/cpp/utils.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
    copying rocksdb/cpp/write_batch_iter_helper.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
    running build_ext
    cythoning rocksdb/_rocksdb.pyx to rocksdb/_rocksdb.cpp
    /tmp/pip-build-x3_bbfl_/python-rocksdb/.eggs/Cython-0.29.7-py3.6-linux-x86_64.egg/Cython/Compiler/Main.py:367: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/pip-build-x3_bbfl_/python-rocksdb/rocksdb/_rocksdb.pyx
      tree = Parsing.p_module(s, pxd, full_module_name)
    building 'rocksdb._rocksdb' extension
    creating build/temp.linux-x86_64-3.6
    creating build/temp.linux-x86_64-3.6/rocksdb
    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.6m -c rocksdb/_rocksdb.cpp -o build/temp.linux-x86_64-3.6/rocksdb/_rocksdb.o -std=c++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing -fno-rtti
    rocksdb/_rocksdb.cpp:617:10: fatal error: rocksdb/slice.h: Нет такого файла или каталога
     #include "rocksdb/slice.h"
              ^~~~~~~~~~~~~~~~~
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
    
    ----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-x3_bbfl_/python-rocksdb/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-qg74bhu7-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-x3_bbfl_/python-rocksdb/

Hard crash on DB.close()

I experience a hard crash when closing a DB.

I have experimented with a number of diverse settings. No matter the load or the contents of the database it always seems to happen.

This is the case both in master with Python 3 and in Python 2 with the fix from #40. I also tested with both librocksdb-dev 5.17 (from Debian stretch-backports) and the facebook/rocksdb master (>6.1.2).

At one point I got a detailed stack trace and memory map but I can't seem to recover them. I will edit this post if I can produce them later on. I do know that the error was "double free or corruption".

Closing a DB is a critical feature for my use case. I need to be able to reliably populate DBs in subprocesses and transfer the control to the master process afterwards. This is not possible unless the database is properly closed before the transfer. I have tried the del db; gc.collect() strategy from #10 and the unit tests, but, as expected, del is not deterministic and the file lock is not freed in most cases. In any case, if del run it would likely also crash in the close() method.

I believe #46 might be experiencing the same issue.

Please, let me know if this is the case for everyone or if it is an environment issue.

VectorMemtableFactory broken on 5.3.6

On 5.3.6, opts.memtable_factory = rocksdb.VectorMemtableFactory() doesn't seem to work. It gives the error: Invalid argument: Memtable doesn't concurrent writes (allow_concurrent_memtable_write). But allow_concurrent_memtable_write doesn't seem to be exposed in the API, so it can't be changed.

Will python-rocksdb 1.0 support bytes?

Hello. I have tried your pybind11 branch and found that new Blob class only supports string.
Therefore, when I try to put a binary file and get it, Python 3.6 will raise an encoding exception.
I wonder if you still want to support bytes?

import rocksdb issuse: undefined symbol: _ZTIN7rocksdb10ComparatorE

1.Collecting python-rocksdb
Requirement already satisfied: setuptools in /home/guosheng/.local/lib/python2.7/site-packages (from python-rocksdb)
Installing collected packages: python-rocksdb
Successfully installed python-rocksdb-0.6.7
2.
guosheng@guosheng:~/code/db_rock/rocksdb/pyrocks_test$ python
Python 2.7.12 (default, Nov 19 2016, 06:48:10)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import rocksdb
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/dist-packages/rocksdb/init.py", line 1, in
from ._rocksdb import *
ImportError: /usr/local/lib/python2.7/dist-packages/rocksdb/_rocksdb.so: undefined symbol: _ZTIN7rocksdb10ComparatorE

Errors when following instructions to build/install python-rocksdb on Ubuntu 16.04 LTS

Followed all directions but still getting lots of errors below on Ubuntu 16.04 LTS.

Why?

Collecting python-rocksdb
Using cached python-rocksdb-0.6.9.tar.gz
Requirement already satisfied: setuptools>=25 in ./lib/python2.7/site-packages (from python-rocksdb) (39.0.1)
Building wheels for collected packages: python-rocksdb
Running setup.py bdist_wheel for python-rocksdb: started
Running setup.py bdist_wheel for python-rocksdb: finished with status 'error'
Complete output from command /home/cs/Ws/Wb/ee/rocksdb/venv/bin/python2 -u -c "import setuptools, tokenize;file='/tmp/pip-install-X7kq1z/python-rocksdb/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" bdist_wheel -d /tmp/pip-wheel-5aKPP5 --python-tag cp27:
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/interfaces.py -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/init.py -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/errors.py -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/merge_operators.py -> build/lib.linux-x86_64-2.7/rocksdb
creating build/lib.linux-x86_64-2.7/rocksdb/tests
copying rocksdb/tests/test_memtable.py -> build/lib.linux-x86_64-2.7/rocksdb/tests
copying rocksdb/tests/init.py -> build/lib.linux-x86_64-2.7/rocksdb/tests
copying rocksdb/tests/test_options.py -> build/lib.linux-x86_64-2.7/rocksdb/tests
copying rocksdb/tests/test_db.py -> build/lib.linux-x86_64-2.7/rocksdb/tests
running egg_info
writing requirements to python_rocksdb.egg-info/requires.txt
writing python_rocksdb.egg-info/PKG-INFO
writing top-level names to python_rocksdb.egg-info/top_level.txt
writing dependency_links to python_rocksdb.egg-info/dependency_links.txt
reading manifest file 'python_rocksdb.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'python_rocksdb.egg-info/SOURCES.txt'
copying rocksdb/_rocksdb.cpp -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/rocksdb.pyx -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/backup.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/cache.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/comparator.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/db.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/env.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/filter_policy.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/iterator.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/logger.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/memtablerep.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/merge_operator.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/options.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/slice.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/slice_transform.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/snapshot.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/status.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/std_memory.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/table_factory.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/universal_compaction.pxd -> build/lib.linux-x86_64-2.7/rocksdb
creating build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/comparator_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/filter_policy_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/memtable_factories.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/merge_operator_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/slice_transform_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/utils.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/write_batch_iter_helper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
running build_ext
skipping 'rocksdb/_rocksdb.cpp' Cython extension (up-to-date)
building 'rocksdb._rocksdb' extension
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/rocksdb
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c rocksdb/_rocksdb.cpp -o build/temp.linux-x86_64-2.7/rocksdb/_rocksdb.o -std=c++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing -fno-rtti
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
rocksdb/_rocksdb.cpp: In function ‘PyObject* __pyx_pf_7rocksdb_8_rocksdb_19PyBloomFilterPolicy_4create_filter(__pyx_obj_7rocksdb_8_rocksdb_PyBloomFilterPolicy*, PyObject*)’:
rocksdb/_rocksdb.cpp:7869:105: warning: conversion to ‘int’ from ‘std::vectorrocksdb::Slice::size_type {aka long unsigned int}’ may alter its value [-Wconversion]
__pyx_t_6->CreateFilter(py_rocks::vector_data(__pyx_v_c_keys), __pyx_v_c_keys.size(), (&__pyx_v_dst));
^
rocksdb/_rocksdb.cpp: In function ‘PyObject* __Pyx_PyInt_From_rocksdb_3a__3a_CompressionType(rocksdb::CompressionType)’:
rocksdb/_rocksdb.cpp:46834:74: warning: the result of the conversion is unspecified because ‘-1’ is outside the range of type ‘rocksdb::CompressionType’ [-Wconversion]
const rocksdb::CompressionType neg_one = (rocksdb::CompressionType) -1, const_zero = (rocksdb::CompressionType) 0;
^
c++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wl,-Bsymbolic-functions -Wl,-z,relro -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/rocksdb/_rocksdb.o -lrocksdb -lsnappy -lbz2 -lz -llz4 -o build/lib.linux-x86_64-2.7/rocksdb/_rocksdb.so
/usr/bin/ld: cannot find -llz4
collect2: error: ld returned 1 exit status
error: command 'c++' failed with exit status 1

---

Running setup.py clean for python-rocksdb
Failed to build python-rocksdb
Installing collected packages: python-rocksdb
Running setup.py install for python-rocksdb: started
Running setup.py install for python-rocksdb: finished with status 'error'
Complete output from command /home/cs/Ws/Wb/ee/rocksdb/venv/bin/python2 -u -c "import setuptools, tokenize;file='/tmp/pip-install-X7kq1z/python-rocksdb/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-record-vYCQI_/install-record.txt --single-version-externally-managed --compile --install-headers /home/cs/Ws/Wb/ee/rocksdb/venv/include/site/python2.7/python-rocksdb:
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/interfaces.py -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/init.py -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/errors.py -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/merge_operators.py -> build/lib.linux-x86_64-2.7/rocksdb
creating build/lib.linux-x86_64-2.7/rocksdb/tests
copying rocksdb/tests/test_memtable.py -> build/lib.linux-x86_64-2.7/rocksdb/tests
copying rocksdb/tests/init.py -> build/lib.linux-x86_64-2.7/rocksdb/tests
copying rocksdb/tests/test_options.py -> build/lib.linux-x86_64-2.7/rocksdb/tests
copying rocksdb/tests/test_db.py -> build/lib.linux-x86_64-2.7/rocksdb/tests
running egg_info
writing requirements to python_rocksdb.egg-info/requires.txt
writing python_rocksdb.egg-info/PKG-INFO
writing top-level names to python_rocksdb.egg-info/top_level.txt
writing dependency_links to python_rocksdb.egg-info/dependency_links.txt
reading manifest file 'python_rocksdb.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'python_rocksdb.egg-info/SOURCES.txt'
copying rocksdb/_rocksdb.cpp -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/rocksdb.pyx -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/backup.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/cache.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/comparator.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/db.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/env.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/filter_policy.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/iterator.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/logger.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/memtablerep.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/merge_operator.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/options.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/slice.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/slice_transform.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/snapshot.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/status.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/std_memory.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/table_factory.pxd -> build/lib.linux-x86_64-2.7/rocksdb
copying rocksdb/universal_compaction.pxd -> build/lib.linux-x86_64-2.7/rocksdb
creating build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/comparator_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/filter_policy_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/memtable_factories.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/merge_operator_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/slice_transform_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/utils.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
copying rocksdb/cpp/write_batch_iter_helper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp
running build_ext
skipping 'rocksdb/_rocksdb.cpp' Cython extension (up-to-date)
building 'rocksdb._rocksdb' extension
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/rocksdb
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c rocksdb/_rocksdb.cpp -o build/temp.linux-x86_64-2.7/rocksdb/_rocksdb.o -std=c++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing -fno-rtti
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
rocksdb/_rocksdb.cpp: In function ‘PyObject* __pyx_pf_7rocksdb_8_rocksdb_19PyBloomFilterPolicy_4create_filter(__pyx_obj_7rocksdb_8_rocksdb_PyBloomFilterPolicy*, PyObject*)’:
rocksdb/_rocksdb.cpp:7869:105: warning: conversion to ‘int’ from ‘std::vectorrocksdb::Slice::size_type {aka long unsigned int}’ may alter its value [-Wconversion]
__pyx_t_6->CreateFilter(py_rocks::vector_data(__pyx_v_c_keys), __pyx_v_c_keys.size(), (&__pyx_v_dst));
^
rocksdb/_rocksdb.cpp: In function ‘PyObject* __Pyx_PyInt_From_rocksdb_3a__3a_CompressionType(rocksdb::CompressionType)’:
rocksdb/_rocksdb.cpp:46834:74: warning: the result of the conversion is unspecified because ‘-1’ is outside the range of type ‘rocksdb::CompressionType’ [-Wconversion]
const rocksdb::CompressionType neg_one = (rocksdb::CompressionType) -1, const_zero = (rocksdb::CompressionType) 0;
^
c++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wl,-Bsymbolic-functions -Wl,-z,relro -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/rocksdb/_rocksdb.o -lrocksdb -lsnappy -lbz2 -lz -llz4 -o build/lib.linux-x86_64-2.7/rocksdb/_rocksdb.so
/usr/bin/ld: cannot find -llz4
collect2: error: ld returned 1 exit status
error: command 'c++' failed with exit status 1

----------------------------------------

Is BlockBasedTableFactory exposed in pybind11?

In options.cpp I can see that there is a table_factory argument in Options and ColumnFamilyOptions. There's also LRUCacheOptions and the cache.cpp file and the usual way to set up a block cache is through a table factory. But I cannot find any other mentions of table factories in the codebase.

I mainly want to be able to disable the block cache, or control its size, is there a way to do that currently in pybind11?

Iterator memory leak

        self.db = rocksdb.DB("test.db", rocksdb.Options(max_open_files=100))
        it = self.db.iteritems()
        it.seek_to_first()
        while True:
            next(it)

This will eventually use up all memory (for a large DB).