tushushu/murmurust

Python binding of MurmurHash3 Rust implementation.

murmurust

What

Python binding of MurmurHash3 Rust implementation.

Requirements

• Python: 3.8+
• OS: Linux, MacOS and Windows

Installation

Run pip install mmr3

Benchmark

mmr3 is faster than mmh3 on average, which is a popular murmurhash3 library written in C/C++ and Python. For the details, please refer to benchmark.md.

Item	XS	S	M	L	XL	Average	Faster
Hash32	1.9x	1.8x	1.8x	1.3x	1.0x	1.6x	Y
Hash128	1.9x	1.9x	1.8x	1.5x	1.0x	1.6x	Y

Examples

>>> import mmr3
# By default, seed=0, return unsigned int.
>>> mmr3.hash32('foo')
4138058784

# When seed = 100.
>>> mmr3.hash32('foo', seed=100)
3757588558

# Return signed int.
>>> mmr3.hash32('foo', signed=True)
-156908512

# hash128 for x64 architecture
>>> mmr3.hash128_x64('foo', seed=20, signed=True)
-114440907836743398687870327469607863688

murmurust's Issues

Improve the benchmark.

By using SIMD, inline, unsafe or any technique to improve the benchmark of this lib.

Compare the benchmark with `mmh3`

The repo link of mmh3 is https://github.com/hajimes/mmh3, and it's a similar project to us, which is implemented by C++ and Python.

Support Numpy array.

By calling Numpy C-API, implement hash functions for Numpy array. For example:

import numpy as np
import mmr3

arr = np.array(['foo', 'bar', 'baz'], dtype=object)
mmr3.hash32(arr)

Here is the Rust bindings for Numpy C-API: https://github.com/PyO3/rust-numpy

Support signed int as return value.

User can decide whether use signed int(i32) or unsigned int (u32) as hash32 function return type. For example:

>>> import mmh3
>>> mmh3.hash('foo', signed=False)
4138058784
>>> mmh3.hash('foo', signed=True)
-156908512
>>> mmh3.hash('foo')
-156908512

Test the benchmark of mmr3 1.3.0

Disable bound check.

Disable the array bound check by using unsafe method or iterators. Related to #6

Benchmark report in `readme.md`

Provide library description in PYPI

Can refer to project such as hyperjson.

Rename the function names.

Rename mmh3_32 as hash32 to make it convenient to use.

Use mmh3 hash functions for unit tests.

• The mmh3 repo link is https://github.com/hajimes/mmh3
• Unit tests are in mmr3/tests path.

Recover clippy test.

The clippy test shows the errors below, which is not expected:

error: deref on an immutable reference
  --> src/lib.rs:80:34
   |
80 | fn hash128_x[64](https://github.com/tushushu/murmurust/runs/7928640580?check_suite_focus=true#step:6:65)(_py: Python, key: &str, seed: u32, signed: bool) -> Py<PyAny> {
   |                                  ^ help: if you would like to reborrow, try removing `&*`: `&`
   |
   = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#borrow_deref_ref

This is going to be fixed by PyO3/pyo3#2555. Currently, I have to disable the clippy errors, and when PYO3 0.17.0 released, we need change it back.

Github workflow to publish this repo.

Please refer to my another repo https://github.com/Rust-Data-Science/ulist/blob/main/.github/workflows/publish.yml

Make parameter `seed` optional.

The default seed of hash32 function is zero, for example:

>>> import mmr3
>>> mmr3.hash32('foo')
4138058784
>>> mmr3.hash32('foo', 0)
4138058784

Implement `hash` function to wrap all the other functions

hash128_x64 function uses default parameters

Add badges in `Readmd.md`

Add badges as below:

• PYPI
• Publish
• Downloads

Please refer to my another repo. https://github.com/Rust-Data-Science/ulist

List the features of mmr3 in `Readme.md`

Directly change array[u8] to array[u32] to improve benchmark.

Write some examples in `Readme.md`

Test the benchmark of `mmr3` 1.2.0

Run the github workflow - benchmark, and paste the report in readme.md.

Implement hash128 function.

Need to implement hash128 function by Rust, and provide a Python wrapper. For the details, please refer to how we implement hash32 in this repo.