expose raw hash APIs

[moka-rs]

expose raw hash APIs

[VladimirBramstedt]

ah, i missed that. thanks a lot :)

[tatsuya6502]

Hi. Thank you for using Moka.

It would not be necessary to expose these APIs. You can write your oun hasher, which simply pass-through the pre-calculated hash value. And then you can give the hasher to the cache. Here is an example:

use std::{
    hash::{BuildHasherDefault, Hash, Hasher},
    sync::Arc,
};

//
// Define a struct for the key, which holds the key and its hash value.
//

#[derive(Clone, PartialEq, Eq)]
pub struct MyKey {
    _key: Arc<Box<[u8]>>,
    hash_value: u64,
}

impl MyKey {
    pub fn new(key: Box<[u8]>, hash_value: u64) -> Self {
        Self {
            _key: Arc::new(key),
            hash_value,
        }
    }
}

// Implement Hash. We will only use the pre-calculated hash value.
impl Hash for MyKey {
    fn hash<H: Hasher>(&self, state: &mut H) {
        self.hash_value.hash(state);
    }
}

//
// Create a Hasher, which just uses a single u64 value as the final hash.
//

#[derive(Default)]
struct PassthroughHasher(u64);

impl Hasher for PassthroughHasher {
    fn write_u64(&mut self, i: u64) {
        self.0 = i;
    }

    fn write(&mut self, _bytes: &[u8]) {
        unimplemented!()
    }

    fn finish(&self) -> u64 {
        self.0
    }
}

// Define a BuildHasher using our Hasher. We will give this to our cache.
type PassthroughBuildHasher = BuildHasherDefault<PassthroughHasher>;

fn main() {
    let cache = moka::sync::Cache::builder()
        .max_capacity(100)
        // Give our build hasher to this cache.
        .build_with_hasher(PassthroughBuildHasher::default());

    let key1 = MyKey::new(b"key1".to_vec().into_boxed_slice(), 0);
    let key2 = MyKey::new(b"key2".to_vec().into_boxed_slice(), 1);

    cache.insert(key1.clone(), "value1");
    cache.insert(key2.clone(), "value2");

    assert_eq!(cache.get(&key1), Some("value1"));
    assert_eq!(cache.get(&key2), Some("value2"));
}

Will this trick work for you?

[VladimirBramstedt]

it would, but i would think it is perhaps more ergonomic to provide something similar to the raw_entry API from hashbrown https://docs.rs/hashbrown/latest/hashbrown/raw/struct.RawTable.html#method.insert, etc, especially if these functions are already implemented.
its a bit a question of API, i guess.

[tatsuya6502]

Unfortunately, exposing sync::Cache's get_with_hash and insert_with_hash does not solve your problem.

Like many other hash tables, Moka cache's internal hash table re-calculates hash values of existing keys when it resizes (reallocates) the storage area for growth. This operation is called rehashing. So even though, you use the pre-calculated hash when inserting and retrieving an entry, the cache will still re-calculate hashes from the keys when the cache grows. If you use a different hash function for pre-calculating the hash to the hash function that the cache is given at the build time, it will break; get_with_hash will no longer be able to locate the key after growing.

On the other hand, the example code above with PassthroughHasher ensures that the hash is never re-calculated from the key, at insertion, retrieval and rehashing times.

[tatsuya6502]

Here is the rehash method in Moka's internal hash table:

moka/src/cht/map/bucket.rs

Line 396 in 915d9b0

pub(crate) fn rehash<H>(

and this is the line in rehash that re-calculates hash from key.

moka/src/cht/map/bucket.rs

Line 439 in 915d9b0

let hash = hash(build_hasher, key);