Memory Leak When Using Rust

[Bruce-Hopkins]

Memory Leak When Using Rust

[amaanq]

yeah there's no memory leak here, valgrind (with and without massif) report nothing and I don't think DRS is exactly something you can go off of (IF that's what you're looking at, sorry, I was noticing this staying high with DRS but not RSS), since it seems to be very different from what massif and kde's system monitor report to me (these 2 are similar, showing a drop to ~800kb-1.2mb after the calls to free)

I'd like to guess that it's the system either reserving that memory for further reuse for that process, or it could be some other OS-level oddity, not sure

[Bruce-Hopkins]

I managed to track down what the issue I'm seeing is. @amaanq You are right that this is not a memory leak. It seems that the memory is being free, but not released to the OS, which is why memory profilers do not see an issue.

If we change the program to this:

use tree_sitter::{Parser};
use libc;

fn main() {
    let source_code = "fn main() {}\n".repeat(100000).to_string();
    let mut parser = Parser::new();
    parser
        .set_language(tree_sitter_rust::language())
        .expect("Error loading Rust grammar");
    let tree = parser.parse(source_code, None).unwrap();

    drop(tree);
    drop(parser);

    // Release free memory in the heap.
    unsafe {libc::malloc_trim(0);}

    loop {
        // Sleep every 10 miliseconds to prevent high CPU usage
        let ten_millis = std::time::Duration::from_millis(10);
        std::thread::sleep(ten_millis);
        
    }
}

We call malloc_trim to release the memory within the heap. Although far from an ideal solution due to the usage of unsafe code, almost all of the memory gets released to the OS and my System Monitor shows very low memory.

[amaanq]

The default allocator is libc's for tree-sitter, but you can set your own allocator with ts_set_allocator if you prefer something that holds onto memory for reusage less aggressively. You can also avoid pulling in libc for malloc_trim by playing with the env var M_TRIM_THRESHOLD or MALLOC_TRIM_THRESHOLD_ - https://www.gnu.org/software/libc/manual/html_node/Malloc-Tunable-Parameters.html