memcached/mc-crusher

ENSIGN CRUSHER

Memcached

Memcached is a high performance multithreaded event-based key/value cache store intended to be used in a distributed system.

See: https://memcached.org/about

A fun story explaining usage: https://memcached.org/tutorial

If you're having trouble, try the wiki: https://memcached.org/wiki

If you're trying to troubleshoot odd behavior or timeouts, see: https://memcached.org/timeouts

https://memcached.org/ is a good resource in general. Please use the mailing list to ask questions, github issues aren't seen by everyone!

Dependencies

• libevent - https://www.monkey.org/~provos/libevent/ (libevent-dev)
• libseccomp (optional, experimental, linux) - enables process restrictions for better security. Tested only on x86-64 architectures.
• openssl (optional) - enables TLS support. need relatively up to date version. pkg-config is needed to find openssl dependencies (such as -lz).

Environment

Be warned that the -k (mlockall) option to memcached might be dangerous when using a large cache. Just make sure the memcached machines don't swap. memcached does non-blocking network I/O, but not disk. (it should never go to disk, or you've lost the whole point of it)

Build status

See https://build.memcached.org/ for multi-platform regression testing status.

Bug reports

Feel free to use the issue tracker on github.

If you are reporting a security bug please contact a maintainer privately. We follow responsible disclosure: we handle reports privately, prepare a patch, allow notifications to vendor lists. Then we push a fix release and your bug can be posted publicly with credit in our release notes and commit history.

Website

• https://www.memcached.org

Contributing

See https://github.com/memcached/memcached/wiki/DevelopmentRepos

mc-crusher's Issues

Crash(SEGV) when running with conf/huge_mget or conf/set_big_values

Description
Crash(SEGV) occurs when running mc-crusher with --conf ./conf/huge_mget or --conf ./conf/set_big_values. The rest of conf/* are fine.

Steps to Reproduce

1. Build and run official memcached-1.6.1. configure options used in my env is --disable-extstore --disable-tls --disable-seccomp --disable-sasl --disable-sasl-pwdb --disable-coverage --disable-docs
2. Execute ./mc-crusher --conf ./conf/huge_mget --ip MEMCACHED_IP. Result: SEGV
3. Execute ./mc-crusher --conf ./conf/set_big_values --ip MEMCACHED_IP. Result: SEGV

System Information

• OS/Distro: Ubuntu
• Version of OS/distro: 19.10
• Version of memcached: 1.6.1
• Hardware detail: VirtualBox

Details
Valgrind was executed after normal execution crash and below are the logs:

==3335== Memcheck, a memory error detector
==3335== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==3335== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==3335== Command: ./mc-crusher --conf ./conf/set_big_values --ip 192.168.56.102
==3335==
ip address default: 192.168.56.102
id 0 for key send value ascii_set
id 1 for key recv value blind_read
id 5 for key conns value 1
id 8 for key key_prefix value foo
id 12 for key value_size value 1047552
id 26 for key key_prealloc value 0
id 20 for key key_count value 4000
done initializing
==3335== Thread 2:
==3335== Invalid write of size 2
==3335== at 0x4841B33: memmove (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3335== by 0x10B9A1: ascii_write_flat_to_client (mc-crusher.c:487)
==3335== by 0x10BB53: run_write (mc-crusher.c:543)
==3335== by 0x10BD33: client_handler (mc-crusher.c:583)
==3335== by 0x4875A10: ??? (in /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6.0.2)
==3335== by 0x487633E: event_base_loop (in /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6.0.2)
==3335== by 0x10D346: thread_runner (mc-crusher.c:1063)
==3335== by 0x4C00668: start_thread (pthread_create.c:479)
==3335== by 0x4D3C322: clone (clone.S:95)
==3335== Address 0x4f2fc78 is 0 bytes after a block of size 66,952 alloc'd
==3335== at 0x483A7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3335== by 0x10BE9D: new_connection (mc-crusher.c:636)
==3335== by 0x10D29A: start_template (mc-crusher.c:1048)
==3335== by 0x10D1D4: parse_config_line (mc-crusher.c:1032)
==3335== by 0x10D9DB: main (mc-crusher.c:1181)
==3335==

==3243== Memcheck, a memory error detector
==3243== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==3243== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==3243== Command: ./mc-crusher --conf ./conf/huge_mget --ip 192.168.56.102
==3243==
ip address default: 192.168.56.102
id 0 for key send value ascii_mget
id 1 for key recv value blind_read
id 5 for key conns value 1
id 15 for key mget_count value 500000
id 8 for key key_prefix value foobar
id 26 for key key_prealloc value 1
done initializing==3243== Thread 2:
==3243== Invalid write of size 2
==3243== at 0x4841B33: memmove (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3243== by 0x10B3AB: ascii_mget_format (mc-crusher.c:393)
==3243== by 0x10B461: ascii_write_flat_mget_to_client (mc-crusher.c:406)
==3243== by 0x10BB53: run_write (mc-crusher.c:543)
==3243== by 0x10BD33: client_handler (mc-crusher.c:583)
==3243== by 0x4875A10: ??? (in /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6.0.2)
==3243== by 0x487633E: event_base_loop (in /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6.0.2)
==3243== by 0x10D346: thread_runner (mc-crusher.c:1063)
==3243== by 0x4C00668: start_thread (pthread_create.c:479)
==3243== by 0x4D3C322: clone (clone.S:95)
==3243== Address 0x4f2fc78 is 0 bytes after a block of size 66,952 alloc'd
==3243== at 0x483A7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3243== by 0x10BE9D: new_connection (mc-crusher.c:636)
==3243== by 0x10D29A: start_template (mc-crusher.c:1048)
==3243== by 0x10D1D4: parse_config_line (mc-crusher.c:1032)
==3243== by 0x10D9DB: main (mc-crusher.c:1181)
==3243==
==3243== Invalid write of size 2
==3243== at 0x4841B33: memmove (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3243== by 0x10DC5B: out2 (itoa_ljust.c:74)
==3243== by 0x10DE02: itoa (itoa_ljust.c:100)
==3243== by 0x10DF40: itoa_u32 (itoa_ljust.c:116)
==3243== by 0x10DFCF: itoa_u64 (itoa_ljust.c:132)
==3243== by 0x10B3D6: ascii_mget_format (mc-crusher.c:394)
==3243== by 0x10B461: ascii_write_flat_mget_to_client (mc-crusher.c:406)
==3243== by 0x10BB53: run_write (mc-crusher.c:543)
==3243== by 0x10BD33: client_handler (mc-crusher.c:583)
==3243== by 0x4875A10: ??? (in /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6.0.2)
==3243== by 0x487633E: event_base_loop (in /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6.0.2)
==3243== by 0x10D346: thread_runner (mc-crusher.c:1063)
==3243== Address 0x4f2fc7a is 2 bytes after a block of size 66,952 alloc'd
==3243== at 0x483A7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3243== by 0x10BE9D: new_connection (mc-crusher.c:636)
==3243== by 0x10D29A: start_template (mc-crusher.c:1048)
==3243== by 0x10D1D4: parse_config_line (mc-crusher.c:1032)
==3243== by 0x10D9DB: main (mc-crusher.c:1181)
==3243==
==3243== Invalid write of size 1
==3243== at 0x4841A48: memmove (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3243== by 0x10DC5B: out2 (itoa_ljust.c:74)
==3243== by 0x10DE26: itoa (itoa_ljust.c:102)
==3243== by 0x10DF40: itoa_u32 (itoa_ljust.c:116)
==3243== by 0x10DFCF: itoa_u64 (itoa_ljust.c:132)
==3243== by 0x10B3D6: ascii_mget_format (mc-crusher.c:394)
==3243== by 0x10B461: ascii_write_flat_mget_to_client (mc-crusher.c:406)
==3243== by 0x10BB53: run_write (mc-crusher.c:543)
==3243== by 0x10BD33: client_handler (mc-crusher.c:583)
==3243== by 0x4875A10: ??? (in /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6.0.2)
==3243== by 0x487633E: event_base_loop (in /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6.0.2)
==3243== by 0x10D346: thread_runner (mc-crusher.c:1063)
==3243== Address 0x4f2fc7c is 4 bytes after a block of size 66,952 alloc'd
==3243== at 0x483A7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3243== by 0x10BE9D: new_connection (mc-crusher.c:636)
==3243== by 0x10D29A: start_template (mc-crusher.c:1048)
==3243== by 0x10D1D4: parse_config_line (mc-crusher.c:1032)
==3243== by 0x10D9DB: main (mc-crusher.c:1181)
==3243==
==3243== Invalid write of size 1
==3243== at 0x10DE2F: itoa (itoa_ljust.c:105)
==3243== by 0x10DF40: itoa_u32 (itoa_ljust.c:116)
==3243== by 0x10DFCF: itoa_u64 (itoa_ljust.c:132)
==3243== by 0x10B3D6: ascii_mget_format (mc-crusher.c:394)
==3243== by 0x10B461: ascii_write_flat_mget_to_client (mc-crusher.c:406)
==3243== by 0x10BB53: run_write (mc-crusher.c:543)
==3243== by 0x10BD33: client_handler (mc-crusher.c:583)
==3243== by 0x4875A10: ??? (in /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6.0.2)
==3243== by 0x487633E: event_base_loop (in /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6.0.2)
==3243== by 0x10D346: thread_runner (mc-crusher.c:1063)
==3243== by 0x4C00668: start_thread (pthread_create.c:479)
==3243== by 0x4D3C322: clone (clone.S:95)
==3243== Address 0x4f2fc7e is 6 bytes after a block of size 66,952 alloc'd
==3243== at 0x483A7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3243== by 0x10BE9D: new_connection (mc-crusher.c:636)
==3243== by 0x10D29A: start_template (mc-crusher.c:1048)
==3243== by 0x10D1D4: parse_config_line (mc-crusher.c:1032)
==3243== by 0x10D9DB: main (mc-crusher.c:1181)
==3243==
==3243== Invalid write of size 1
==3243== at 0x10B3DF: ascii_mget_format (mc-crusher.c:395)
==3243== by 0x10B461: ascii_write_flat_mget_to_client (mc-crusher.c:406)
==3243== by 0x10BB53: run_write (mc-crusher.c:543)
==3243== by 0x10BD33: client_handler (mc-crusher.c:583)
==3243== by 0x4875A10: ??? (in /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6.0.2)
==3243== by 0x487633E: event_base_loop (in /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6.0.2)
==3243== by 0x10D346: thread_runner (mc-crusher.c:1063)
==3243== by 0x4C00668: start_thread (pthread_create.c:479)
==3243== by 0x4D3C322: clone (clone.S:95)
==3243== Address 0x4f2fc7e is 6 bytes after a block of size 66,952 alloc'd
==3243== at 0x483A7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3243== by 0x10BE9D: new_connection (mc-crusher.c:636)
==3243== by 0x10D29A: start_template (mc-crusher.c:1048)
==3243== by 0x10D1D4: parse_config_line (mc-crusher.c:1032)
==3243== by 0x10D9DB: main (mc-crusher.c:1181)
==3243==
==3243== Invalid write of size 1
==3243== at 0x4841B63: memmove (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3243== by 0x10B3AB: ascii_mget_format (mc-crusher.c:393)
==3243== by 0x10B461: ascii_write_flat_mget_to_client (mc-crusher.c:406)
==3243== by 0x10BB53: run_write (mc-crusher.c:543)
==3243== by 0x10BD33: client_handler (mc-crusher.c:583)
==3243== by 0x4875A10: ??? (in /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6.0.2)
==3243== by 0x487633E: event_base_loop (in /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6.0.2)
==3243== by 0x10D346: thread_runner (mc-crusher.c:1063)
==3243== by 0x4C00668: start_thread (pthread_create.c:479)
==3243== by 0x4D3C322: clone (clone.S:95)
==3243== Address 0x4f2fc7f is 7 bytes after a block of size 66,952 alloc'd
==3243== at 0x483A7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3243== by 0x10BE9D: new_connection (mc-crusher.c:636)
==3243== by 0x10D29A: start_template (mc-crusher.c:1048)
==3243== by 0x10D1D4: parse_config_line (mc-crusher.c:1032)
==3243== by 0x10D9DB: main (mc-crusher.c:1181)
==3243==
==3243== Invalid write of size 1
==3243== at 0x4841B63: memmove (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3243== by 0x10DC5B: out2 (itoa_ljust.c:74)
==3243== by 0x10DE02: itoa (itoa_ljust.c:100)
==3243== by 0x10DF40: itoa_u32 (itoa_ljust.c:116)
==3243== by 0x10DFCF: itoa_u64 (itoa_ljust.c:132)
==3243== by 0x10B3D6: ascii_mget_format (mc-crusher.c:394)
==3243== by 0x10B461: ascii_write_flat_mget_to_client (mc-crusher.c:406)
==3243== by 0x10BB53: run_write (mc-crusher.c:543)
==3243== by 0x10BD33: client_handler (mc-crusher.c:583)
==3243== by 0x4875A10: ??? (in /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6.0.2)
==3243== by 0x487633E: event_base_loop (in /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6.0.2)
==3243== by 0x10D346: thread_runner (mc-crusher.c:1063)
==3243== Address 0x4f2fc85 is 13 bytes after a block of size 66,952 alloc'd
==3243== at 0x483A7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3243== by 0x10BE9D: new_connection (mc-crusher.c:636)
==3243== by 0x10D29A: start_template (mc-crusher.c:1048)
==3243== by 0x10D1D4: parse_config_line (mc-crusher.c:1032)
==3243== by 0x10D9DB: main (mc-crusher.c:1181)
==3243==
==3243== Invalid write of size 1
==3243== at 0x4841B63: memmove (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3243== by 0x10DC5B: out2 (itoa_ljust.c:74)
==3243== by 0x10DE26: itoa (itoa_ljust.c:102)
==3243== by 0x10DF40: itoa_u32 (itoa_ljust.c:116)
==3243== by 0x10DFCF: itoa_u64 (itoa_ljust.c:132)
==3243== by 0x10B3D6: ascii_mget_format (mc-crusher.c:394)
==3243== by 0x10B461: ascii_write_flat_mget_to_client (mc-crusher.c:406)
==3243== by 0x10BB53: run_write (mc-crusher.c:543)
==3243== by 0x10BD33: client_handler (mc-crusher.c:583)
==3243== by 0x4875A10: ??? (in /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6.0.2)
==3243== by 0x487633E: event_base_loop (in /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6.0.2)
==3243== by 0x10D346: thread_runner (mc-crusher.c:1063)
==3243== Address 0x4f2fc87 is 15 bytes after a block of size 66,952 alloc'd
==3243== at 0x483A7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3243== by 0x10BE9D: new_connection (mc-crusher.c:636)
==3243== by 0x10D29A: start_template (mc-crusher.c:1048)
==3243== by 0x10D1D4: parse_config_line (mc-crusher.c:1032)
==3243== by 0x10D9DB: main (mc-crusher.c:1181)
==3243==
==3243== Invalid write of size 2
==3243== at 0x4841B33: memmove (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3243== by 0x10DC5B: out2 (itoa_ljust.c:74)
==3243== by 0x10DE26: itoa (itoa_ljust.c:102)
==3243== by 0x10DF40: itoa_u32 (itoa_ljust.c:116)
==3243== by 0x10DFCF: itoa_u64 (itoa_ljust.c:132)
==3243== by 0x10B3D6: ascii_mget_format (mc-crusher.c:394)
==3243== by 0x10B461: ascii_write_flat_mget_to_client (mc-crusher.c:406)
==3243== by 0x10BB53: run_write (mc-crusher.c:543)
==3243== by 0x10BD33: client_handler (mc-crusher.c:583)
==3243== by 0x4875A10: ??? (in /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6.0.2)
==3243== by 0x487633E: event_base_loop (in /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6.0.2)
==3243== by 0x10D346: thread_runner (mc-crusher.c:1063)
==3243== Address 0x4f2fc92 is 18 bytes after a block of size 66,960 in arena "client"
==3243==
==3243== Invalid write of size 1
==3243== at 0x4841A48: memmove (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3243== by 0x10DC5B: out2 (itoa_ljust.c:74)
==3243== by 0x10DE02: itoa (itoa_ljust.c:100)
==3243== by 0x10DF40: itoa_u32 (itoa_ljust.c:116)
==3243== by 0x10DFCF: itoa_u64 (itoa_ljust.c:132)
==3243== by 0x10B3D6: ascii_mget_format (mc-crusher.c:394)
==3243== by 0x10B461: ascii_write_flat_mget_to_client (mc-crusher.c:406)
==3243== by 0x10BB53: run_write (mc-crusher.c:543)
==3243== by 0x10BD33: client_handler (mc-crusher.c:583)
==3243== by 0x4875A10: ??? (in /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6.0.2)
==3243== by 0x487633E: event_base_loop (in /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6.0.2)
==3243== by 0x10D346: thread_runner (mc-crusher.c:1063)
==3243== Address 0x4f2ff92 is 2 bytes after a block of size 16 alloc'd
==3243== at 0x483CD99: calloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==3243== by 0x4878F02: evmap_io_add_ (in /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6.0.2)
==3243== by 0x48740CD: event_add_nolock_ (in /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6.0.2)
==3243== by 0x48745A9: event_add (in /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6.0.2)
==3243== by 0x10C4E2: new_connection (mc-crusher.c:767)
==3243== by 0x10D29A: start_template (mc-crusher.c:1048)
==3243== by 0x10D1D4: parse_config_line (mc-crusher.c:1032)
==3243== by 0x10D9DB: main (mc-crusher.c:1181)
==3243==

EXPIRE is not fully implemented

The enum is there, but the implementation is missing

Hi, The error when I ran the benchmark was as follows.Please help to see where the problem is.

../test-suites/test-nvdimm mm

numactl --cpunodebind=0 --membind=0 /root/memcached/memcached-1.5.12/memcached -u root -l 127.0.0.1 -t 32 -c 32000 -m 600000 -o no_lru_crawler -b 4096 -s /tmp/memcached.sock -l 127.0.0.1 -p 11211
memc started (main)
Can't use an undefined value as a symbol reference at Memcached/McCrusher.pm line 105.
Signal handled: Terminated.

command as follows:
$server_args = '-u root -l 127.0.0.1 -t 32 -c 32000 -m 600000 -o no_lru_crawler -b 4096 -s /tmp/memcached.sock';

How to build the project?

I want to build the project and run it. But I have no idea how to build it. In readme it says I should install libevent and headers but I have no what headers is.

And then I need to exec ./compile but still, I couldn't find that binary executable file and I couldn't run make, which prompts me the following error.
mc-crusher.c:59:19: error: ‘NI_MAXHOST’ undeclared here (not in a function) char host_default[NI_MAXHOST] = "127.0.0.1";

Is this because I haven't installed the headers?