segfault error in gemini inside the docker instance

[temichus]

segfault error in gemini inside the docker instance

[roydahan]

@dkropachev any chance you can take a look at the crashes we have in Gemini?

[fruch]

Happens again this week, looks like we are leaking memory :

Installation details

Kernel Version: 5.19.0-1022-aws
Scylla version (or git commit hash): 5.3.0~dev-20230415.1da02706ddb8 with build-id f7ac5cd90e63ace5065c583d6d1d9c381f39b5c2

Cluster size: 3 nodes (i3.4xlarge)

Scylla Nodes used in this run:

• gemini-1tb-10h-master-oracle-db-node-08087b94-1 (107.21.57.78 | 10.12.10.56) (shards: 14)
• gemini-1tb-10h-master-db-node-08087b94-5 (54.242.182.146 | 10.12.10.38) (shards: 14)
• gemini-1tb-10h-master-db-node-08087b94-4 (54.91.135.97 | 10.12.8.140) (shards: 14)
• gemini-1tb-10h-master-db-node-08087b94-3 (54.226.251.100 | 10.12.10.164) (shards: 14)
• gemini-1tb-10h-master-db-node-08087b94-2 (54.91.159.223 | 10.12.11.63) (shards: 14)
• gemini-1tb-10h-master-db-node-08087b94-1 (67.202.18.69 | 10.12.11.108) (shards: 14)

OS / Image: ami-0501eb17c8c79b6d2 (aws: us-east-1)

Test: gemini-1tb-10h
Test id: 08087b94-c377-4cea-8788-4cb54d2f5263
Test name: scylla-master/gemini-/gemini-1tb-10h
Test config file(s):

• gemini-1tb-10h.yaml

Logs and commands

• Restore Monitor Stack command: $ hydra investigate show-monitor 08087b94-c377-4cea-8788-4cb54d2f5263
• Restore monitor on AWS instance using Jenkins job
• Show all stored logs command: $ hydra investigate show-logs 08087b94-c377-4cea-8788-4cb54d2f5263

Logs:

• db-cluster-08087b94.tar.gz - https://cloudius-jenkins-test.s3.amazonaws.com/08087b94-c377-4cea-8788-4cb54d2f5263/20230421_105600/db-cluster-08087b94.tar.gz
• sct-runner-events-08087b94.tar.gz - https://cloudius-jenkins-test.s3.amazonaws.com/08087b94-c377-4cea-8788-4cb54d2f5263/20230421_105600/sct-runner-events-08087b94.tar.gz
• sct-08087b94.log.tar.gz - https://cloudius-jenkins-test.s3.amazonaws.com/08087b94-c377-4cea-8788-4cb54d2f5263/20230421_105600/sct-08087b94.log.tar.gz
• monitor-set-08087b94.tar.gz - https://cloudius-jenkins-test.s3.amazonaws.com/08087b94-c377-4cea-8788-4cb54d2f5263/20230421_105600/monitor-set-08087b94.tar.gz
• loader-set-08087b94.tar.gz - https://cloudius-jenkins-test.s3.amazonaws.com/08087b94-c377-4cea-8788-4cb54d2f5263/20230421_105600/loader-set-08087b94.tar.gz

Jenkins job URL