"Segfault/invalid stdio handle/double free or corruption (!prev)" errors when using highs

[bluenote-1577]

"Segfault/invalid stdio handle/double free or corruption (!prev)" errors when using highs

[bluenote-1577]

Hi all,

Thanks for responding so quickly. I've manged to upload what seems to be a reproduce-able version, but unfortunately it comes from a specific problem (a network flow problem) that my algorithm works on with a few hundred variables and constraints. This problem didn't occur with a simple LP and I can only reproduce it using a specific dataset.

git clone https://github.com/bluenote-1577/glopp
cd glopp
git checkout flow (will take a while, the required input file is > 600mb. requires git-lfs installed otherwise inputs will not be downloaded)
cargo build (requires cmake version > 3.12 in path)
./target/debug/glopp -b tests/long-10305070.bam -c tests/10kcutoff-lofreq_500k_unif.vcfc_header -e 0.05

[lovasoa]

Hello, and thanks for the report ! Do you have a rust project for reproduction ?

@jajhall , does that ring a bell ? Do you think this is a bug in HiGHS itself, or is it the use of HiGHS C API in this library that is unsafe ?

[jajhall]

It doesn't, sorry, we've not had any similar issues raised by people using the C API.

I don't recognise any of the output as coming from HiGHS.

As for the Highs.log file being empty, we don't flush the buffer on return from HiGHS methods, so a segfault like this might mean that output is lost. Adding the flush would be wise. I've also got an issue to look at relating to Highs.log being opened, even if HighsOptions::output_flag is immediately set to false, so I'll look at both together.

[lovasoa]

The log.txt file does not come from HiGHS, it's what @bluenote-1577 logs from his own code. It represents the problem that he is solving with HiGHS.

[lovasoa]

@jajhall : I think there may be a double-free when a problem is created with Highs_create, then incoherent values are passed with Highs_passLp, which returns an error status, and then the problem is freed with Highs_destroy.
I added a fuzzer which found that issue almost immediately. It may be pertinent for you to add fuzzing directly to HiGHS.

[jajhall]

Thanks: I'll see if I can reproduce this locally

[lovasoa]

Looking into it deeper, it looks like the problem (or at least one problem) comes from the fclose(log_file_stream) in Highs' destructor. HiGHS may be trying to close an already closed file.

[lovasoa]

I'm not sure this is the same issue, but just running the tests in this repository with AdressSanitizer on reproduces the issue.

[lovasoa]

Since the memory corruption happens in HiGHS when changing the default value of the log_file option, I published v0.5.1 where I disabled the suppression of HiGHS default behavior of trying to create a log file even when it is used as a library.

I'll re-enable it when HiGHS fixes the error on their side. In the meantime, the library will create a Highs.log file that you probably don't want, but it should't segfault anymore.