juliasmoothoptimizers / hsl.jl Goto Github PK
View Code? Open in Web Editor NEWJulia interface to the HSL Mathematical Software Library
License: Other
Julia interface to the HSL Mathematical Software Library
License: Other
We should add Project.toml
and Manifest.toml
files so it complies with Julia 1.X Package Manager.
This issue is used to trigger TagBot; feel free to unsubscribe.
If you haven't already, you should update your TagBot.yml
to include issue comment triggers.
Please see this post on Discourse for instructions and more details.
If you'd like for me to do this for you, comment TagBot fix
on this issue.
I'll open a PR within a few hours, please be patient!
Dear all,
I came across a weird problem trying to replace OpenBLAS with MKL. Although I have installed the MKL package (] add MKL ; test MKL), in deps/build.jl I added as first statement ( using MKL ). But when the code runs it complains that:
(@v1.7) pkg> build --verbose HSL
Building HSL →~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/bfe35eb36d387c1d0913aa2359210dac32764e19/build.log
ERROR: LoadError: ArgumentError: Package MKL not found in current path:
import Pkg; Pkg.add("MKL")
to install the MKL package.Stacktrace:
[1] require(into::Module, mod::Symbol)
@ Base ./loading.jl:967
[2] include(fname::String)
@ Base.MainInclude ./client.jl:451
[3] top-level scope
@ none:5
Any ideas?
As a follow up to #12, I've been trying to build HSL's ma57
branch, but I've been having trouble.
My steps were:
rm -rf
removing it) withPkg.add("Homebrew")
hsl_ma57-5.2.0.tar.gz
into downloads
build.jl
file with my SHA and comment out the ma97 stuff (for now)brew tap dpo/openblas
& brew tap-pin dpo/openblas
Pkg.build("HSL")
Step 5 above led to the following error:
failed process: Process(`patch -p1 -i /Users/jakeroth/.julia/v0.6/HSL/deps/downloads/get_factors.patch`, ProcessExited(2)) [2]
with similar output to @lrsantos11:
julia> Pkg.build("HSL")
INFO: Building Homebrew
INFO: Building HSL
INFO: looking for /Users/jakeroth/.julia/v0.6/HSL/deps/downloads/hsl_ma57-5.2.0.tar.gz
INFO: hsl_ma57 found
INFO: Attempting to create directory /Users/jakeroth/.julia/v0.6/HSL/deps/src
INFO: Directory /Users/jakeroth/.julia/v0.6/HSL/deps/src already exists
INFO: Attempting to create directory /Users/jakeroth/.julia/v0.6/HSL/deps
INFO: Directory /Users/jakeroth/.julia/v0.6/HSL/deps already exists
INFO: Path /Users/jakeroth/.julia/v0.6/HSL/deps/src/hsl_ma57-5.2.0 already exists
INFO: Changing directory to /Users/jakeroth/.julia/v0.6/HSL/deps/src/hsl_ma57-5.2.0
patch: **** Can't open patch file /Users/jakeroth/.julia/v0.6/HSL/deps/downloads/get_factors.patch : No such file or directory
===================================[ ERROR: HSL ]====================================
LoadError: failed process: Process(`patch -p1 -i /Users/jakeroth/.julia/v0.6/HSL/deps/downloads/get_factors.patch`, ProcessExited(2)) [2]
while loading /Users/jakeroth/.julia/v0.6/HSL/deps/build.jl, in expression starting on line 48
=====================================================================================
==================================[ BUILD ERRORS ]===================================
WARNING: HSL had build errors.
- packages with build errors remain installed in /Users/jakeroth/.julia/v0.6
- build the package(s) and all dependencies with `Pkg.build("HSL")`
- build a single package by running its `deps/build.jl` script
=====================================================================================
What is the get_factors.patch
? Any thoughts would be appreciated! Additionally, if I'm able to get it working, would it be useful for me to add a bit of detail to the documentation?
Hi
When I try to build HSL I have the following error:
LoadError: Checksum mismatch! Expected:
e3133eba520abcabaf1700106f8a1310d4286c61721f707e94fb7d45a220abf2
Calculated:
552114617bddbc722ef014856463d0d29c2398abda60a05debccb7a6e8e1db7a
Delete /home/.../.julia/v0.5/HSL/deps/downloads/hsl_ma97-2.3.0.tar.gz and try again
while loading /home/.../.julia/v0.5/HSL/deps/build.jl, in expression starting on line 36
I have deleted it and tried again and it didn't work. Also I have this this just before the rror message
INFO: Building HSL
INFO: looking for /home/.../.julia/v0.5/HSL/deps/downloads/hsl_ma97-2.3.0.tar.gz
INFO: hsl_ma97 found
I don't understand why it's not found since it's there..
Sam
Edit: If I manually go change the expected number in the build.jl I get this new error:
LoadError: Provider BinDeps.BuildProcess failed to satisfy dependency libhsl_ma97
while loading /home/.../.julia/v0.5/HSL/deps/build.jl, in expression starting on line 36
hsl_ma57_patch.jl
.Hi,
I have followed the installation instructions for HSL.jl, namely to set the "HSL_ARCHIVES_PATH" environment variable to the directory where I unzipped the coinhsl.zip archive, and then to build using Pkg. However, the installation does not seem to work, as the Ma97 and Ma57 symbols are not exported. I also tried setting the environment variables "HSL_MA57_PATH" and "HSL_MA97_PATH" to the directory, which unfortunately didn't work either. Any help would be greatly appreciated!
Thanks!
Edit: After pointing "HSL_ARCHIVES_PATH" to a folder with the uncompressed coinhsl.zip, it seems that the build process has compiled libcoinhsl.so, and "HSL.available_hsl_algorithms" contains the key-value pair "coinhsl" => v"2021.5.5" . However, this still does not expose Ma57 or Ma97, and it's not clear to me how I can interact with the solvers.
Our use case is a square, nonsymmetric KKT system, with multiple right hand sides.
It would be good if ma**_info and ma**_control can be constructed without specifying the initial values
The current installation steps require the user to copy the HSL archive to the package's directory, before running build HSL
.
Unless the package is dev-ed, this will typically be somewhere like ~/.julia/packages/HSL/XYZ
, possibly changing at every new version.
Commercial optimization solvers' wrappers like Gurobi.jl require the user to, instead, specify the location of their Gurobi path before before the Julia wrapper.
In the HSL case (assuming I saved the archives to ~/hsl/
, this could look like
ENV["HSL_MA57_PATH"] = "~/hsl/hsl_ma57-5.2.0.tar.gz"
ENV["HSL_MA97_PATH"] = "~/hsl/hsl_ma97-2.6.0.tar.gz"
Pkg.build("HSL")
From a user perspective it would remove the need to find the current package location, and manually copy the archive there.
As far as the build script is concerned, one could check whether these environment variables are defined (and, if not, look for a default location), then copy the archive to the deps/download
and build the package as usual.
cc @amontoison
After some discussions about performances with @geoffroyleconte, I compared OpenBLAS32 / MKL backend and a basic compilation with a " -O3" compilation.
We should add -03
after gfortran
and gcc
by default.
const HSL_FC = haskey(ENV, "HSL_FC") ? ENV["HSL_FC"] : "gfortran -O3"
const HSL_F77 = haskey(ENV, "HSL_F77") ? ENV["HSL_F77"] : HSL_FC
const HSL_CC = haskey(ENV, "HSL_CC") ? ENV["HSL_CC"] : "gcc -O3"
# Current version of HSL
using HSL, MatrixMarket, SuiteSparseMatrixCollection
using LinearAlgebra, Printf, BenchmarkTools
ssmc = ssmc_db(verbose=false)
matrix = ssmc_matrices(ssmc, "Boeing", "pwtk")
path = fetch_ssmc(matrix, format="MM")
n = matrix.nrows[1]
A = MatrixMarket.mmread(joinpath(path[1], "$(matrix.name[1]).mtx"))
b = ones(n)
b_norm = norm(b)
# Solve Ax = b.
LDL = @btime Ma57($A) # 7.566 s (36 allocations: 343.44 MiB)
@btime ma57_factorize($LDL) # 39.155 s (2 allocations: 851.30 KiB)
@btime ma57_solve($LDL, $b) # 497.909 ms (6 allocations: 4.16 MiB)
import LinearAlgebra, MKL_jll
LinearAlgebra.BLAS.lbt_forward(MKL_jll.libmkl_rt_path, clear=true, verbose=true)
# Solve Ax = b.
LDL = @btime Ma57($A) # 7.466 s (36 allocations: 343.44 MiB)
@btime ma57_factorize($LDL) # 25.038 s (2 allocations: 851.30 KiB)
@btime ma57_solve($LDL, $b) # 230.605 ms (6 allocations: 4.16 MiB)
# HSL compiled with -O3
using HSL, MatrixMarket, SuiteSparseMatrixCollection
using LinearAlgebra, Printf, BenchmarkTools
ssmc = ssmc_db(verbose=false)
matrix = ssmc_matrices(ssmc, "Boeing", "pwtk")
path = fetch_ssmc(matrix, format="MM")
n = matrix.nrows[1]
A = MatrixMarket.mmread(joinpath(path[1], "$(matrix.name[1]).mtx"))
b = ones(n)
b_norm = norm(b)
# Solve Ax = b.
LDL = @btime Ma57($A) # 3.123 s (36 allocations: 343.44 MiB)
@btime ma57_factorize($LDL) # 14.857 s (2 allocations: 851.30 KiB)
@btime ma57_solve($LDL, $b) # 314.188 ms (6 allocations: 4.16 MiB)
import LinearAlgebra, MKL_jll
LinearAlgebra.BLAS.lbt_forward(MKL_jll.libmkl_rt_path, clear=true, verbose=true)
# Solve Ax = b.
LDL = @btime Ma57($A) # 3.345 s (36 allocations: 343.44 MiB)
@btime ma57_factorize($LDL) # 9.488 s (2 allocations: 851.30 KiB)
@btime ma57_solve($LDL, $b) # 186.227 ms (6 allocations: 4.16 MiB)
How likely is it that some of the older things in https://www.hsl.rl.ac.uk/archive/ might be released for a rewrite to other languages (obviously, I am thinking of Julia!)?
We should find how to do that in CUDA.jl.
Hi, I need to be able to convert a Ma57{Float32}
to a Ma57{Float64}
without having to compute the symbolic analysis again. Is there an easy way I could do this? The default constructor for the Ma57
struct has been erased.
Below is the full list of HSL packages from https://www.hsl.rl.ac.uk/catalogue.
This issue is used to track the progress on HSL interfaces and to assign a priority to each.
Next to each package, please indicate:
As a guideline:
Tick the box when an interface is complete and the pull request has been merged.
We will proceed in stages.
Stage 1:
Stage 2:
Next stages: TBD
See also
HSL_MA86 (Parallel sparse symmetric indefinite/Hermitian)
HSL_MA87 (Parallel sparse Cholesky)
HSL_MA97 (Bit-compatible parallel sparse symmetric/Hermitian)
We should add in the documentation that HSL.jl supports LBT and also how to switch a BLAS vendor with LBT.
using ReferenceBLAS32_jll, LAPACK32_jll
LinearAlgebra.BLAS.lbt_forward(libblas32)
LinearAlgebra.BLAS.lbt_forward(liblapack32)
using blis32_jll, LAPACK32_jll
LinearAlgebra.BLAS.lbt_forward(blis32)
LinearAlgebra.BLAS.lbt_forward(liblapack32)
import LinearAlgebra, OpenBLAS32_jll
LinearAlgebra.BLAS.lbt_forward(OpenBLAS32_jll.libopenblas_path)
import LinearAlgebra, MKL_jll
LinearAlgebra.BLAS.lbt_forward(MKL_jll.libmkl_rt_path)
using AppleAccelerate
import LinearAlgebra, blis_jll, LAPACK_jll
LinearAlgebra.BLAS.lbt_forward(blis)
LinearAlgebra.BLAS.lbt_forward(liblapack)
# To easily reset some BLAS/LAPACK backends.
LinearAlgebra.__init__()
julia> BLAS.get_config()
LinearAlgebra.BLAS.LBTConfig
Libraries:
└ [ILP64] libopenblas64_.s
MKL.__init__()
julia> BLAS.get_config()
LinearAlgebra.BLAS.LBTConfig
Libraries:
├ [ILP64] libmkl_rt.so
└ [ LP64] libmkl_rt.so
I am reviewing the CITATION usage in the repos and in particular moving to CFF files. The bib can still be generated using Github menu.
Here are a couple of suggestions to adapt this repo:
Date: 06/12/2022
checksums.txt
After #93.
Hey all,
First, let me say thanks for putting this package together -- this looks like it's going to be really helpful :-)
When trying to build this package with HSL_MA57, following the instructions in the README, I encounter the following error (taken from the logfile):
Making install in example
make[1]: Entering directory '/home/dmrosen/.julia/dev/HSL/deps/usr/src/hsl_ma57-5.2.0/example'
make[2]: Entering directory '/home/dmrosen/.julia/dev/HSL/deps/usr/src/hsl_ma57-5.2.0/example'
make[2]: Nothing to be done for 'install-exec-am'.
make[2]: Nothing to be done for 'install-data-am'.
make[2]: Leaving directory '/home/dmrosen/.julia/dev/HSL/deps/usr/src/hsl_ma57-5.2.0/example'
make[1]: Leaving directory '/home/dmrosen/.julia/dev/HSL/deps/usr/src/hsl_ma57-5.2.0/example'
make[1]: Entering directory '/home/dmrosen/.julia/dev/HSL/deps/usr/src/hsl_ma57-5.2.0'
make[2]: Entering directory '/home/dmrosen/.julia/dev/HSL/deps/usr/src/hsl_ma57-5.2.0'
make[2]: Nothing to be done for 'install-exec-am'.
make[2]: Nothing to be done for 'install-data-am'.
make[2]: Leaving directory '/home/dmrosen/.julia/dev/HSL/deps/usr/src/hsl_ma57-5.2.0'
make[1]: Leaving directory '/home/dmrosen/.julia/dev/HSL/deps/usr/src/hsl_ma57-5.2.0'
/usr/bin/ld: /home/dmrosen/.julia/dev/HSL/deps/usr/lib/libhsl_ma57.a(hsl_ma57s.o): relocation R_X86_64_PC32 against symbol__hsl_ma57_single_MOD_ma57_regen' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld: final link failed: Bad value collect2: error: ld returned 1 exit status [ Info: building ma57 ERROR: LoadError: LoadError: failed process: Process(
gfortran -fPIC -shared -Wl,--whole-archive /home/dmrosen/.julia/dev/HSL/deps/usr/lib/libhsl_ma57.a -L/home/dmrosen/.julia/dev/HSL/deps/usr/lib -lopenblas -lcoinmetis -lgomp -Wl,--no-whole-archive -o /home/dmrosen/.julia/dev/HSL/deps/usr/lib/libhsl_ma57.so`, ProcessExited(1)) [1]
Stacktrace:
[1] error(::String, ::Base.Process, ::String, ::Int64, ::String) at ./error.jl:42
[2] pipeline_error at ./process.jl:705 [inlined]
[3] #run#505(::Bool, ::Function, ::Cmd) at ./process.jl:663
[4] run(::Cmd) at ./process.jl:661
[5] top-level scope at none:0
[6] include at ./boot.jl:317 [inlined]
[7] include_relative(::Module, ::String) at ./loading.jl:1044
[8] include(::Module, ::String) at ./sysimg.jl:29
[9] include(::String) at ./client.jl:392
[10] top-level scope at /home/dmrosen/.julia/dev/HSL/deps/build.jl:78
[11] include at ./boot.jl:317 [inlined]
[12] include_relative(::Module, ::String) at ./loading.jl:1044
[13] include(::Module, ::String) at ./sysimg.jl:29
[14] include(::String) at ./client.jl:392
[15] top-level scope at none:0
in expression starting at /home/dmrosen/.julia/dev/HSL/deps/build_hsl_ma57.jl:15
in expression starting at /home/dmrosen/.julia/dev/HSL/deps/build.jl:22
It looks here like the operative lines are:
/usr/bin/ld: /home/dmrosen/.julia/dev/HSL/deps/usr/lib/libhsl_ma57.a(hsl_ma57s.o): relocation R_X86_64_PC32 against symbol `__hsl_ma57_single_MOD_ma57_regen' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
Any idea what might be causing this error? (I'm running Julia 1.0.5 on Ubuntu 18.04, using the Julia editor in Atom)
Any help would be greatly appreciated!
@dpo Do you think is worth to implement a wrapper for HSL_MA57? What is the biggest difference? I'm not sure which one should I use.
BTW, are you going to ISMP at Bordeaux? If so, we should drink some wine and talk (live).
Cheers,
Rafa
... and remove logging.
Hello,
I am trying to install ma97 and interface it with Julia in a Windows machine. I tried your code, but I get this error:
error("Your platform (\"$(Sys.MACHINE)\", parsed as \"$(triplet(platform_key_abi()))\") is not supported by this package because of dependency $vname")
From the comments, I can't quite understand what that error exactly means.
Also, I have just recieved the latest version of Ma97 and it is 2.5.0, while the file is written for 2.4.0. Hence, I changed the string to be compatible with the file I have. Could it be that this is the reason for the problem? Or is this code created only for specific machines? It is not clear in the readme file, which extra packages I should install in order for the build.jl function to work.
I would really appreciate it if you could help me.
Many thanks
A new version version of MA97 is available.
We should update the following lines :
https://github.com/JuliaSmoothOptimizers/HSL.jl/blob/master/deps/build.jl#L15-L17
I want to restrict Ma57 to using a single thread.
Ideally, I would only need to do
LinearAlgebra.BLAS.set_num_threads(1)
but, if I understand the build steps correctly, the package builds its own OpenBlas libary.
Is there any way one can control the number of threads there?
I'm getting a strange behavior on an x86 AMD machine (I can run everything just fine on my Intel laptop). When trying to run the tests I get:
ERROR: LoadError: could not load library "/home/brian/.julia/dev/HSL/deps/usr/lib/libhsl_ma97.so"
libopenblas.so: cannot open shared object file: No such file or directory
It's not clear to me what this message is saying, I'm guessing it's having troubles linking again libopenblas? The build step runs fine, I even get this output during the configure step:
checking for sgemm_ in -L/home/brian/.julia/artifacts/7fca6a59f5903d74ab82fd8c36b7656125da4318/lib -lopenblas... yes
checking for cheev_... yes
checking for METIS library... checking for metis_nodend_ in -L/home/brian/.julia/artifacts/21426dc9da406ede1def01afaf9e86678d0f84ad/lib -lmetis... yes
So it found the library just fine. Any ideas as to what the problem is?
Following NLPModelsKnitro.jl example:
using HSL
JuliaSmoothOptimizers/NLPModelsKnitro.jl#98Thanks for taking the lead on hooking up HSL.
When trying to build the package on my Mac with julia 0.4.1, I'm getting the following error:
julia> Pkg.build("HSL")
INFO: Building Homebrew
HEAD is now at 4705b23 fzf: update 0.10.9 bottle.
HEAD is now at b178725 Merge pull request #83 from staticfloat/staging
INFO: Building HSL
Error: No available formula with the name "metis4"
===========================================[ ERROR: HSL ]============================================
LoadError: ArgumentError: Cannot find formula for metis4!
while loading /Users/lruthot/.julia/v0.4/HSL/deps/build.jl, in expression starting on line 21
=====================================================================================================
==========================================[ BUILD ERRORS ]===========================================
WARNING: HSL had build errors.
- packages with build errors remain installed in /Users/lruthot/.julia/v0.4
- build the package(s) and all dependencies with `Pkg.build("HSL")`
- build a single package by running its `deps/build.jl` script
=====================================================================================================
I installed metis4, but still the Pkg.build fails with the same error. Any thoughts?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.