Failed cargo install with SSL feature under linux (RustPython 0.3.0)

[Beanzilla]

Failed cargo install with SSL feature under linux (RustPython 0.3.0)

[fanninpm]

Did you install the libssl-dev package?

[carrt123]

Have you solved the problem?

[carrt123]

I have exact same issue on AlmaLinux release 8.9 (Midnight Oncilla). openssl-devel-1.1.1k-9.el8_7.x86_64 is installed.
Have you solved the problem?

[oroppas]

Okay. I'm able to build on Windows 11. Here's how.

Use winget to install Strawberry Perl

winget install StrawberryPerl.StrawberryPerl

then

cargo build --release --features ssl-vendor

(It takes really long time to build openssl-sys v0.9.98. Please be patient!)

C:\msys64\home\ryuta\packages\rustpython>.\rustpython\target\release\rustpython.exe
Welcome to the magnificent Rust Python 0.3.1 interpreter 😱 🖖
RustPython 3.12.0
Type "help", "copyright", "credits" or "license" for more information.
>>>>> import sys
>>>>> sys.getwindowsversion()
sys.getwindowsversion(major=6, minor=2, build=9200, platform=2, service_pack='', service_pack_major=0, service_pack_minor=0, suite_mask=256, product_type=1, platform_version=(6, 2, 9200))
>>>>>

Unfortunately, rustpython -m ensurepip failed. Can I file it as a separate issue?
Already reported as in #5146. It is related to missing _ctypes.

[youknowone]

There must be a patch we moved from 1.x to 3, but I have no idea which one. Trying revert may be worth if you need it.

[oroppas]

Okay. I needed to set X86_64_PC_WINDOWS_MSVC_OPENSSL_DIR, for example on my environment

PowerShell

PS D:\packages\rustpython\rustpython> cargo install cargo-vcpkg
PS D:\packages\rustpython\rustpython> cargo vcpkg build
PS D:\packages\rustpython\rustpython> $env:X86_64_PC_WINDOWS_MSVC_OPENSSL_DIR="D:\packages\rustpython\rustpython\target\vcpkg\packages\openssl_x64-windows-static-md"
PS D:\packages\rustpython\rustpython> cargo build --release --features ssl

Command Prompt

D:\packages\rustpython\rustpython> cargo install cargo-vcpkg
D:\packages\rustpython\rustpython> cargo vcpkg build
D:\packages\rustpython\rustpython> set X86_64_PC_WINDOWS_MSVC_OPENSSL_DIR=D:\packages\rustpython\rustpython\target\vcpkg\packages\openssl_x64-windows-static-md
D:\packages\rustpython\rustpython> cargo build --release --features ssl

rustpython -m ensurepip still fails with ModuleNotFoundError: No module named '_ctypes'

[Beanzilla]

Did you install the libssl-dev package?

libssl-dev (1.1.1f-1ubuntu2.22)

Updated initial comment (added libssl-dev version and added attempt with [d0c827e])

[oroppas]

I have exact same issue on AlmaLinux release 8.9 (Midnight Oncilla).
openssl-devel-1.1.1k-9.el8_7.x86_64 is installed.

[Priler]

Same on Windows with open-ssl installed.

[theoparis]

Encountered the same issue with aws-lc. It is also based on openssl 1.1.x so maybe that has something to do with it?

Edit: Reverting the changes from 32f662a in stdlib/src/ssl seems to fix it, however I ran into a bunch of other errors during link time 👀

[oroppas]

Yeah, I suspect openssl 1.1.x is a culprit since I can build without an issue on Fedora 39 (openssl 3.1.1) and Ubutnu 22.04 (openssl 3.0.2).

[carrt123]

Yeah, I suspect openssl 1.1.x is a culprit since I can build without an issue on Fedora 39 (openssl

You are right. I consulted with project contributor, and his anwer was the same as yours.

[youknowone]

Thank you for confirming. How about using --features ssl-vendor? Does it help to solve the problem?

[oroppas]

Thanks! --features ssl-vendor works on AlmaLinux 8.9

On Windows, it appears we need to install perl separately

error: failed to run custom build command for `openssl-sys v0.9.98`

Caused by:
  process didn't exit successfully: `C:\msys64\home\ryuta\packages\rustpython\rustpython\target\release\build\openssl-sys-0e5ebd593dca87ba\build-script-main` (exit code: 101)
  --- stdout
  cargo:rerun-if-env-changed=X86_64_PC_WINDOWS_MSVC_OPENSSL_NO_VENDOR
  X86_64_PC_WINDOWS_MSVC_OPENSSL_NO_VENDOR unset
  cargo:rerun-if-env-changed=OPENSSL_NO_VENDOR
  OPENSSL_NO_VENDOR unset
  running "perl" "./Configure" "--prefix=C:/msys64/home/ryuta/packages/rustpython/rustpython/target/release/build/openssl-sys-d6016873a007658f/out/openssl-build/install" "--openssldir=SYS$MANAGER:[OPENSSL]" "no-dso" "no-shared" "no-ssl3" "no-tests" "no-comp" "no-zlib" "no-zlib-dynamic" "--libdir=lib" "no-md2" "no-rc5" "no-weak-ssl-ciphers" "no-camellia" "no-idea" "no-seed" "no-capieng" "no-asm" "VC-WIN64A"

  --- stderr
  thread 'main' panicked at C:\Users\ryuta\.cargo\registry\src\index.crates.io-6f17d22bba15001f\openssl-src-300.2.1+3.2.0\src\lib.rs:611:9:



  Error configuring OpenSSL build:
      Command: "perl" "./Configure" "--prefix=C:/msys64/home/ryuta/packages/rustpython/rustpython/target/release/build/openssl-sys-d6016873a007658f/out/openssl-build/install" "--openssldir=SYS$MANAGER:[OPENSSL]" "no-dso" "no-shared" "no-ssl3" "no-tests" "no-comp" "no-zlib" "no-zlib-dynamic" "--libdir=lib" "no-md2" "no-rc5" "no-weak-ssl-ciphers" "no-camellia" "no-idea" "no-seed" "no-capieng" "no-asm" "VC-WIN64A"
      Failed to execute: program not found```

[youknowone]

Thank you for checking it!
I wish it is a big problem for windows users. Windows does not have a system vendored openssl. If they could install openssl, installing openssl3 also will be possible.

[carrt123]

I successfly install the rustpython with --features ssl-vendor. but the method fail to install pip.
SystemError: install-pip requires rustpython be build with '--features=ssl'

My System info：
Linux: Ubuntu 20.04
Rust: rustc 1.76.0
Openssl: openssl (1.1.1f-1)

[oroppas]

rustpyhon -m ensurepip works on AlmaLinux.

[youknowone]

@carrt123 Thank you, #5330 will fix it.

[youknowone]

@oroppas If you want to avoid perl, how about using ssl-vendor for linux and just ssl for windows? Our CI installs openssl with vcpkg.

[oroppas]

@youknowone, does that mean the Windows users can cargo build --release --features ssl without explicitly vcpkg install openssl? That would definitely obviate the need to install perl and I love the idea!

[youknowone]

@oroppas I am sorry, I am not good enough at windows ecosystem to say yes or no to the answer. Could you check if this command is working for your windows environment?

https://github.com/RustPython/RustPython/blob/main/.github/workflows/ci.yaml#L251-L256

The expected command is cargo build --release --features ssl as you said.

[oroppas]

cargo vcpkg build seems to install openssl:

    Fetching vcpkg
    Checkout rev 2024.02.14
  Installing openssl
    Finished in 37.78s

but cargo build --release --features ssl fails with the same error:

error[E0599]: no method named `all_certificates` found for reference `&X509StoreRef` in the current scope
   --> stdlib\src\ssl.rs:706:18
    |
703 |               let certs = self
    |  _________________________-
704 | |                 .ctx()
705 | |                 .cert_store()
706 | |                 .all_certificates()
    | |                 -^^^^^^^^^^^^^^^^ method not found in `&X509StoreRef`
    | |_________________|
    |

For more information about this error, try `rustc --explain E0599`.
error: could not compile `rustpython-stdlib` (lib) due to 1 previous error

[youknowone]

Isn't it because your windows env already have openssl 1.1.1 somewhere? openssl-sys has prioritized resolver. Probably vcpkg has lower priority than the other installation.

[oroppas]

yeah, that's what I'm guessing. Let me do clean-up.

[oroppas]

It still fails. I did cargo build --verbose --release --features ssl. The last line of the log caught my eyes:

 --cfg ossl101 --cfg ossl102 --cfg ossl110 --cfg ossl110g --cfg ossl111 --cfg "osslconf=\"OPENSSL_NO_SSL3_METHOD\""`

[theoparis]

Does the mentioned PR fix openssl 1.x? Although its not in rustpython I use openssl 1.x forks over openssl 3 due to openssl not implementing QUIC properly.

[youknowone]

@theoparis No, RustPython only supports openssl 3 now

[oroppas]

Hmm. X86_64_PC_WINDOWS_MSVC_OPENSSL_DIR may not be necessary after all.
Rebooting my windows machine and building from scratch without this environment variable still worked.
In any case, one less concern to build on windows!