PyInstaller and Wexpect

[raczben]

PyInstaller and Wexpect

[raczben]

Hi All,

Pyinstaller support is now beta!

You should use the new-structured wexpect, by setting WEXPECT_SPAWN_CLASS environment variable to SpawnPipe, and you should create wexpect executable manually, but it works!

Read this wiki page, for more information.

((The stable release, will come along with the new-structured wexpect aprox. a month later.))

[dgburr]

Great work, @raczben! I am using wexpect in a project and I was able to create a packaged .exe file based upon the instructions you provided. However I would like to make a couple of suggestions:

1. The instructions on the wiki say to use os.environ['WEXPECT_SPAWN_CLASS']=SpawnPipe whereas it should say os.environ['WEXPECT_SPAWN_CLASS']='SpawnPipe'
2. The current implementation is difficult to use if pyinstaller is run with the --onefile option. I was able to make it work as follows:

a) Add the following code before and after calls to wexpect.spawn():

real_executable = sys.executable
try:
    if sys._MEIPASS is not None:
        sys.executable = os.path.join(sys._MEIPASS, "wexpect", "wexpect.exe")
except AttributeError:
    pass
p = wexpect.spawn(cmd)
sys.executable = real_executable

b) Run pyinstaller with the following command:

pyinstaller --onefile --add-data <path_to_wexpect>\dist;. myscript.py

However it would be much nicer if you could add an environment variable (e.g. WEXPECT_EXECUTABLE) which can be used to point to a custom location of wexpect.exe, instead of the current implementation which hard-codes the path as sys.executable + "\..\wexpect\wexpect.exe"

[raczben]

There are couple of other open issues, which causes instant crash, so this issue will fixed later. Sorry for this decision.

[piypiy]

Same error in PyInstaller and cx_Freeze

I hope this error will fix it soon, I can’t build a project without ur wexpect (
thx for ur wexpect.
[windows version] Windows 10 [Version 10.0.18362.476]
[Python version] v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05 64-bit
[wexpect version] 2.3.9
[pyinstaller version] 3.5

[RyanLiuF]

Same error in PyInstaller.
I hope this error will fix it soon.
[windows version] Windows 10
[Python version] v3.7.4 64-bit
[wexpect version] 3.2.0
[pyinstaller version] 3.5

[raczben]

Hi all,

Sorry for the late reply, but I am the only maintainer...

The root problem is that pyinstaller has no explicit python executable. While wexpect call it, when it creates child. See https://stackoverflow.com/q/8888813/2506522 for more details.

This pyinstaller issue describe the same scenario, which wont be solved....

It seems a bit complicated issue, it will took 1-2 weeks to solve.

[piypiy]

Yeah..im waiting for solve,thx u😗

[RyanLiuF]

Pretty cool, tks!!

[ZionP92]

@raczben I have the same problem, and I did it just as https://github.com/raczben/wexpect/wiki/Wexpect-with-pyinstaller says, but in my case it brings another GUI not spawning a script that I put. I'm using pyqt5 for main.py and spawn child.py which has excel handling functions with an input line. I already asked it on stackflow as well, I'm sorry if you are busy but if you don't mine I hope you can tell me if it's the wexpect collides with pyinstaller. my codes are working fine on python

[raczben]

Hi,

Please answer all of these points:

1. Can you use pyinstaller in simple cases successfully?
2. Please describe your environment. (Wexpect version, Windows version, Python version, etc)
3. Please turn on logging. Set WEXPECT_LOGGER_LEVEL environment variable to INFO. It will create one log-file for the host and another for the child. Please send me the logs of correct and faulty behavior. (Note, that logs can contain sensitive information. In this case, remove those lines.)
4. Can you send a Minimal, Reproducible Example? Please reduce your code to as minimal as it is possible, but with the same error. It will speed up the fix.

[cnaferdie]

Hi @raczben, can you please provide an example of the following.

You should use the new-structured wexpect, by setting WEXPECT_SPAWN_CLASS environment variable to SpawnPipe, and you should create wexpect executable manually, but it works!

Sorry I'm new to python and below line from my script is not working after I use pyinstaller to convert .py to .exe. But my script is working fine without converting it to .exe. Hope to hear from you soon.

child = wexpect.SpawnPipe('ssh "-o StrictHostKeyChecking=no" ' + ap); 

[raczben]

Hi @cnaferdie ,

Please read this wiki page thoroughly.

If you still have a problem please answer all of these points:

1. Can you use pyinstaller in simple cases successfully? (I mean bundling my hello-wexpect example script.)
2. Please describe your environment. (Wexpect version, Windows version, Python version, etc)
3. Please turn on logging. Set WEXPECT_LOGGER_LEVEL environment variable to INFO. It will create one log-file for the host and another for the child. Please send me the logs of correct and faulty behavior. (Note, that logs can contain sensitive information. In this case, remove those lines.)
4. Can you send a Minimal, Reproducible Example? Please reduce your code to as minimal as it is possible, but with the same error. It will speed up the fix.

[cnaferdie]

Hi @raczben,
I read the wiki again and followed the structure below and it works! So it looks like the users have to install python and wexpect.exe also on their computers for this work? Or I can distribute the wexpect and my_script folders to the users and they can run the scripts? Please advise.

dist
├─wexpect
│ ├─ ...
│ ├─ wexpect.exe
│ └─ ...
└─your_script
├─ ...
├─ your_script.exe
└─ ...

[raczben]

Hi @cnaferdie

I'm glad to hear that it's working.
Installing Python isn't needed. (It would make no sense...)
You should deliver only the two bundles: the script of yours and the wexpect.

[cnaferdie]

Hi @cnaferdie

I'm glad to hear that it's working.
Installing Python isn't needed. (It would make no sense...)
You should deliver only the two bundles: the script of yours and the wexpect.

Okay. So I guess it's not yet possible to use the --onefile of the pyinstaller for now. I hope this will be possible in the future.

Thank you so much for your help. Really appreciate it. :)

[raczben]

No, wexpect does not support onefile mode right now. Please read this for a workaround.

[cnaferdie]

Hi @cnaferdie

I'm glad to hear that it's working.
Installing Python isn't needed. (It would make no sense...)
You should deliver only the two bundles: the script of yours and the wexpect.

Okay. So I guess it's not yet possible to use the --onefile of the pyinstaller for now. I hope this will be possible in the future.

Thank you so much for your help. Really appreciate it. :)

I just tried distributing the folders to one of my collegues. It's not woring on her PC.

[cnaferdie]

No, wexpect does not support onefile mode right now. Please read this for a workaround.

I have this in my script:

            real_executable = sys.executable
            try:
                if sys._MEIPASS is not None:
                    sys.executable = os.path.join(sys._MEIPASS, "wexpect", "wexpect.exe")
            except AttributeError:
                pass
            child = wexpect.spawn('ssh -o "StrictHostKeyChecking=no" ' + ap)
            sys.executable = real_executable

But I'm having this error message when I run the .exe file.

ExceptionPexpect: Wexpect executable: >>C:\Users\FORMAN~1\AppData\Local\Temp_MEI137202\wexpect..\wexpect\wexpect.exe<< not found.
It's not working either. Check device manually

[Starstorm]

Hi all,

So I'm having an issue with this, it's probably quite easy to fix but I can't seem to get it to work. So I followed the wiki instructions and ensured both my script folder and the wexpect I compiled are in the same directory (I'm using Windows 10, Python 3.7). Then I attempt to run a minimalist main.py script:

import wexpect, os
os.environ['WEXPECT_LOGGER_LEVEL']='INFO'
cmd_string = r"C:\Windows\System32\cmd.exe"
child = wexpect.spawn(cmd_string)

This works fine when I just do python main.py, but it does not run after using PyInstaller and then running the exe. The error it gives me when executing the .exe is the following:

Traceback (most recent call last):
  File "cli.py", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "d:\pykerasenv\gocheater\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 623, in exec_module
    exec(bytecode, module.__dict__)
  File "main.py", line 4, in <module>
  File "site-packages\wexpect-4.0.1.dev4-py3.7.egg\wexpect\host.py", line 878, in __init__
  File "site-packages\wexpect-4.0.1.dev4-py3.7.egg\wexpect\host.py", line 247, in __init__
  File "site-packages\wexpect-4.0.1.dev4-py3.7.egg\wexpect\host.py", line 897, in connect_to_child
wexpect.wexpect_util.TIMEOUT: Connect to child has been timed out.
[10188] Failed to execute script cli

Any ideas whats causing this?

Thanks!

[Starstorm]

UPDATE:
Solved the problem! The issue was that I compiled the wexpect executable under setuptools 45.0.0 and I didn't have 'pkg_resources.py2_warn' added as a hiddenimport - thus it would successfully find wexpect, attempt to load it, and it would instantly crash.

Yeesh...

[raczben]

Closing this topic...

If you have a problem using pyinstaller please read this wiki page, for more information.

[eirisdg]

Hi @raczben

I've followed the wiki, but i cannot do ir work.

When I call wexpect.spawn I get error.
'wexpect' has no attribute 'spawn'

The structure is the same than in the wiki.

My script is very simple, but still doesn't work:

import wexpect

prompt = '[A-Za-z]\:.+>'
child = wexpect.spawn('cmd.exe')
child.expect(prompt)

[freQniK]

Hai - Do you have a Bitcoin or Monero address I can send some funds to as a donation for a wonderful project that saved my rear end when porting a Linux & OS X app that relied heavily on pexpect but ran into issues on Windows with. It turns out, the Pyinstaller work-a-round was a success and I'd like to show my appreciate by sending some neros or satoshis your way.

Thanks you very much for a wonderful wrapper library!