pyinfra-dev/pyinfra-windows

pyinfra-windows's Issues

winrm parallel execution failure

Describe the bug

There is a bug in the winrm connector that manifests itself sometimes depending on the facts/ops running when run in parallel with multiple hosts. This does not happen with --serial so it's something to do with parallel execution.

The ultimate exception raised is from ntlm_auth -- "The signature checksum does not match, message has been altered"

This is not on all parallel runs, only when a certain op/fact is triggered across amongst the host set. I will try getting a better repro in the next few days and figure out what is happening.

To Reproduce

No generic repro yet.

Expected behavior

The parallel execution works the same as serial.

Windows @local/@docker support (Hello World example from tutorial doesn't work)

Describe the bug

I got error as in log below. I repeated command, but the error persist.

PowerShell 7.1.3
Copyright (c) Microsoft Corporation.

https://aka.ms/powershell
Type 'help' to get help.

PS C:\moje\programowanie\projekty2021_2\pyinfra> & c:/moje/programowanie/projekty2021_2/pyinfra/.venv_win/Scripts/Activate.ps1
(pyinfra_venv) PS C:\moje\programowanie\projekty2021_2\pyinfra> pyinfra @docker/ubuntu:18.04 exec -- echo "hello world"
--> Loading config...
--> Loading inventory...        

--> Connecting to hosts...      
    [@docker/ubuntu:18.04] Connected

--> Proposed changes:
    Groups: @docker
    [@docker/ubuntu:18.04]   Operations: 1   Commands: 1   

--> Beginning operation run...
--> Starting operation: Server/Shell (echo hello world)    
[@docker/ubuntu:18.04] 'sh' is not recognized as an internal or external command,
[@docker/ubuntu:18.04] operable program or batch file.
    [@docker/ubuntu:18.04] Error
localhost: sha256:3a3098f7fb5bb4affdd2b9a2c31579ea4c83a144d2ca9bbb7bbe8cc803b8b40a
localhost: 45aa42d84b87499a9aa88a9592c241497c4b7466c1c5e3f31cba7737fcd03f9f
    [@docker/ubuntu:18.04] docker build complete, image ID: 3a3098f7fb5b
No hosts remaining!
--> pyinfra error: 
(pyinfra_venv) PS C:\moje\programowanie\projekty2021_2\pyinfra> 

To Reproduce

pyinfra @docker/ubuntu:18.04 exec -- echo "hello world"

Expected behavior

No error

Meta

• Include output of pyinfra --support:

pyinfra --support
--> Support information:

    If you are having issues with pyinfra or wish to make feature requests, please
    check out the GitHub issues at https://github.com/Fizzadar/pyinfra/issues .
    When adding an issue, be sure to include the following:

    System: Windows
      Platform: Windows-10-10.0.19042-SP0
      Release: 10
      Machine: AMD64
    pyinfra: v1.4.11
    Executable: C:\moje\programowanie\projekty2021_2\pyinfra\.venv_win\Scripts\pyinfra.exe
    Python: 3.9.6 (CPython, MSC v.1929 64 bit (AMD64))

• How was pyinfra installed (source/pip)?

pip in venv

• Include pyinfra-debug.log (if one was created)

not created

• Consider including output with -vv and --debug.

 pyinfra -vv @docker/ubuntu:18.04 exec -- echo "hello world" 
--> Loading config...
--> Loading inventory...

--> Connecting to hosts...
localhost: >>> docker container inspect ubuntu:18.04
localhost: >>> docker run -d ubuntu:18.04 tail -f /dev/null
    [@docker/ubuntu:18.04] Connected

--> Proposed changes:
    Groups: @docker
    [@docker/ubuntu:18.04]   Operations: 1   Commands: 1

--> Beginning operation run...
--> Starting operation: Server/Shell (echo hello world)
[@docker/ubuntu:18.04] >>> sh -c 'docker exec -i 7a450bc13cd064cf993ab2988ba8775fbdeea08e1349bf91aedf9c9fc01d02f8 sh -c '"'"'sh -c '"'"'"'"'"'"'"'"'echo hello world'"'"'"'"'"'"'"'"''"'"''
[@docker/ubuntu:18.04] 'sh' is not recognized as an internal or external command,
[@docker/ubuntu:18.04] operable program or batch file.
    [@docker/ubuntu:18.04] Error
localhost: >>> docker commit 7a450bc13cd064cf993ab2988ba8775fbdeea08e1349bf91aedf9c9fc01d02f8
localhost: sha256:0b141c126885c0a356078ffad346607dfacd1dbfb82babeef90d2712686502cc
localhost: >>> docker rm -f 7a450bc13cd064cf993ab2988ba8775fbdeea08e1349bf91aedf9c9fc01d02f8
localhost: 7a450bc13cd064cf993ab2988ba8775fbdeea08e1349bf91aedf9c9fc01d02f8
    [@docker/ubuntu:18.04] docker build complete, image ID: 0b141c126885
No hosts remaining!
--> pyinfra error: 

pyinfra --debug @docker/ubuntu:18.04 exec -- echo "hello world"
    [pyinfra_cli\main] Deploy directory remains as cwd
--> Loading config...
--> Loading inventory...
    [pyinfra_cli\inventory] Creating fake inventory...

--> Connecting to hosts...
[pyinfra\api\connectors\util] --> Waiting for exit status...
[pyinfra\api\connectors\util] --> Command exit status: 1
[pyinfra\api\connectors\util] --> Waiting for exit status...
[pyinfra\api\connectors\util] --> Command exit status: 0
    [@docker/ubuntu:18.04] Connected
    [pyinfra\api\state] Activating host: @docker/ubuntu:18.04
    [pyinfra\api\operation] Adding operation, {'Server/Shell'}, opOrder=(0,), opHash=784a97bf1955d5f7a2b9dd6c1e371e17b73c42bc

--> Proposed changes:
    Groups: @docker
    [@docker/ubuntu:18.04]   Operations: 1   Commands: 1

--> Beginning operation run...
--> Starting operation: Server/Shell (echo hello world)
    [pyinfra\api\operations] Starting operation Server/Shell on @docker/ubuntu:18.04
[pyinfra\api\connectors\local] --> Running command on localhost: sh -c 'docker exec -i e1f7dbe16ca169f6f33d6f33ca42aa220a249466d26fa35e8cc7556b4da7aaab sh -c '"'"'sh -c '"'"'"'"'"'"'"'"'echo hello world'"'"'"'"'"'"'"'"''"'"''
[@docker/ubuntu:18.04] 'sh' is not recognized as an internal or external command,
[@docker/ubuntu:18.04] operable program or batch file.
[pyinfra\api\connectors\util] --> Waiting for exit status...
[pyinfra\api\connectors\util] --> Command exit status: 1
    [@docker/ubuntu:18.04] Error
    [pyinfra\api\state] Failing hosts: @docker/ubuntu:18.04
localhost: sha256:3c26a9596d24efca7f17e60683f33a4193fc1004fa8b022a01ce87c6d92f9141
[pyinfra\api\connectors\util] --> Waiting for exit status...
[pyinfra\api\connectors\util] --> Command exit status: 0
localhost: e1f7dbe16ca169f6f33d6f33ca42aa220a249466d26fa35e8cc7556b4da7aaab
[pyinfra\api\connectors\util] --> Waiting for exit status...
[pyinfra\api\connectors\util] --> Command exit status: 0
    [@docker/ubuntu:18.04] docker build complete, image ID: 3c26a9596d24
No hosts remaining!
--> pyinfra error: 

Some windows_files ops broken

Describe the bug

It appears some of the windows file operations weren't fully converted from legacy to current. Many of them use _create_remote_dir which proxies to the directory op but is attempting to pass along state and host too which is now incorect (https://github.com/Fizzadar/pyinfra/blob/2.x/pyinfra/operations/windows_files.py#L342).

To Reproduce

Try to use [at least] the windows link op.

Expected behavior

The link is created