netinvent/command_runner

Substitute for subprocess that handles all hassle that comes from different platform and python versions, and allows live stdout and stderr capture for background job/interactive GUI programming ;)

NetInvent SASU

Committed to open source Development

Happy to code our projects and help improve others.

command_runner's Issues

Cannot add environment variables with command runner

Hi,

I am trying to run the following windows command with command runner:

set TOTO=1 && <run some executable here>

This should create a temporary env variable that will be accessible during the time that some executable runs.
However, I got an error "file not found: [WinError 2] The system cannot find the file specified:"

That may be because set is not a file but an internal command ?
Either way, I would need to execute it, is there a solution ?

Allow raw output

By using a default encoding value, we enforce string output.
We need to add a encoding=False value so we can get raw binary output.

Investigate python 2.7 rare issue

command_runner_init_.py:474: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
for line in iter(stream.readline, sentinel_char):

sentinel_char is compared to line we get from stdout/stderr stream.
Both should be unicode, since we use from __future__ import unicode_literals

Nevertheless, we don't get a equal comparaison.

failed because of OS: [WinError 5] access denied

Command "ffmpeg ..." failed because of OS: [WinError 5] access denied, rarely prompt autonomly this error.

Print stdout in real time ?

I wanted to know if there's any way to display the output in real time while a command is running ?
This is possible when using subprocess, was wondering if I can do the same here

Thanks

Investigate Python 2.7 os.kill permission error

______________________ test_standard_ping_with_encoding _______________________

    def test_standard_ping_with_encoding():
        """
        Test command_runner with a standard ping and encoding parameter
        """
        for method in methods:
            print('method={}'.format(method))
            exit_code, output = command_runner(PING_CMD, encoding=ENCODING, method=method)
            print(output)
>           assert exit_code == 0, 'Exit code should be 0 for ping command with method {}'.format(method)
E           AssertionError: Exit code should be 0 for ping command with method monitor
E           assert -253 == 0
E             --253
E             +0

tests\test_command_runner.py:105: AssertionError
---------------------------- Captured stdout call -----------------------------
method=monitor\nCommand "ping 127.0.0.1 -n 4" failed because of OS: [Error 5] Acc\u03a6s refus
------------------------------ Captured log call ------------------------------
ERROR    command_runner:__init__.py:243 No psutil module present. Can only kill direct pids, not child subtree.
ERROR    command_runner:__init__.py:306 Could not properly kill process with pid 119160: [Error 5] Accs refus
ERROR   \x1b[1m\x1b[31mERROR   \x1b[0m command_runner:__init__.py:243 No psutil module present. Can only kill direct pids, not child subtree.\n\x1b[1m\x1b[31mERROR   \x1b[0m command_runner:__init__.py:306 Could not properly kill process with pid 119160: [Error 5] Accs refus\n\x1b[1m\x1b[31mERROR   \x1b[0m command_runner:__init__.py:867 Command "ping 127.0.0.1 -n 4" failed because of OS: [Error 5] Acc\u03a6s refus
========================== 1 failed in 5.69 seconds ===========================

We still get this sporadic error from time to time where OS Permission deneid happens when we try to kill an existing process with os.kill(pid, 15) on Python 2.7.
The pid exists or else we would have [Error 87] Incorrect parameter

Need to investigate whether access is denied because some lock is on process.

`command_runner` with `psutil` 5.5.1 (or lower)

I tried installing command_runner on via an embedded linux build system targeting a Debian 10 Buster distribution.
The build effectively does a pip install command-runner inside a ARM qemu system.

However the install fails due to the requirement for psutil>=5.6.0. pip tries to download this and build it, however I don't have the necessary build tools (i.e. gcc and friends) installed in the qemu system.

Debian 10 does have psutil 5.5.1 available as a package so I can install that easily.

Is there anyway to make command_runner work (install) with psutil 5.5.1 ??

What features of command_runner specifically need psutil > 5.6.0?

Interestingly command_runner installs ok for Python 2 (pip install command_runner), but not for Python 3 (pip3 install command_runner). I had to use psutil-5.7.2 from buster-backports to get it to install for Python 3.

# pip list | grep "runner\|psutil"
command-runner                1.4.0      
psutil                        5.5.1    

# pip3 list | grep "runner\|psutil"
command-runner     1.4.0  
psutil             5.7.2  

NOTE:

• psutil 5.7.2 is available from Debian Buster Backports repo, but only for Python 3.
• I could install the build tools (gcc, etc) but I didn't want them on my embedded filesystem (might take up too much space) - I might be able to remove those tools after the build?
• I might be able to build a wheel manually and locate that somewhere where I could install it.
• Might be time to finally upgrade legacy Python2 codebase to Python3 ;-)

Using elevate disables all print outputs in compiled console window

I'm writing this piece of software currently: https://github.com/Zenahr/ALACS
and am using this lib to allow it to work properly.

It does solve the problem I had (monitoring keyboard input while having the program window being not focused), however after implementing elevate the console window remains blank.

Why is that? Am I using the API wrong? Is there an option I don't know of that I could pass to elevate()?

This is my main code for reference:

import pynput
from pynput import keyboard
import lib
from lib import click_random_legend, click_main_legends
import json
from command_runner.elevate import elevate

ACTIVATION_BUTTON = json.load(open('./config.json'))['key']

def main():
    def on_press(key):
                if key == keyboard.Key[ACTIVATION_BUTTON]:
                    if not json.load(open('./config.json'))['select_random_legend']:
                        click_main_legends()
                    else:
                        click_random_legend()

    def on_release(key):
        pass

    print('BOOTING UP ALACS ...')
    print('BOOTED UP ALACS')
    print('MAKE SURE TO CLOSE THIS WINDOW AFTER CLOSING APEX')
    print('TO CHANGE THE ACTIVATION KEY READ THE INSTRUCTIONS FOUND IN instructions.txt')
    print('ACTIVATION KEY IS SET TO:', '>>> ', ACTIVATION_BUTTON, ' <<<')
    print('LISTENING FOR ACTIVATION KEY ...')

    with keyboard.Listener(
            on_press=on_press,
            on_release=on_release) as listener:
        listener.join()

if __name__ == '__main__':
    elevate(main)

How to get program stdout input when running as elevated

Hello team,

Great module - but I am having a slight problem - when I execute my main() so that the entire script runs elevated, how can I maintain the stdout that was appearing previously?

Thanks,
Joe Avanzato

Document piping runners into each other

Hi,

Apologies if I’ve missed it in the documentation.

Would it be possible to explain/illustrate how it would be possible to effectively execute a pipeline, with command_runner?

Example (made up): zcat in.txt.gz | tee -a out.txt | less.

I get that there’s a way to create queues, and potentially use those to redirect them to runners, but I don’t quite see a simple way of implementing the above.

Thanks!

README.md examples

Using ping 127.0.0.1 as example in README.md is not viable since it will run forever in Linux.
We need to find a command that exists on both Windows & Linux platforms by default.

Getting non-blocking live output for `stdout` and `stderr`

Is there a way to get live output for both stdout and stderr at the same time?

If there is a long running process that outputs to stdout with some errors on stderr interspersed, how can I read them in real-time (so that I can display both stdout and stderr as they happen.

Terminate command that has been elevated

I am trying to use command_runner to start a dhcp server which requires elevation without blocking my main program.
So, from my main program I call command_runner_threaded('python3 dhcp.py). Within dhcp.py, I call elevate(main).

This works fine but I cannot figure out how to stop the dhcp server from the main program. I tried cancelling the future returned by command_runner_threaded (returns True) but the dhcp server is still running (it is still sending offers, etc). I think cancelling a future is not correct here.

Is there a way to run a command that requires elevation without blocking the main program that I can later terminate?

Disabled all console outputs but still do

err_code, stdout, stderr = command_runner(params, timeout=1, encoding='utf-8', method='monitor', live_output=False, stdout=False, stderr=False, split_streams=True)

When timeout prints error to console:
"Timeout 1 seconds expired for command "..." execution. Original output was: None"

How to suspress outputs completely?