Giter Site home page Giter Site logo

stdio-mgr's Introduction

stdio Manager: Context manager for mocking/wrapping stdin/stdout/stderr

Current Development Version:

https://travis-ci.org/bskinn/stdio-mgr.svg?branch=master

Most Recent Stable Release:

Info:


Have a CLI Python application?

Want to automate testing of the actual console input & output of your user-facing components?

stdio Manager can help.

While some functionality here is more or less duplicative of redirect_stdout and redirect_stderr in contextlib within the standard library, it provides (i) a much more concise way to mock both stdout and stderr at the same time, and (ii) a mechanism for mocking stdin, which is not available in contextlib.

First, install:

$ pip install stdio-mgr

Then use!

All of the below examples assume stdio_mgr has already been imported via:

>>> from stdio_mgr import stdio_mgr

Mock stdout:

>>> with stdio_mgr() as (in_, out_, err_):
...     print('foobar')
...     out_cap = out_.getvalue().replace(os.linesep, '\n')
>>> out_cap
'foobar\n'
>>> in_.closed and out_.closed and err_.closed
True

By default print appends a newline after each argument, which is why out_cap is 'foobar\n' and not just 'foobar'.

As currently implemented, stdio_mgr closes all three mocked streams upon exiting the managed context.

Mock stderr:

>>> import warnings
>>> with stdio_mgr() as (in_, out_, err_):
...     warnings.warn("foo has no bar")
...     err_cap = err_.getvalue().replace(os.linesep, '\n')
>>> err_cap
'...UserWarning: foo has no bar\n...'

Mock stdin:

The simulated user input has to be pre-loaded to the mocked stream. Be sure to include newlines in the input to correspond to each mocked Enter keypress! Otherwise, input will hang, waiting for a newline that will never come.

If the entirety of the input is known in advance, it can just be provided as an argument to stdio_mgr. Otherwise, .append() mocked input to in_ within the managed context as needed:

>>> with stdio_mgr('foobar\n') as (in_, out_, err_):
...     print('baz')
...     in_cap = input('??? ')
...
...     _ = in_.append(in_cap[:3] + '\n')
...     in_cap2 = input('??? ')
...
...     out_cap = out_.getvalue().replace(os.linesep, '\n')
>>> in_cap
'foobar'
>>> in_cap2
'foo'
>>> out_cap
'baz\n??? foobar\n??? foo\n'

The _ = assignment suppresses printing of the return value from the in_.append() call--otherwise, it would be interleaved in out_cap, since this example is shown for an interactive context. For non-interactive execution, as with unittest, pytest, etc., these 'muting' assignments should not be necessary.

Both the '??? ' prompts for input and the mocked input strings are echoed to out_, mimicking what a CLI user would see.

A subtlety: While the trailing newline on, e.g., 'foobar\n' is stripped by input, it is retained in out_. This is because in_ tees the content read from it to out_ before that content is passed to input.

Want to modify internal print calls within a function or method?

In addition to mocking, stdio_mgr can also be used to wrap functions that directly output to stdout/stderr. A stdout example:

>>> def emboxen(func):
...     def func_wrapper(s):
...         from stdio_mgr import stdio_mgr
...
...         with stdio_mgr() as (in_, out_, err_):
...             func(s)
...             content = out_.getvalue()
...
...         max_len = max(map(len, content.splitlines()))
...         fmt_str = '| {{: <{0}}} |\n'.format(max_len)
...
...         newcontent = '=' * (max_len + 4) + '\n'
...         for line in content.splitlines():
...             newcontent += fmt_str.format(line)
...         newcontent += '=' * (max_len + 4)
...
...         print(newcontent)
...
...     return func_wrapper

>>> @emboxen
... def testfunc(s):
...     print(s)

>>> testfunc("""\
... Foo bar baz quux.
... Lorem ipsum dolor sit amet.""")
===============================
| Foo bar baz quux.           |
| Lorem ipsum dolor sit amet. |
===============================

Available on PyPI (pip install stdio-mgr).

Source on GitHub. Bug reports and feature requests are welcomed at the Issues page there.

Copyright (c) 2018-2019 Brian Skinn

License: The MIT License. See LICENSE.txt for full license terms.

stdio-mgr's People

Contributors

bskinn avatar jayvdb avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

stdio-mgr's Issues

Testing against console libraries

colorama < 0.4 is incompatible, as its StreamWrapper only gained context manager features in 0.4 tartley/colorama@2f4b564 and it pushes instances of its StreamWrapper into sys.std*

We could potentially work around this by detecting and patching its objects, but it probably isnt worth the effort. coala was forced to use < 0.4 because of radon, but they have bumped their min dependency, so we will too. Not many projects will be still wanting colorama < 0.4.

So this is probably best addressed with documentation of known incompatibilities.

However, it would be worth checking out other console libraries, and creating tests for each.

Stdlib:

  • doctest (implictly tested with doctests)
  • readline/pyreadline/anyreadline

Other:

  • colorama
  • click
  • tqdm
  • cleo
  • better-exceptions

Add 'echo' multistate toggle capability to stdio_mgr?

Some users may not want the content read at stdin to be echoed. This should be pretty straightforward.

Some may not want any stdin prompts to be echoed to the mocked stdout. This would be considerably more challenging, if not impossible, to achieve.

Might be doable by overriding input in the relevant frame? Messy, though.

Add flake8-2020

Expected not used here, but good to check & participate in initial adoption

Timestamped index to buffer

#64 (comment) mentions merging stdout and stderr to reflect that both typically appear on the same user console, interleaved.

Another similar feature is timestamping the activity occurring on the "console", so those timestamps can be shown to the user, or timing info used in test logic.

Add option to capture stdout and stderr to the same stream?

If an operation generates mixed stdout/stderr output, it's basically impossible to reconstruct what would have been pushed to the screen in the correct order.

A new class analogous to TeeStdin could probably be implemented that would tee one output to the other.

Definitely stderr --> stdout is of interest. Are there any situations where stdout --> stderr would be valuable?

Add typing

As noted by jayvdb in #53 -- potentially valuable for use-cases where stdio-mgr context managers are instantiated & passed through layers of logic/function calls before/during/after(?) use.

Cannibalize coala-utils ContextManagers

There are some useful bits of coala-utils ContextManagers which may be desirable features here, especially simulate_console_inputs which is extensively used in https://github.com/coala/coala/ testsuite.

Most of the concepts in coala-utils ContextManagers are likely already pending issues here, or can be achieved using the existing functionality. I envisage this issue here is mostly to map coala-utils to stdio-mgr, discuss and track progress.

Created https://gitlab.com/coala/coala-utils/issues/88 for the other side, once most of the necessary enhancements are done here. There is no need to replicate exactly the same functionality, or do it all in one release. coala would contain to use its own wrapper, but it would become a thin layer to stdio-mgr , and possibly (lightly) deprecating some bits where the coala-utils tool has no benefits over the stdio-mgr replacement.

Stdlib logging references sys.error on import

As special case of #71 , and likely to be one of several cases, stdlib logging creates a reference to sys.error on import. As a result the following fairly common scenario of using a main() to test a cli without subprocesses fails if logging is first imported within the context:

from stdio_mgr import StdioManager

def invoke_cli_main():
    import logging
    logging.basicConfig()
    ...

def test_cli():
    with StdioManager():
        invoke_cli_main()
    import logging
    logging.warning('BOOM')
>>> test_cli()
--- Logging error ---
Traceback (most recent call last):
  File "/usr/lib64/python3.7/logging/__init__.py", line 1037, in emit
    stream.write(msg + self.terminator)
  File "/home/jayvdb/projects/python/stdio-mgr/src/stdio_mgr/stdio_mgr.py", line 90, in write
    super().write(*args, **kwargs)
ValueError: I/O operation on closed file.
Call stack:
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 5, in test_cli
Message: 'BOOM'
Arguments: ()

After the context handler exits, the wrapper streams are closed, and the logging is still using them.

Easy solution: stdio_mgr imports logging, like we would need to for #43

Even the following doesnt dislodge the problem:

def invoke_cli_main():
    import logging
    logging.basicConfig()
    ...
    from logging.config import _clearExistingHandlers
    _clearExistingHandlers()

To clear it properly we need to do:

from stdio_mgr import StdioManager

def invoke_cli_main():
    import logging
    logging.basicConfig()
    ...
    from logging.config import _clearExistingHandlers
    _clearExistingHandlers()
    logging.Logger.root = logging.root = logging.RootLogger(logging.WARNING)
    logging.Logger.manager = logging.Manager(logging.Logger.root)

def test_cli():
    with StdioManager():
        invoke_cli_main()
    import logging
    logging.info('OK')

test_cli()

or the following also seems to work

def invoke_cli_main():
    import logging
    logging.basicConfig()
    ...
    from logging.config import _clearExistingHandlers
    _clearExistingHandlers()
    logging.Logger.manager._clear_cache()

We can hide that logging cleanup inside of StdioManager.
It also works when import colorama is added to that mixture, but I do not feel comfortable that it is solved.

Thankfully there doesnt seem to be too many references to sys.std* in the stdlib, so we might be able to check and workaround them all, whenever feasible and appropriate.

Does stdio_mgr play nice in pytest?

My unittest runner framework doesn't play nice with pytest -- it appears pytest doesn't have a way to blindly pass commandline arguments through to my tests.py?

So, I'll be most grateful for anyone who posts here success/fail stories of attempts to use stdio_mgr with pytest.

Per here, it looks like pytest already natively supports stdout and stderr capture, so that's of less interest. I don't know if it has a mechanism for feeding content to a mocked stdin, though.

Debug level logging

For our own development needs, logging would be helpful, as print doesnt work.

But real-time logging of activity on the fake stdio is a useful feature as often tools doing that only capture stdout and stderr, and spit them both out at the end.

Consider explicit deletion of stream objects on exit

Depends on the balance between the number of times stdio_mgr is used, the frequency of calls, and the frequency of suitable garbage collection.

Prompted by this SO answer. That example uses a class for the context manager, though, with a recommendation to retain the instance for storing the captured stream contents. With stdio_mgr being implemented as a function, especially one where the streams are closed on context exit, there may not be as big of a problem with memory consumption.

stdio_mgr leaves REPL in munged state after exception inside context

Demo, from the newly merged master @ 48527c6:

>>> from stdio_mgr import stdio_mgr
>>> with stdio_mgr(close=False) as (i, o, e):
...     1/0
...
>>> 1/0
>>> i
>>> o
>>> e
>>> dir(o)
>>> 1 + '1'
>>> import sys
>>> sys.stdout = sys.__stdout__
>>> sys.stderr = sys.__stderr__
>>> print('abd')
abd
>>> 1/0
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ZeroDivisionError: division by zero
>>> 

The problem predates the recent changes toward v2.0. With v1.0.1 checked out:

>>> from stdio_mgr import stdio_mgr
>>> with stdio_mgr() as (i, o, e):
...     1/0
...
>>> i
>>> o
>>> e
>>> 1/0
>>> print('abcd')
>>> 

It would seem that the "__exit__" code after the yield in the decorated stdio_mgr function is not actually being run? Perhaps it is very important to implement #10??

May need to rewrite stdio_mgr as explicit class, to allow better curation of stream contents on error in managed code?

Definitely don't want stdio_mgr to swallow any information in the enclosing code that might help to diagnose an error.

Exception information should propagate back out without a problem, since the traceback & message &c. are encapsulated in the Exception that was raised.

However, the input/output itself may be informative, and as currently written that content is just dumped when the streams are closed.

Could either:

  1. Push the stdout/stderr mock streams to their respective real streams on error, and do... something?... with the contents of the stdin mock; or,
  2. Implement a custom Exception that holds the contents of all three streams (or--all three streams themselves), and raise from whatever exception was raised.

Augment tests & CI to allow running non-warning tests with warnings as errors

The universal invocation of pytest with -p no:warnings may be now, and will likely at various points in the future, mask warnings (e.g., deprecations) that should show up as errors in CI.

So, the tests/test-suite will need to be modified so that tests intentionally involving warnings don't spuriously cause CI to fail.

Probably the best approach is something like a flag to skip tests (or individual asserts) that raise warnings as part of their intended actions, and then:

  1. Run most of the CI jobs with -p no:warnings and this subset of tests enabled
  2. Run a selected CI job without -p no:warnings and this subset of tests disabled

Document and test direct use of stdio_mgr, not as a context manager

jayvdb:

Also since you mentioned __enter__. We should have some tests and documented behaviour for using stdio-mgr without with. i.e. on Py37

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: cannot unpack non-iterable _GeneratorContextManager object
>>> m = stdio_mgr(close=False)
>>> m
<contextlib._GeneratorContextManager object at 0x7f7d6b8d3be0>
>>> dir(m)
['__abstractmethods__', '__call__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__enter__', '__eq__', '__exit__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__slots__', '__str__', '__subclasshook__', '__weakref__', '_abc_impl', '_recreate_cm', 'args', 'func', 'gen', 'kwds']
>>> list(m.gen)
[(TeeStdin(tee=<_io.TextIOWrapper encoding='utf-8'>, init_text='', _encoding='utf-8'), <_io.TextIOWrapper encoding='utf-8'>, <_io.TextIOWrapper encoding='utf-8'>)]
>>> m = stdio_mgr(close=False)
>>> (i, o, e) = list(m.gen)[0]
>>> i
TeeStdin(tee=<_io.TextIOWrapper encoding='utf-8'>, init_text='', _encoding='utf-8')
>>> o
<_io.TextIOWrapper encoding='utf-8'>
>>> e
<_io.TextIOWrapper encoding='utf-8'>
>>> o.buffer
<_io.BufferedRandom>
>>> o.buffer.raw
<stdio_mgr.stdio_mgr._PersistedBytesIO object at 0x7f7d6b985200>

If we did change stdio_mgr to a class, the resulting class would be like a text fifo between i and o?
How can it be useful? They can access it anyway using the above loop-hole.
This might also help when trying to come up with a class name for it, if the function is to be replaced with a class.
Or can we close the loop-hole, unless moving to a class.

If someone really wants to use it this way, they probably know enough about context managers and Python objects that they don't need stdio_mgr converted into a class (cf. #10). But, if using the machinery this way enables significant/valuable new/different/unexpected usage of stdio_mgr generally, then it's probably worth at least documenting & testing it, even if it doesn't warrant going all the way to #10.

Windows CI

This library needs Windows CI. It could be AppVeyor, Azure, and the Travis Windows beta is also worth considering.

My recent thoughts on them at https://github.com/storyscript/storyscript/issues/1037 , but you need to make a decision. I'm happy to implement whichever you prefer, or wait until you've done it.

How to handle repeated stream closure in various circumstances?

With #53, #64 et al. allowing instantiation of StdioManager objects with a useful lifetime before and after use as a context manager, as well as independent of a managed context, the question arises as to what the behavior should be when a given stream in the internal tuple is .close()d more than once.

At this point, I believe a repeated .close() on a stream will raise ValueError? In some situations this is probably desired; but in others it might not be.

Or, for consistency, the behavior should perhaps always be the same.

This is loosely related to #42, in that the exception behavior is more a part of the API than not.

Re-evaluate/re-design capturing stderr by default

For our own debugging, it is useful to have one stream to write debugging to, if there is no logging (#43)

This is also useful if a program is only expecting activity on stdout; capturing stderr could hide errors the caller should have propagated out to the developer/user.

Structure of v2.0 public API

For now, this assumes that stdio_mgr will remain a @contextmanager-decorated function, rather than a full StdioMgr class.

stdio_mgr must be public.

IIUC the main other consideration is whether to make RandomTextIO and TeeStdin themselves public; or, to make them private but provide a public means for users to identify whether they are in use, or whether a given stdio stream is (most likely) the real one from sys.

The latter could be achieved by, e.g., a public stub superclass.

@bskinn best stab at a summary of the options:

  • Keep them public
    • Advantages
      • TeeStdin was implicitly public in v1.0.x
    • Disadvantages
      • A risk here is possibly making it harder to change the API/signature of RandomTextIO/TeeStdin if anything other than optional keyword arguments are needed.
      • There is also the additional thought that would have to go into making sure the classes are semantically configured well for public use... good names, useful & documented APIs & behaviors, etc. Advantage
  • Make them private
    • Advantages
      • Behavior and API become implementation detail and can be freely changed
      • No external need to carefully document or name the classes
      • Users could isinstance check on a single public class, rather than having to remember both classes
    • Disadvantages
      • No external need to carefully document or name the classes
      • Pulls TeeStdin out of ~public API, a break from v1.0.x
      • Expands the inheritance structure of the package

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.