Wait for multiple processes at the same time

[pexpect]

Wait for multiple processes at the same time

[takluyver]

It's easy enough to start multiple processes to run at the same time, but pexpect can currently only wait for a pattern on one spawned process at a time. I've been thinking about integrating it with asyncio to make this kind of thing easier.

[calbare]

Thanks, do you have any idea on when this would happen? Also, does pexpect only work for python2 (not 3)? Thanks.

[takluyver]

It's not an immediate plan, I'm afraid. Pexpect does work on Python 2 and 3, though.

[takluyver]

Sourceforge feature request 5 asked for the same thing:

There is a feature of TCL-expect that I really miss on pexpect: being able to wait on a number of spawn children within a single command. I guess my way of better describing it would be by example:

child1 = pexpect.spawn ('ftp site1.org')
child2 = pexpect.spawn ('ftp site2.org')
child3 = pexpect.spawn ('ftp site3.org')

strExpect = 'Name .*: '

result, childMatch = expectSelect( (child1, strExpect), (child2, strExpect), (child3, strExpect) )

if not result:
    print "timed out on all children"

if (childMatch == 1):
    print "matched on child 1"

The main idea of this feature request is that we obtain a select behavior such that a single execution thread can continue as soon as the first match (or timeout) takes place in any of the children in a list provided.

TCL expect allows for that by providing a list of spawned ids in the -i flag. This is extremely useful
for performance reasons when dealing with multiple children that are spawned concurrently.

Thanks!

[jquast]

asyncio in upcoming release. This implements a Waiter pattern, but no such feature (multiplexing over multiple children) is provided.

[takluyver]

With an asyncio future interface, I think you should be able to multiplex over multiple child processes using functions in asyncio like asyncio.wait().

I haven't tried, but it should be something like this:

done, pending = yield from asyncio.wait([
    p1.expect('foo', async=True),
    p2.expect('bar', async=True)
], timeout=20, return_when=concurrent.futures.FIRST_COMPLETED)