Comments (5)
Hmm, you're right. I misremembered what imap
actually meant in this context. map_unordered
does seem to be a better fit in that case, given that aiomultiprocess doesn't really do anything "lazy" in processing tasks/results.
from aiomultiprocess.
I think this would be a relatively easy addition to pool.Pool. I'd ideally like to see it called imap()
to match the standard multiprocessing module, but would need to implement an unordered version of PoolResult
and/or Pool.results()
. Happy to give guidance if someone wants to work on this before I have the time.
from aiomultiprocess.
I think if we want to match multiprocessing
module naming then it should be imap()
and imap_unordered()
.
source: https://stackoverflow.com/a/26521507/8896457
from aiomultiprocess.
were you able to get this behavior from other library or any other way? @karolzlot
from aiomultiprocess.
I think I just added results to shared queue, a and processed from it in another process. You can find this solution in another issue in this repo.
from aiomultiprocess.
Related Issues (20)
- ClientConnectorError while using proxies in aiohttp requests with aiomultiprocess.Pool
- Pool not utilising available CPUs HOT 2
- cpu_count is NOT correct under container environment!
- aiomultiprocess never closes out the pool if there are more than a certain # number of items to process HOT 1
- Changing parameters didn't make me perform the task any faster HOT 1
- what do you mean by "AsyncIO still can't exceed the speed of GIL" HOT 1
- I use it in linux and windows and the program gets stuck after 1-2 days of running, see the process is not killed, it just hibernates
- Make log.exception the default but with a way to silence it HOT 1
- RuntimeError: HOT 1
- Worker process seems to keep restarting
- allow child jobs to schedule into the pool HOT 1
- Using multiprocessing.Value silently terminates the Process HOT 1
- Can't get the example Pool code to work HOT 1
- App is getting stuck, pool never closed with no exception in docker container
- _pickle.PicklingError: Can't pickle <function get at 0x12cb5ea60>: attribute lookup get on __main__ failed
- Add py.typed file to mark package PEP 561-compatible
- PoolTest:test_pool_worker_{exceptions,max_tasks,stop} are flaky and get the test suite stuck
- eventloop blocked on queue.get on subprocess death HOT 1
- Pass instance method to map/starmap
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aiomultiprocess.