Comments (6)
This is why I don't like concurrent.futures
API. They look simple&dandy but they lead to situations like these where namespaces get polluted.
I don't think coming up with a custom enough keyword is the correct solution. I think that there should be a clear distinction between what you send to the Pool and what to your scheduled function.
Another solution I was entertaining was to add the timeout
as a Pool attribute but it makes the logic cumbersome when you need to specify different timeouts per each call.
# This is also non-thread safe
pool.timeout = 5
pool.submit(fn, *args, **kwargs)
I think I will re-instate the schedule
function as the recommended one (now deprecated) and expose the submit
one just to support the asyncio.loop.run_in_executor(fn, TIMEOUT, *args)
Use Case.
This is the second issue (#110) which clearly means the new API is not good enough.
from pebble.
@noxdafox Actually, I just remembered position-only parameters, which solves this case:
def test(timeout, /, **kwargs):
print(timeout)
print(kwargs)
test(3, timeout=10)
Prints:
3
{'timeout': 10}
from pebble.
Yes, that would be the correct way to handle your case if you have Python >= 3.9.
Yet it is not backwards compatible and does not solve the issue highlighted in #110 where if you don't need to specify a timeout, you need to schedule your function as submit(fn, None, 'foo', 'bar', 'baz')
.
The ideal solution would be to have pattern-matching based function declarations such as Erlang or Elixir languages allow.
def submit(fn, /, *args, **kwargs):
schedule_without_timeout()
def submit(fn, timeout, /, *args, **kwargs):
schedule_with_timeout()
from pebble.
3.8 actually, but your point still stands; compatibility matters so it's not fixable that way for now.
Perhaps it could be argued that since 3.7 is reaching EOL in 8 months it might make sense to implement it this way in 8 months? (and bump the major version with it)
It indeed still wouldn't solve #110, but I would say that's a totally different issue. The only way I can see that fixes both issues and keeps a call-specific timeout for submit
calls is a specific kwarg that defaults to None
, but as you said that isn't the cleanest either.
from pebble.
I agree with @PLPeeters and much prefer adding a pebble-specific keyword argument (e.g. pebble_timeout
) to lower the barriers to entry for people familiar with concurrent.futures (though I see such compatibility is not Pebble's focus) and simplify the API (instead of keeping both submit
and schedule
).
from pebble.
Release 5.0.3
reinstates schedule
in place of submit
.
submit
is left to ensure compatibility with asyncio.loop.run_in_executor
.
from pebble.
Related Issues (20)
- Pebble wheels pushed to PyPI are incorrectly tagged HOT 3
- The timeout argument of ProcessPool().submit() is inconsistent with ThreadPool().submit() HOT 3
- Logging process name inside concurrent.process HOT 2
- How to handle errors when using pool.schedule HOT 1
- shutdown of main program HOT 2
- @concurrent.process returned future blocks/hangs on running(), cancelled(), done() calls HOT 2
- Get information about broken process HOT 4
- Documentation for Pebble indicates threads created with a ThreadPool are cancellable HOT 1
- Channel mutex timeout HOT 6
- How can I use a multiprocessing.manager alongside with pebble to avoid re-importing the function everytime? HOT 1
- Type hint error of wrapped function HOT 7
- ISSUE with with ProcessPool when scheduled function return exception (not raise it) HOT 1
- BUG with ProcessPool when scheduled function raise BaseException HOT 1
- issue with handling frozen exceptions in worker thread or process HOT 1
- Cannot create pebble.ProcessPool() multiple times within a single run, all task got stuck and time out HOT 3
- error while running @concurrent.process repetitively HOT 4
- using multiprocess context (for dill support) no longer working HOT 2
- Workers using 100% CPU not getting killed after timeout HOT 11
- Behavior of Process.map with chunksize > 1 and a timeout HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pebble.