Comments (10)
We'll probably be able to get some level of support going. I'll try to take a look soon.
from asyncpg.
Sure, if you use separate connections (which you are), it is possible to run an arbitrary number of queries concurrently. That's basically the main feature of asyncpg :-).
Now, whether or not the queries actually execute concurrently depends on the locking that your queries cause. For example, if you run ALTER TABLE
on the same table concurrently, the execution will actually be sequential since ALTER TABLE locks the whole table on Postgres side.
Thanks for this library
Cheers!
from asyncpg.
Thanks a lot @elprans for you response.
I came across an issue using the simple example from the documentation :
import asyncio
import asyncpg
async def connect_to_db():
pool = await asyncpg.create_pool("postgres://user:pass@localhost:5555/dev",
command_timeout=60)
conn1 = await pool.acquire()
try:
a = await conn1.fetch("""SELECT 1;""")
print(a)
finally:
await pool.release(conn1)
loop = asyncio.get_event_loop()
loop.run_until_complete(connect_to_db())
Here is the stacktrace :
[<Record ?column?=1>]
Traceback (most recent call last):
File "test.py", line 18, in <module>
loop.run_until_complete(connect_to_db())
File "/Users/nolanemirot/miniconda/envs/pyt35/lib/python3.5/asyncio/base_events.py", line 387, in run_until_complete
return future.result()
File "/Users/nolanemirot/miniconda/envs/pyt35/lib/python3.5/asyncio/futures.py", line 274, in result
raise self._exception
File "/Users/nolanemirot/miniconda/envs/pyt35/lib/python3.5/asyncio/tasks.py", line 241, in _step
result = coro.throw(exc)
File "test.py", line 14, in connect_to_db
await pool.release(conn1)
File "/Users/nolanemirot/miniconda/envs/pyt35/lib/python3.5/site-packages/asyncpg/pool.py", line 184, in release
await connection.reset()
File "/Users/nolanemirot/miniconda/envs/pyt35/lib/python3.5/site-packages/asyncpg/connection.py", line 329, in reset
''')
File "/Users/nolanemirot/miniconda/envs/pyt35/lib/python3.5/site-packages/asyncpg/connection.py", line 120, in execute
return await self._protocol.query(script, timeout)
File "asyncpg/protocol/protocol.pyx", line 224, in query (asyncpg/protocol/protocol.c:46945)
File "/Users/nolanemirot/miniconda/envs/pyt35/lib/python3.5/asyncio/futures.py", line 361, in __iter__
yield self # This tells Task to wait for completion.
File "/Users/nolanemirot/miniconda/envs/pyt35/lib/python3.5/asyncio/tasks.py", line 296, in _wakeup
future.result()
File "/Users/nolanemirot/miniconda/envs/pyt35/lib/python3.5/asyncio/futures.py", line 274, in result
raise self._exception
asyncpg.exceptions.PostgresSyntaxError: syntax error at or near "ALL"
Looking at the source code the error https://github.com/MagicStack/asyncpg/blob/master/asyncpg/exceptions/__init__.py#L554 I don't understand the relation with a postgres syntax error. (I'm connecting to a redshift cluster)
Replacing await pool.release(conn1)
with pool.close()
doesn't trigger any error.
Do you have any idea why ?
from asyncpg.
This is triggered by Connection.reset()
: https://github.com/MagicStack/asyncpg/blob/master/asyncpg/connection.py#L349
It is likely that Redshift's version of postgres doesn't support either CLOSE ALL
or RESET ALL
. Which version is running there?
from asyncpg.
Calling SELECT VERSION();
gives me
PostgreSQL 8.0.2 on i686-pc-linux-gnu, compiled by GCC gcc (GCC) 3.4.2 20041017 (Red Hat 3.4.2-6.fc3), Redshift 1.0.1096
from asyncpg.
Ouch! That's ancient! That version was released over 10 years ago.
from asyncpg.
Yes it is: https://docs.aws.amazon.com/redshift/latest/dg/c_redshift-and-postgres-sql.html
from asyncpg.
OK, we'll need to find a way to close the cursors on that version. Meanwhile, you can just drop the CLOSE ALL;
line from that statement.
from asyncpg.
Are there any updates to this issue? I've tried the proposed workaround (drop the CLOSE ALL;) but it doesn't help.
from asyncpg.
Interested in asyncpg for Redshift too.
@elprans @emirot Just to understand : asyncpg does not work at all with redshift, or only connection pooling ?
from asyncpg.
Related Issues (20)
- Support a Pool `before_connect` callback to be used before a connection is created HOT 1
- Use anyio instead of asyncio for trio compatibility
- Support typed prepared statements
- issue with connection pooling when using QueuePool (via sqlalchemy) HOT 2
- Not using transactions but receiving errors with: Resetting connection with an active transaction
- Feature request: Make transaction state public
- no attribute `_aborted` on `Connection.__del__` HOT 1
- `inspect.iscoroutinefunction` returns false on pool -> connection -> execute HOT 1
- Query does not processed HOT 3
- connect through socks5 proxy HOT 2
- Error: there is no unique or exclusion constraint matching the ON CONFLICT specification HOT 2
- Introspection TYPE_BY_OID running for long time HOT 3
- invalid input for query argument (can't subtract offset-naive and offset-aware datetimes)
- Feature request: Support for asynchronous command processing
- Asyncpg (asyncio/asyncpg.py, line 62, in get_query) get_query throws index error.
- no binary format encoder for type jsonb
- Whenever i create a table with a `real` column and read the value i get something different then what i saved
- insufficient data in buffer
- GCP CloudSQL Connector with asyncpg Pool
- cursor prefetch with bounded lag
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from asyncpg.