python-gino/gino-starlette

An extension for GINO to support Starlette server.

gino-starlette's Issues

Failing to run with Alembic with FastAPI

Expected Behavior

I am trying to run the alembic make migrations and expected to succeed

Actual Behavior

exception is raised

Steps to Reproduce the Problem

1. clone https://github.com/ahmednafies/fastapi_gino.git
2. run make makemigrations

command
`alembic revision --autogenerate`

error
AttributeError: type object 'AsyncpgDBAPI' has no attribute 'connect'

Specifications

• Python version: 3.8.3
• GINO version: 1.0.1
• Starlette version: 0.13.4

Cannot release connection after query is canceled.

Expected Behavior

When canceling query, connection should be released back into pool.

Actual Behavior

After canceling query (and catching errors in code) gino_startlette stills throws error asyncpg.exceptions.QueryCanceledError when trying to release connection.

Steps to Reproduce the Problem

from asyncpg import QueryCanceledError
from fastapi import FastAPI
from gino_starlette import Gino

app = FastAPI()

db = Gino(
    app,
    kwargs=dict(server_settings=dict(statement_timeout="1")), # cancel query after 1 ms
)


@app.get("/")
async def root():
    message = "success"
    try:
        await db.scalar("SELECT 1")
    except QueryCanceledError:
        message = "timeout"

    return {"message": message}

When called, Endpoint will correctly return {"message": "timeout"} but app will still throw QueryCanceledError

Traceback (most recent call last):
  File "****/site-packages/uvicorn/protocols/http/httptools_impl.py", line 385, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "****/site-packages/uvicorn/middleware/proxy_headers.py", line 45, in __call__
    return await self.app(scope, receive, send)
  File "****/site-packages/fastapi/applications.py", line 181, in __call__
    await super().__call__(scope, receive, send)
  File "****/site-packages/starlette/applications.py", line 102, in __call__
    await self.middleware_stack(scope, receive, send)
  File "****/site-packages/starlette/middleware/errors.py", line 181, in __call__
    raise exc from None
  File "****/site-packages/starlette/middleware/errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "****/site-packages/gino_starlette.py", line 83, in __call__
    await conn.release()
  File "****/site-packages/gino/engine.py", line 300, in release
    await dbapi_conn.release(True)
  File "****/site-packages/gino/engine.py", line 48, in release
    return await self._release()
  File "****/site-packages/gino/engine.py", line 84, in _release
    await self._pool.release(conn)
  File "****/site-packages/gino/dialects/asyncpg.py", line 280, in release
    await self._pool.release(conn)
  File "****/site-packages/asyncpg/pool.py", line 654, in release
    return await asyncio.shield(ch.release(timeout))
  File "****/site-packages/asyncpg/pool.py", line 216, in release
    raise ex
  File "****/site-packages/asyncpg/pool.py", line 206, in release
    await self._con.reset(timeout=budget)
  File "****/site-packages/asyncpg/connection.py", line 1137, in reset
    await self.execute(reset_query, timeout=timeout)
  File "****/site-packages/asyncpg/connection.py", line 295, in execute
    return await self._protocol.query(query, timeout)
  File "asyncpg/protocol/protocol.pyx", line 316, in query
asyncpg.exceptions.QueryCanceledError: canceling statement due to statement timeout

Specifications

• Python version: 3.8
• GINO version: 1.0.1
• Starlette version: 0.1.1

Workarounds

I can suppress the error message when replacing this line
https://github.com/python-gino/gino-starlette/blob/master/src/gino_starlette.py#L83
with this:

try:
    await conn.release()
except QueryCanceledError:
    pass

But I am not quite sure if this is the best approach

pip install gino-starlette does not work

Expected Behavior

Describe what you were trying to get done and what you expected to happen.

Ran pip install gino-starlette and expected it to install gino-starlette

Actual Behavior

pip can not find the package

$ pip install gino-starlette
Collecting gino-starlette
Could not find a version that satisfies the requirement gino-starlette (from versions: )
No matching distribution found for gino-starlette

Steps to Reproduce the Problem

Collecting gino-starlette
  Could not find a version that satisfies the requirement gino-starlette (from versions: )
No matching distribution found for gino-starlette

Can confirm that it can connect to pypi:

$ pip install fastapi
Collecting fastapi
Downloading https://files.pythonhosted.org/packages/f9/eb/f899ccd1f052bb306d3938696458a403ee61a116fd784bc9fb266f6ce211/fastapi-0.53.0-py3-none-any.whl (47kB)
100% |████████████████████████████████| 51kB 916kB/s

Specifications

• Python version: 3.8.1
• GINO version: n/a
• Starlette version: n/a

Please Bump Starlette Version

Expected Behavior

No dependency conflicts with FastAPI

Actual Behavior

Newer FastAPI versions have incompatible Starlette version (72.0+)

Steps to Reproduce the Problem

1. Install FastAPI 72.0 or newer
2. See dependencies version error

Specifications

• Python version: ^3.9
• GINO version: n/a
• Starlette version: ^0.17.1
• FastAPI version: ^72.0

Use mypy to type check a project

Expected Behavior

Mypy should know the types from gino-starlette to check our creations against it

Actual Behavior

Mypy warns about it. Cannot infer the types because they are not declared

Steps to Reproduce the Problem

1. Clone sample project: https://github.com/python-gino/gino-starlette/tree/master/examples/prod_fastapi_demo
2. Install mypy
3. Execute mypy

mypy src/
src/gino_fastapi_demo/models/__init__.py:1: error: Skipping analyzing 'gino_starlette': found module but no type hints or library stubs
src/gino_fastapi_demo/models/__init__.py:1: note: See https://mypy.readthedocs.io/en/latest/running_mypy.html#missing-imports
Found 1 error in 1 file (checked 8 source files)

Specifications

• Python version: 3.9.1
• GINO version: 1.0.1
• Starlette version: 0.13.6
• gino-starlette: 0.1.1

Solution

Maybe we can do a package with the needed stubs like the one for sqlalchemy on pypi

I've generated a sample with the instructions provided at mypy documentation

from typing import Any, Optional

from fastapi.applications import FastAPI
from gino.api import Gino as _Gino
from gino.api import GinoExecutor as _Executor
from gino.engine import GinoConnection as _Connection
from gino.engine import GinoEngine as _Engine
from gino.strategies import GinoStrategy
from starlette.types import Receive as Receive
from starlette.types import Scope as Scope
from starlette.types import Send as Send

logger: Any

class StarletteModelMixin:
    @classmethod
    async def get_or_404(cls, *args: Any, **kwargs: Any): ...

class GinoExecutor(_Executor):
    async def first_or_404(self, *args: Any, **kwargs: Any): ...

class GinoConnection(_Connection):
    async def first_or_404(self, *args: Any, **kwargs: Any): ...

class GinoEngine(_Engine):
    connection_cls: Any = ...
    async def first_or_404(self, *args: Any, **kwargs: Any): ...

class StarletteStrategy(GinoStrategy):
    name: str = ...
    engine_cls: Any = ...

class _Middleware:
    app: Any = ...
    db: Any = ...
    def __init__(self, app: Any, db: Any) -> None: ...
    async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None: ...

class Gino(_Gino):
    model_base_classes: Any = ...
    query_executor: Any = ...
    config: Any = ...
    def __init__(
        self, app: Optional[FastAPI] = ..., *args: Any, **kwargs: Any
    ) -> None: ...
    def init_app(self, app: FastAPI) -> None: ...
    async def first_or_404(self, *args: Any, **kwargs: Any): ...
    async def set_bind(self, bind: Any, loop: Optional[Any] = ..., **kwargs: Any): ...

Prod FastAPI Demo Dockerfile doesn't build image

I try the prod fastAPI demo example on my own machine.

Expected Behavior

Demo run with docker compose

Actual Behavior

Image doesn't build correctly and docker compose cannot up.

Steps to Reproduce the Problem

1. Cloning the repo
2. Go to prod_fastapi_demo folder
3. run docker compose command

git clone https://github.com/python-gino/gino-starlette
cd gino-starlette/examples/prod_fastapi_demo
docker-compose up

Specifications

• Python version: 3.8
• GINO version: na
• Starlette version: na
• WSL 2 (Ubuntu 20.04 LTS)
• Docker : 20.10.2, build 2291f61
• Docker Compose: 1.29.2, build 5becea4c

starlette version bound too tightly in pyproject.toml

Expected Behavior

Can use gino-starlette with fresh FastAPI version (which depends on fresh starlette)

Actual Behavior

Dependency resolving fails

for "^0.19" the caret means that only 0.19.x patch versions are considered valid. This is different from "^1.2" where any 1.x version including and above 1.2 would be valid.

Steps to Reproduce the Problem

1. make package depend on gino[starlette]
2. try to depend on fastapi 0.88

to fix

Change https://github.com/python-gino/gino-starlette/blob/master/pyproject.toml#L26 to:

starlette = ">=0.22,<1.0"  # caret behaviour on 0.x is to lock to 0.x.*

While at it I would also change https://github.com/python-gino/gino-starlette/blob/master/pyproject.toml#L27 to:

gino = "^1.0"

so when 1.1 is released gino-starlette does not fail dependency resolution.

DB connection for request is not released when there are more than one database

With use_connection_for_request=True and two DBs:

db1.init_app(app)
db2.init_app(app)

And in the fastapi router, query db1 and db2.

1. Call the api
2. Change file contents to make it reload
3. Try exiting

Then you'll see the server gets stuck.

Please, bump Starlette version

Expected Behavior

No dependency conflicts with FastAPI

Actual Behavior

Newer FastAPI versions have incompatible Starlette version

Steps to Reproduce the Problem

1. Install FastAPI 0.69+ via pipenv
2. See dependencies version error

$ pipenv install
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
 Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: Could not find a version that matches starlette<0.15.0,==0.16.0,>=0.13.0 (from -r /var/folders/7y/z1m2c48104b74g3qwbsjr5sc0000gq/T/pipenvwmto7eg4requirements/pipenv-mdqpmq80-constraints.txt (line 7))
Tried: 0.1.0, 0.1.1, 0.1.2, 0.1.3, 0.1.4, 0.1.5, 0.1.6, 0.1.7, 0.1.8, 0.1.9, 0.1.10, 0.1.11, 0.1.12, 0.1.13, 0.1.14, 0.1.15, 0.1.16, 0.1.17, 0.2.0, 0.2.1, 0.2.2, 0.2.3, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.3.5, 0.3.6, 0.3.7, 0.4.0, 0.4.1, 0.4.2, 0.5.0, 0.5.1, 0.5.2, 0.5.3, 0.5.4, 0.5.5, 0.6.0, 0.6.1, 0.6.2, 0.6.3, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.7.4, 0.8.0, 0.8.1, 0.8.2, 0.8.3, 0.8.4, 0.8.5, 0.8.6, 0.8.7, 0.8.8, 0.9.0, 0.9.1, 0.9.2, 0.9.3, 0.9.4, 0.9.5, 0.9.6, 0.9.7, 0.9.8, 0.9.9, 0.9.10, 0.9.11, 0.10.0, 0.10.1, 0.10.2, 0.10.3, 0.10.4, 0.10.5, 0.10.6, 0.10.7, 0.11.0, 0.11.1, 0.11.2, 0.11.3, 0.11.4, 0.12.0, 0.12.1, 0.12.2, 0.12.3, 0.12.4, 0.12.5, 0.12.6, 0.12.7, 0.12.8, 0.12.9, 0.12.10, 0.12.11, 0.12.12, 0.12.13, 0.13.0, 0.13.1, 0.13.1, 0.13.2, 0.13.2, 0.13.3, 0.13.3, 0.13.4, 0.13.4, 0.13.5, 0.13.5, 0.13.6, 0.13.6, 0.13.7, 0.13.7, 0.13.8, 0.13.8, 0.14.0, 0.14.0, 0.14.1, 0.14.1, 0.14.2, 0.14.2, 0.15.0, 0.15.0, 0.16.0, 0.16.0, 0.17.0, 0.17.0
Skipped pre-versions: 0.12.0b1, 0.12.0b2, 0.12.0b3
There are incompatible versions in the resolved dependencies:
  starlette (from -r /var/folders/7y/z1m2c48104b74g3qwbsjr5sc0000gq/T/pipenvwmto7eg4requirements/pipenv-mdqpmq80-constraints.txt (line 7))
  starlette<0.15.0,>=0.13.0 (from gino-starlette==0.1.2->-r /var/folders/7y/z1m2c48104b74g3qwbsjr5sc0000gq/T/pipenvwmto7eg4requirements/pipenv-mdqpmq80-constraints.txt (line 4))
  starlette==0.16.0 (from fastapi==0.70.0->-r /var/folders/7y/z1m2c48104b74g3qwbsjr5sc0000gq/T/pipenvwmto7eg4requirements/pipenv-mdqpmq80-constraints.txt (line 3))

Specifications

• Python version: 3.9.7
• GINO version: na
• Starlette version: na

gino-starlette mysql

use [gino-mysql-support](https://github.com/wwwjfy/gino/tree/mysql-support ,"gino-mysql-support")

Run error：
TypeError: Invalid argument(s) 'min_size','max_size' sent to create_engine(), using configuration AiomysqlDialect/GinoEngine.
Please check that the keyword arguments are appropriate for this combination of components.

After repair
rv = await User.create(nickname=user.nickname) Unable to commit to database ，Need to add “autocommit”=True

At last

Upgrade Starlette version

Expected Behavior

No dependency conflict

Actual Behavior

New Fastapi version causes dependency versions conflict

Steps to Reproduce the Problem

1. Try to freeze dependencies with Fastapi >= 0.76.0 and gino-starlette
2. Get an error (pip-tools for example)
   

Btw, there are newer versions of starlette: 0.20.*
tiangolo/fastapi#4820
tiangolo/fastapi#4936
So, if you just change ^0.17 to ^0.19, it wont work for long

Specifications

• Python version: 3.10
• GINO version: 1.0.1
• Starlette version: 0.19.1

Cannot connect to the database; max retries reached.

Expected Behavior

Describe what you were trying to get done and what you expected to happen.
Cannot connect to the database; max retries reached.
ERROR: Traceback (most recent call last):
File "/home/dilshod/Desktop/dev_appointment/my_env/lib/python3.7/site-packages/starlette/routing.py", line 526, in lifespan
async for item in self.lifespan_context(app):
File "/home/dilshod/Desktop/dev_appointment/my_env/lib/python3.7/site-packages/starlette/routing.py", line 467, in default_lifespan
await self.startup()
File "/home/dilshod/Desktop/dev_appointment/my_env/lib/python3.7/site-packages/starlette/routing.py", line 502, in startup
await handler()
File "/home/dilshod/Desktop/dev_appointment/my_env/lib/python3.7/site-packages/gino_starlette.py", line 183, in startup
**config["kwargs"],
File "/home/dilshod/Desktop/dev_appointment/my_env/lib/python3.7/site-packages/gino_starlette.py", line 226, in set_bind
return await super().set_bind(bind, loop=loop, **kwargs)
File "/home/dilshod/Desktop/dev_appointment/my_env/lib/python3.7/site-packages/gino/api.py", line 417, in set_bind
bind = await create_engine(bind, loop=loop, **kwargs)
File "/home/dilshod/Desktop/dev_appointment/my_env/lib/python3.7/site-packages/gino/strategies.py", line 55, in create
pool = await dialect.init_pool(u, loop, pool_class=pool_class)
File "/home/dilshod/Desktop/dev_appointment/my_env/lib/python3.7/site-packages/gino/dialects/asyncpg.py", line 465, in init_pool
return await pool_class(url, loop, init=self.on_connect(), **self._pool_kwargs)
File "/home/dilshod/Desktop/dev_appointment/my_env/lib/python3.7/site-packages/gino/dialects/asyncpg.py", line 218, in _init
self.pool = await asyncpg.create_pool(**args)
File "/home/dilshod/Desktop/dev_appointment/my_env/lib/python3.7/site-packages/asyncpg/pool.py", line 398, in async__init
await self._initialize()
File "/home/dilshod/Desktop/dev_appointment/my_env/lib/python3.7/site-packages/asyncpg/pool.py", line 426, in _initialize
await first_ch.connect()
File "/home/dilshod/Desktop/dev_appointment/my_env/lib/python3.7/site-packages/asyncpg/pool.py", line 125, in connect
self._con = await self._pool._get_new_connection()
File "/home/dilshod/Desktop/dev_appointment/my_env/lib/python3.7/site-packages/asyncpg/pool.py", line 472, in _get_new_connection
**self._connect_kwargs)
File "/home/dilshod/Desktop/dev_appointment/my_env/lib/python3.7/site-packages/asyncpg/connection.py", line 1727, in connect
max_cacheable_statement_size=max_cacheable_statement_size)
File "/home/dilshod/Desktop/dev_appointment/my_env/lib/python3.7/site-packages/asyncpg/connect_utils.py", line 674, in _connect
raise last_error
File "/home/dilshod/Desktop/dev_appointment/my_env/lib/python3.7/site-packages/asyncpg/connect_utils.py", line 666, in _connect
connection_class=connection_class)
File "/home/dilshod/Desktop/dev_appointment/my_env/lib/python3.7/site-packages/asyncpg/connect_utils.py", line 633, in _connect_addr
connector, timeout=timeout)
File "/usr/local/lib/python3.7/asyncio/tasks.py", line 412, in wait_for
return fut.result()
File "uvloop/loop.pyx", line 1914, in create_connection
socket.gaierror: [Errno -2] Name or service not known

ERROR: Application startup failed. Exiting.

I could not connect my database.

Can I use Gino[starlette] with SQLite?

Tests using SQLite

Hi all, I'm trying to make the Gino mock, but I keep seeing the error
gino.exceptions.UninitializedError: Gino engine is not initialized.

Code

My code is formed like this:

# __init__.py
@functools.lru_cache
def get_db_service():
    db = Gino(dsn=settings.get_settings().postgresql_conn_url)

    return db

# model 

_db = get_db_service()

class EdaTableInstance(_db.Model):
    __tablename__ = "eda_table_instance"
#...

   @classmethod
    async def get_all(cls) -> List['EdaTableInstance']:
        async with _db.acquire():
              return await EdaTableInstance.query.gino.all()

Now let's see how I'm writing the tests (various attempts)

# conftest.py
@pytest.fixture(autouse=True)
def mock_get_db_service(mocker):
    db = Gino(dsn="sqlite//:memory:")
    async_mock = AsyncMock(db)
    mocker.patch("gateway_api.services.get_db_service", return_value=async_mock)
    yield

or

# conftest.py

@pytest.fixture
async def db_initialize():
   await db.set_bind('sqlite:///:memory:')
   await db.gino.create_all()
   await EdaTableInstance.create_eda_table_instance(
       EdaTableInstanceInputOnCreate({"name":"table_server1", "host":"123.123.123.123"})
   )
   yield

or

# test_models.py
@pytest.fixture
def mock_gino_get_all(mocker):
    mocker.patch("gino.api.GinoExecutor.all", return_value=[])

@pytest.mark.asyncio
@pytest.mark.parametrize("id, expected", [(None, None)])
async def test_01_table_instance_get_all(id, expected):
    mock_cursor = MagicMock()
    mock_cursor.configure_mock(
        **{
            "get_one.return_value":[id]
        }
    )
    res = await EdaTableInstance().get_one(mock_cursor)
    assert res == expected

I would like to use SqLite in memory, so I don't have to connect from a database, if you know better methods to mock the database, it would help me so much.
Thank you.

Specifications

• Python version: 3.9.5
• GINO version: 1.0.1
• Gino Starlette: 0.1.5
• Starlette version: 0.16.0

Need support for the latest version of Starlette

I've found that the latest release of gino-stralette (0.1.1) doesn't support the latest version of Starlette (0.14.2).
It requires >0.14.0.

gino.exceptions.UninitializedError: Gino engine is not initialized.

Steps to Reproduce the Problem

1. gino.exceptions.UninitializedError: Gino engine is not initialized.
   

2. view
   
   3 settings
   

3. model
   

Specifications

• Python version: 3.6.5
• GINO version:0.8.5
• Starlette version:0.13.1