laudio/pyodbc

Docker image with python 3.11 and pyodbc. Includes ODBC drivers for MSSQL, PostgreSQL and MySQL.

pyodbc

Docker image with python 3.11 and pyodbc. Includes ODBC drivers for MSSQL, PostgreSQL and MySQL.

Usage

Pull the image

$ docker pull laudio/pyodbc

Usage as a base image

Use laudio/pyodbc as a base image for your python script.

FROM laudio/pyodbc:3.0.0

WORKDIR /app

# Add your source files.
COPY ["src", "./src"]
COPY ["setup.py", "./"]

RUN pip install .

CMD ["python", "src/main.py"]

Development (Contributing)

# 1. Clone this repository.
$ git clone [email protected]:laudio/pyodbc.git

# 2. Go to the image directory.
$ cd pyodbc/images/pyodbc

# 3. Build docker image.
$ make build

# 4. Run the container
$ docker run laudio/pyodbc:<tag>

Testing

You can build the test container image providing the flag --target=test and run it.

# Create .env.test file with your database connection creds
# using the the example file .env.example.
# You'll need to update .env.test with your values after this.
$ cp .env.example .env.test

# Build the test container image and run tests.
$ make clean build test

Examples

Basic Connections

Examples showing a basic use case.

1. MSSQL Example
2. PostgreSQL Example
3. MySQL Example

Data Pipelines

Examples illustrating data pipelines using pyodbc.

1. Data Transfer Example (PostgreSQL)
2. Data Transfer Example (MSSQL)
3. Data Transfer Example (MySQL)

License

Licensed under MIT.

pyodbc's Issues

Example improvements - Print the number of rows transferred

Need to print an INFO message here on how many rows were transferred.

For example:

Transferred 3 rows of fruits data from source {database} to destination {database}.

Originally posted by @kabirbaidhya in #19

Support MySQL and add tests to verify database connection works

Test to ensure MySQL connection works.

Requirements

• Add MySQL ODBC driver
• Add test for MySQL connection
• Integrate it in Travis

Take reference from PR #1 on how it's done for Postgres and MSSQL; need to add MySQL in that.

Review existing tests here https://github.com/laudio/pyodbc/tree/master/test

Add examples for usage with MySQL

Add examples for usage with MySQL.

Need two examples:

• Basic Example (under examples/app-mysql) (#41)
• Data Transfer Example (under examples/data-transfer-mysql) (#43)

Check the existing examples for mssql and pg first before starting as the everything needs to follow the same convention and structure with mysql.

Generate significant fake data for the examples using faker

Generate significant volume (~10,000 records) of data using faker in each of these available examples:

• Postgres - Simple Example (#27)
• Postgres - Data Transfer Example (#29)
• MSSQL - Simple Example (#28)
• MSSQL - Data Transfer Example (#31)

Reference: https://github.com/joke2k/faker.

Contributors can contribute pick any one (or more) of the examples mentioned above.

Setup an example data pipeline task (container) for data transfer using pyodbc - pg

Setup an example data pipeline task (container) for data transfer using pyodbc.

Requirements:

• Add an example project - examples/data-transfer-pg
• Have two mssql databases - source and destination; and transfer data from first database's table to the destination database's table.

Configure and Integrate microsoft/pyright in the CI to enforce static typing

Requirements:

• Configure & Integrate this - https://github.com/microsoft/pyright
• Add guidelines in the README to run it on the local.
• Travis Integration to validate code changes in the pull requests.

Release and CI - avoid automatic releasing / publishing of new image if there aren't any changes in the image

Current Behavior
For any commit pushed into the master branch, CI will automatically release a new tag and publish a new docker image for laudio/pyodbc to docker hub.

If there has been really a change in the actual docker image or the contents of the image eg: Dockerfile, requirements.txt etc it makes sense to release a new image. But several times there have been changes pushed including only the documentation update, tests, example code, etc which doesn't bring any change in the actual image - in these cases releasing a new tag and the image doesn't make sense.

Expected Behavior
Update CI / release script to not release a new image if the actual docker image hasn't changed or any of the underlying dependencies of the image hasn't changed. For example for the changes like doc updates, examples, addition of new tests etc which shouldn't get a new version bump.

App not updating

Hi @laudio

I love your docker image however, the app does not pull up to date data from the database within the docker container, hence the app does not update itself.

What can I do to solve this please

Setup an example data pipeline task (container) for data transfer using pyodbc - mssql

Setup an example data pipeline task (container) for data transfer using pyodbc.

Requirements:

• Add an example project - examples/data-transfer-mssql
• Have two mssql databases - source and destination; and transfer data from first database's table to the destination database's table.

Move from Docker Hub to GitHub Registry

Setup an example project using postgres

Add an example project using postgres database connection.

Requirements:

• Create a project examples/app-pg.
• Should have dockerfile; make use of laudio/pyodbc as a base image.
• App should make postgres db connection and demonstrate running few queries on it
• Should have docker-compose.yml with two services - pg and app; so running docker-compose up should run the app.

CI - Integrate a linter and have it fail for PRs / branches for lint errors

CI - Integrate a linter and have it fail for PRs / branches for lint errors.

Image support for ARM64 arch

Hello, I am currently a developer who uses the laudio/pyodbc container image.
I was wondering if there would be a way to get support for Apple Silicone products with M1/M2 chips.

This should help anyone running on an arm64 raspberry pi as well.
Currently there is only support for X86_64.

If you do not have access to test on a Mac or other arm64 box please let me know I am more than happy to help test :)
Example of issue

docker pull laudio/pyodbc
Using default tag: latest
latest: Pulling from laudio/pyodbc
no matching manifest for linux/arm64/v8 in the manifest list entries

Verify checksum of MYSQL_CONNECTOR

Background

I was reading through this article which led me to create this issue.

https://www.vitavonni.de/blog/201503/2015031201-the-sad-state-of-sysadmin-in-the-age-of-containers.html

Problem

We are using a raw curl operation to download the MYSQL_CONNECTOR driver. The least we can do to make sure that we are downloading the right driver and not a malicious package is to check the checksum of MYSQL_CONNECTOR file.

Thoughts? Being a little more aware of the security implications and the probable users of this image can help us in the long run.

Add static typing for all the python code base

Enable static typing for all the python codebase (tests, examples). The base image already uses python >= 3.6 which supports static typing - we should make good use of it.

Reference: https://docs.python.org/3/library/typing.html#module-typing.

Note: some part of the existing code already has types - eg: here - this has to be done consistently where ever possible.

Setup an example project using mssql

Add an example project using mssql database connection.

Requirements:

• Create a project examples/app-mssql.
• Should have dockerfile make use of laudio/pyodbc as a base image.
• App should make mssql connection and demonstrate running few queries on it
• Should have docker-compose.yml with two services - mssql and app; so running docker-compose up should run the app.