realkarych/postamt

Email client. Powered by Telegram Forums and WebApps.

POSTAMT [on russian — Почтамт] is a Telegram ChatBot that replaces mobile and desktop Email clients in 99% of daily task cases

---

🚀 Features

• Email Management: Perform various actions on your emails, such as reading, replying, sending, attaching files, and more.

• Power of Threads: Your emails are filtered by email addresses in a single Telegram supergroup. Telegram Threads technology allows us to organize the user experience similarly to native email clients (like Gmail, Outlook, etc.).

• Multi-Account Support: Connect and manage multiple email accounts within a single interface.

• Security: POSTAMT never requests two-factor authentication and does not have access to the danger zone of your Email account. It does not delete emails, manage passwords and personal data. Connection is made via auto-generated IMAP/SMTP access keys. Your emails are not saved or cached anywhere after having read (even on the Telegram servers)

🖥️ Installation

We acknowledge your desire for full control over your data. Therefore, all source codes for the project are open, allowing you to host the project locally on your own.

To launch project locally:

• Check our Docs: DOCS.md
• Follow the Guideline: INSTALLATION.md

📊 Roadmap [to the first beta]

• Scope: Define project scope and objectives
• Architecture: Set up basic project structure
• Docker: Set up docker to simplify development & delivery
• Cryptography: Set up cryptography infrastructure to encrypt and decrypt secret data
• Core service: Implement IMAP-service
• Migrations: Setup migrations (alembic)
• Database: Implement data schemas and repositories
• Telegram "UI" stage 1: Implement base functionality with registration, adding Email account, group setup
• Message broker (Kafka): Setup Kafka as a single interface to send emails, answer messages
• Telegram "UI" stage 2: Implement Telegram WebViews to check emails
• GPT Model: Integrate GPT API as a emails' summary executor. Candidates: OpenAI, LLaMa
• Flood controller: Set up anti-spam system (on middlewares layer)
• Core service: Implement SMTP-service
• Telegram "UI" stage 3: Implement UI to send emails
• Channel subscription: To use bot, user need to be subscribed to https://t.me/postamt_channel (create filter)
• Grafana: Add Grafana dashboard to check statistics

❞ Why named POSTAMT?

This project is named in honor of the St. Petersburg Main Post Office [Главпочтамт / Glavpochtamt], a monument of Russian history and culture.

"Postamt" is a German word that translates to "post office" in English. It refers to a facility or building where postal services are provided. Post offices are responsible for handling mail and packages, selling postage stamps, offering various postal services, and providing a range of other services related to mail delivery and communication. Postamt plays a crucial role in the postal system, facilitating the sending and receiving of mail and packages within a region or country.

Our POSTAMT semantically resembles a post office.

🙏 Contributing

We welcome contributions from developers and non-developers alike! If you're a developer, you can make a direct impact by contributing code. Clone the repository, make your changes, and submit a pull request.

Not a developer? No problem! You can still contribute by creating issues, reporting bugs, or suggesting new ideas for project improvement (do this via GitHub Issues). Your insights and feedback are invaluable to our collective success and helps us to create the best non-native Email client.

Thank you for your contribution and support to the project!

postamt's Issues

DockerHub

Add image to DockerHub. It will simplify installation.

IMAP session

Write class that establish connection via imap protocol and provides interface to interact with it.

Add dependabot

Implement mappers

Whole DAOs' modules should contain mappers as converters from DTO to db-model and from db-model to DTO

IMAP Repository

Write ImapRepository class that implements base functionality with imap:

• get last n emails
• get email by id
• download attachments into temporary directory

It should works only in async context manager mode.

DI: establish session is provided into class constructor.

Implement after #8

Host web on GitHub pages

I have two variants to host web for web apps:

1. Ngrok
2. GitHub pages.

In my opinion, GitHub pages will better because of stability.

Decoded and Encoded DTOs

In current architecture, db layer will implement encoding-decoding mechanism. We need to create Encoded and Decoded objects and incapsulate logic to this new layer.

Handler `bot ... from forum`

If bot was restricted / kicked or forum has been removed — we need to remove forum id from database emailbox.

Forum creating guideline

Create post at https://blog.karych.ru and provide a url with guideline include screenshots and descriptions how to setup supergroup and add bot to this chat correctly.

Cryptography implementation

To store personal data (like email addresses, keys etc.), we need to encrypt them.
I think, this solution would be appropriate.

Fernet keys will be provided via .env file into docker container.

DockerHub

Create CD that performs pushing current version of project to DockerHub repo via docker compose push.

Secrets should be stored in GitHub secrets.

String -> BinaryFields

Now secrets' bytes stores in database as String.
It's bad. In-need to replace them with Binary format.

Development Pipeline

Add development pipeline to README.md

Error handling | logger

Now in some cases we have raising exception and logging error.

I think the better way is to raise exception and move all logging tracebacks to error handler

Remove `*.pot`, `*.po`, `*.mo` from .gitignore

This provides troubles with deployment

Implement Topic entity repository

Implement User entity repository

Add topic Fernet key

Needs to encrypt and decrypt topic titles.

Topic title is email address from income email.

Fix makefile debug command

If container is not created, rmi causes error. Make rmi optional or something like that. No container -> skip rmi step.

GPT

GPT API integration to provide summary from Email.

Make cryptographers know about fernet keys and do not provide into them

Integrate BotAPI server

Now we have a trouble with large income/outcome files (larger than 20mb) — Telegram BotAPI limit.

In-need:

• Setup custom Telegram BotAPI server

This will provide an opportunity to work with large files (<=2gb).

Repository, DAO -> Repo

Add vulnerability-checker

Check CI codeql.yml

Remove `poetry.lock` from `.gitignore`

According to official documentation, poetry recommends to provide poetry.lock file to repository. It helps with deployment.
If we will not provide it, use need to install poetry on server. It's so bad

Redis integration

Now, interim data like states stores in Aiogram memory storage. That causes problem with losing data on app crashes.
That is not critical for 1-3 seconds states, but pipeline of Email auth requires longer process.

I want to integrate Redis storage that will replace Aiogram memory storage.

Fix CI

Add running unittests to GitHub Actions pipeline

Pump Postgres version to 16.2

Error handler

Implement error handler

Implement Email entity repository

Refactor config

Provide config-data via middlewares.

Feature: migrations

Setup migrations with alembic.

Poetry dependency

Required poetry dependency

`docker compose` instead of `docker-compose` in Makefile

In MacOS docker-compose is correct, but in *nix works only docker compose. Fix Makefile

suppressing not found message

If bot tries to remove message that not exists currently (bot blocked, chat not found, user removed message for both etc.).

So in-need to wrap all remove calls with suppressing errors associated with Telegram BotAPI.

Docker

Create DockerFile and MakeFile to automate the deployment process

imap lib

In-need to choose library to establish connection via imap protocol.
Requirements:

• asynchronously
• simple
• up-to-date

Open bot on app logo click

Add tag to README.md on clicking app icon

Fernet provides different outputs for same data

Now, I check if model is exists in encrypted format. But fernet provides different outputs for same data, wow.

Solution: compare models in decrypted format.

Rewrite Installation guide

Add all stages of deployment pipeline on Debian / Ubuntu.

ui-1

Tasks to implement:

• Registration in Bot: add user to database, add base commands: /start, /help.
• Pipeline of adding Email account: entering secrets, choosing Email server etc.
• Forum setup: bot should set icon, titles, setup base topics.
• Preparing to fetch emails (but not implement it, because Kafka is not currently integrated — check roadmap in README).
  In fact, it will be similar with https://github.com/realkarych/lesst.

Use aiofiles instead of os

Now we uses a mail-parser library to parse email body.
The attachments loads via os library — it's bad if we'll load large files.

In-need:

• Rewrite loading logic with using aiofiles library.

DB schema: Email

Requirements

• an entity can be represented using multiple tables
• each table must be reduced to the 3rd normal form
• all fields with personal information (email_address, passwords etc.) should be stored in encrypted format.

Tables

EmailBox

• id — represents unique key for EmailBox
• owner_id — unique Telegram id of EmailBox owner
• forum_id — unique chat id connected to EmailBox
• last_handled_email_id — id of last sent (handled) email id. Email id is unique for EmailBox
• is_active — status of the EmailBox. If forum has been removed or user didn't payed for the app, status will become False

EmailAuthData

• emailbox_id — unique key for EmailBox
• email_server_id — code of the Email server (yandex, gmail, icloud etc.). All email servers represented here: https://github.com/realkarych/postamt/blob/dev/app/entities/email.py.
• email_address — Email address ([email protected] e.g.). Encrypted.
• email_password — Email IMAP/SMTP access key. Encrypted.

Topic

• forum_id — Topic connected to Forum id
• topic_id — unique topic id in forum
• topic_title — Topic represents chat with all emails from some Email (topic_title). Decrypted format: address@domain. Stored in Encrypted format.

CD

Add CD:

• After pull request / push to main branch, fetch changes on server and recreate Docker images: PostgreSQL, postamt-bot.

docker compose plugin dependency

docker-compose plugin required.

Update server auth guideline

Update server auth guideline: https://blog.karych.ru/postamt-setup

None in field registered_date

Adding / merging User entity with bug: registered_date field with server_default seems to be empty.