muety/telepush

🤖 A simple bot to translate JSON HTTP requests into Telegram push messages

Telepush

Send Telegram push notifications easily via HTTP

---

📄 Description

A simple Telegram Bot to translate POST requests with JSON payload into Telegram push messages. Similar Gotify and ntfy.sh, except without an extra app. Useful for server monitoring, alerting, and anything else.

⌨️ How to use?

Step 1: Get a token

Open Telegram, start a chat with the TelepushBot (or your own bot, respectively) and type /start to obtain a recipient token.

Step 2: Send messages

Messages are sent via POST to https://telepush.dev/api/[[inlets/<inlet_name>]|messages]/<recipient>.

RECIPIENT_TONEN=... # The token you received using /start.
curl -XPOST \
     -H 'Content-Type: application/json' \
     -d '{ "text": "*Hello World!* (yes, this is Markdown)" }' \
     "https://telepush.dev/api/messages/${RECIPIENT_TONEN}"

When hosting your own instance, replace the URL respectively.

✉️ Message options

Key	Type	Description
text*	string	Actual message in plain text or Markdown format
origin	string	What will be shown as the sender of your message
type	[TEXT, FILE]	Message type, i.e. to send text or a file (default: TEXT)
file**	base64	Base64-encoded file content
filename**	string	Name of the file to send
options	object	Additional options to pass
options.disable_link_previews	bool	Whether to show previews of links in the chat

* = required for type: TEXT, ** = required for type: FILE

More details to be found here.

🏃‍♀️ How to run?

☁️ Option 1: telepush.dev

Simply use the official hosted instance. Rate-limited to 240 requests per recipient per day.

🌐 Option 2: Self-hosted

When hosting your own Telepush instance, you need to create a new bot with @BotFather first. As a result, you will get a token that you then pass to Telepush when starting the server (see below).

🐳 Option 2.1: With Docker

$ docker volume create telepush_data
$ docker run -d \
    -p 8080:8080 \
    -v telepush_data:/srv/data \
    --name telepush \
    ghcr.io/muety/telepush \
    -mode webhook \
    -token <YOUR_BOTFATHER_TOKEN>

🐋 Option 2.2: With Kubernetes

See here.

🛠 Option 2.3: Compile from source

# Install
$ go install github.com/muety/telepush@latest

# Run (webhook mode)
$ ./telepush -token <YOUR_BOTFATHER_TOKEN> -mode webhook

↔️ Webhook vs. long-polling

You can either run the bot in long-polling- or webhook mode (-mode [webhook|poll]). For production use the latter option is recommended for various reasons. However, you'll need a server with a static IP and a TLS certificate.

More details about webhook setup can be found in Marvin's Marvellous Guide to All Things Webhook.

🔓 HTTPS

In webhook mode, Telegram requires your updates endpoint to use HTTPS. To enable such, either run Telepush behind a reverse proxy (like Caddy), that terminates TLS.

Or, let Telepush itself handle TLS. You'll need a certificate for this, so either get one from Let's Encrypt or create a self-signed one, then use -useHttps in combination with -certPath and -keyPath pointed to your certificate and private key files respectively.

For self-signed certificates, you'll need to pass your public key to Telegram's setWebhook method in addition, see these instructions.

🔧 Configuration options

• -address (string) – Network address (IPv4) to bind to. Defaults to 127.0.0.1.
• -address6 (string) – Network address (IPv6) to bind to. Defaults to ::1.
• -disableIPv6 (bool) – Whether to disable listening on both IPv4 and IPv6 interfaces. Defaults to false.
• -port (int) – TCP port to listen on. Defaults to 8080.
• -proxy (string) – Proxy connection string to be used for long-polling mode. Defaults to none.
• -urlSecret (string) – Random suffix to append to your updates route called by Telegram's servers to prevent spam. Defaults to none.
• -useHttps (bool) – Whether to use HTTPS. Defaults to false.
• -certPath (string) – Path of your SSL certificate when using webhook mode with useHttp. Default to none.
• -keyPath (string) – Path of your private SSL key when using webhook mode with useHttp. Default to none.
• -dataDir (string) – File system location where to store persistent data. Defaults to ..
• -inlets (string) – Path to folder containing config-based inlet definitions in YAML format. Defaults to ./inlets.d.
• -blacklist (string) – Path to a line-separated blacklist file containing user IDs (send /help to get your id). Defaults to blacklist.txt.
• -whitelist (string) – Path to a line-separated whitelist file containing user IDs (send /help to get your id). Defaults to whitelist.txt.
• -rateLimit (int) – Maximum number of messages to be delivered to each recipient per hour. Defaults to 100.
• -truncateMsgs (bool) – Truncate too long messages to 4096 characters instead of rejecting them. Defaults to false.
• -metrics (bool) – Whether to expose Prometheus metrics under /metrics. Defaults to false.

When using the Docker image, you can alternatively set most of the above config options via environment variables (passed to docker run using -e), e.g. APP_USE_HTTPS=true, APP_CERT_PATH=/etc/telepush.crt, etc. For details, see entrypoint.sh.

Whitelist vs. Blacklist

(aka. allow-list vs. block-list)

📥 Inlets

Inlets provide a mechanism to pre-process incoming data that comes in a format different from what is normally expected by the bot.

This is especially useful if data is sent by external, third-party applications which you cannot modify.

For instance, you might want to deliver alerts from Prometheus' Alertmanager as Telegram notifications. However, Alertmanager's webhook requests look much different from Telepush's default input format. To still make them fit, you can write an Inlet to massage the data accordingly.

To directly address an inlet, request https://telepush.dev/api/inlets/<inlet_name>. Note that /api/inlets/default is equivalent to /api/messages.

Following inlets are currently available:

Name	Description	Status
default	Simply passes the request through without any changes	✅
alertmanager	Consumes Alertmanager webhook requests	✅
grafana	Consumes Grafana webhook requests	✅
webmentionio	Accepts Webmention.io webhook requests to notify about a new Webmention of one of your articles	✅
bitbucket	Accepts Bitbucket webhook requests to notify about a pipeline status change	⏳

You can also define your own, custom inlets in YAML. Further documentation on this and about the individual pre-existing inlets is available here.

For all inlets, the following options are available to be passed as query parameters:

Parameter	Description
disable_link_previews	Disable a preview version of web links detected in the message. Default: false.
disable_markdown	Disable the message being attempted to be parsed as Markdown. Default: false.

Example: POST https://telepush.dev/api/inlets/default?disable_markdown=true&disable_link_previews=true.

📊 Metrics

Fundamental Prometheus metrics are exposed under /metrics, if the -metrics flag gets passed. They include:

• telepush_messages_total{origin="string", type="string"}
• telepush_requests_total{success="string"}

🐞 Troubleshooting

Error: panic: template: pattern matches no files: *.tpl.html

When running Telepush as a single executable, you must not use dev mode unless Telepush's source code is located inside the same directory. Make sure to not pass -env dev in this case.

📓 License

MIT @ Ferdinand Mütsch

telepush's Issues

Make `go install`able

Embed all static asset, including version.txt

Pass on Telegram API errors to requester by defaut

Client calling Telepush should be made aware of bad requests.

Satisfy golint

https://goreportcard.com/report/github.com/muety/telegram-middleman-bot

Prometheus exporter

Allow disable link previews when sending message

Is it possible to disable link previews when sending a message with this bot? If not, can that functionality be added?

It looks like it's possible through the Telegram Bot API, but I can't get it to work with the webhook2telegram bot. Specifically, the sendMessage method takes a disable_web_page_preview but when I pass that parameter into my POST request for this bot, it has no effect.

Example

I'm using the Python requests library to send a message that contains a link.

import requests

payload = {
    "recipient_token": "my-token-value",
    "text": "here's [a link](https://github.com/muety/webhook2telegram)",
    "origin": "MyBot",
    "disable_web_page_preview": True,  # this has no effect
}

url = "https://apps.muetsch.io/webhook2telegram/api/messages"
mmb_response = requests.post(url, json=payload)  # this sends the message

Here's what the message looks like when I receive it. I don't want the link preview included in the message.

Support multi-part file uploads

Having to base64-encode files is not ideal.

Sender blacklist

Introduce a way to blacklist certain user IDs for both incoming updates and outgoing messages. Update messages from them as well as outgoing messages to them will simply be discarded.

Feature Request: Docker compose example

Hello, first of all thanks for this excellent tool.

I see in the README that there's the Docker run command to execute it in a container. However I was wondering if you could test and provide also a Docker Compose example.

I'm getting an issue where the endpoint to test complains that 'passed token does not relate to a valid user', and I don't know if it might be because I made the wrong conversion to Docker Compose and provided wrongly the access token. The bot works on other applications so it's not a matter of the bot

My docker compose looks currently as follows

services:
  telepush:
    image: ghcr.io/muety/telepush
    container_name: telepush
    command: '-mode webhook -token ${TELEGRAM_BOT_TOKEN}'
    restart: unless-stopped
    environment:
      - TELEGRAM_BOT_TOKEN=${TELEGRAM_BOT_TOKEN}
    networks:
      proxy:
    volumes:
      - /home/my/telepush/directory:/srv/data

Worth mentioning also, I obtained the recipient token you mention by sending a /start message to the bot then copying the Chat ID. Is that correct?

Thanks in advance

Resolve messages asynchronously

Add request option to disable parsing text as Markdown

When a message contains reserved Markdown characters without actually being Markdown (e.g. a single _, a single [, etc.). Telegram will respond with an error.

'Bad Request: can't parse entities: Can't find end of the entity starting at byte offset 20'

There should be a request option so that suers can explicitly disable Markdown.

Idea: Web Reporting API Inlet

https://developers.google.com/web/updates/2018/09/reportingapi

Example

1. Make some web server return Report-To header

Report-To { "group": "reporting-1", "max_age": 10886400, "endpoints": [{ "url": "https://example.org/webhook2telegram/api/inlets/reporting?token=some-recipient-token" }] }

2. Make webhook2telegram accept JSON like this

[{
  "type": "csp",
  "age": 10,
  "url": "https://example.com/vulnerable-page/",
  "user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0",
  "body": {
    "blocked": "https://evil.com/evil.js",
    "directive": "script-src",
    "policy": "script-src 'self'; object-src 'none'",
    "status": 200,
    "referrer": "https://evil.com/"
  }
}]

Auto-reload inlet configs upon file change

Be alerted through inotify about changes inside inlets.d.

Provide GitHub releases including pre-compiled binaries

Support multiple tokens per user

A user should be able to register more than one recipient token at a time. There shall be a command to revoke active tokens.

Idea: GitHub Inlet

Bot is leaking ...

After few days of idling the bot is responding with '500 Internal Server Error', and it looks like it's leaking file descriptors:

ls /proc/25802/fd |wc -l
1024

Provide official Docker image

Use specific, platform-independent data types

Hi, I build my own docker image for Raspberry Pi (armhf) using your repo. I needed to make one small change in Dockerfile (add one line - RUN apk add git - so should not change anything). The build was fine, but after starting the container and starting a chat with my bot, I get the following error (of course, ID changed):

2022/04/14 12:19:19 error getting updates: json: cannot unmarshal number 1234567890 into Go struct field TelegramUser.result.message.from.id of type int

Any idea what might be wrong? Could it be connected to a different arch?

Rate limiting for token updates

❗️ Webhook2Telegram becomes Telepush ❗️

Please refer to the release notes of version 3.

Support multiple recipients per requests

Allow to pass comma-separated list of recipients as path parameters.

Idea: Webmentions Inlet

Middleman bot should be able to act as a receiver for Webmentions. That is, it should be able to process an x-www-form-urlencdoded POST request like the following, send out a notification and return 202.

Request Example:

POST /api/inlets/webmention HTTP/1.1
Host: middleman.yourserver.tld
Content-Type: application/x-www-form-urlencoded

source=https://blog.bob.com/some-post.html&
target=https://alice.com/blog/original_post.html

The only problem here is that the Webmention request performed by its sender will not contain any information about the Telegram recipient, i.e. there is no way to transmit a recipient_token and therefore no way to route the notification to a registered user.

Proposal: Introduce a way to configure the bot using commands, including the ability to set inlet-specific parameters. E.g. a new /config command might be introduced that requires two arguments, corresponding to key and value. These key-value pairs are persisted at the server and used for any kind of configuration – in this case for routing Webmention notifications. Specifically, a Middleman user might "subscribe" herself to incoming Webmentions for a certain domain: /config inlet.webmentions.subscription "alice.com". This will tell the bot to notify Alice about a new Webmention containing any URL under "alice.com" as a target.

One problem with the above proposal is that it doesn't feature any type of authorization. If you are Alice, hosting a middleman instance with multiple users, any of those users could practically subscribe to your Webmentions. I'd consider that acceptable for now, but if anyone comes up with a better approach, please let me know!

Idea: HackerNews Inlet

Issue: webhook not used

• current docker image ( ghcr -> muety -> telepush)

• tried to set secretUrl for setWebhook, thingy did still want me to send /api/updates .. inacceptable (!)

• from the go code it should be
  /api/updates_YOURcustomTOKEN
  ( the default is /api/updates , just if any one googles this )

• since the thingy did not show the message from source code ( currently LINE 138 , Updates from Telegram are accepted under '/api%s'. Set the webhook accordingly (see https://core.telegram.org/bots/api#setwebhook)\n", botConfig.GetUpdatesPath( )
  nginx was (ab)used and curl was used to set the webhook

---

relevant parts (nginx):

location /api/ {

    location = /api/updates {
        deny all; return 403;
    }
    location ~ ^/api/messages$ {
         return 404;
         access_log /dev/null;
    }

      location = /api/updates_customstring {
      rewrite  ^/api/updates_customstring$ /api/updates break;
      client_max_body_size 4M;

      proxy_http_version 1.1;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_cache_bypass $http_upgrade;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection $http_connection;
      proxy_pass http://upstream_signals ;
      proxy_set_header Host $host;
      proxy_redirect     off;

      }
}

relevant parts(curl)

# clear webhook
curl -F "url=" https://api.telegram.org/bot<YOURTOKEN>/setWebhook
curl -F "url=https://<YOURDOMAIN.EXAMPLE>/api/updates_customstring" https://api.telegram.org/bot<YOURTOKEN>/setWebhook

---

Your Question was ?

• why is the thingy not using the setUrl parameter ( used in docker compose: APP_URL_SECRET: customtokenthingy)
• are PR's welcome to document at least the /api/updates endpoint ?

regards

Add URL secret to Telegram updates route

See https://github.com/muety/anchr/blob/458b07be021594ad51faf80c044a90e2a2eaac96/app/controllers/telegram.js#L268

Truncate messages to fit maximum length limitation

Discussed in #65. Roughly copy alertmanager's implementation. Also, add a config option so that a server admin can choose whether to reject too long messages (as it's currently the case) or truncate them.

Feature: return 404 on /api/inlets/default and /api/messages in current version

(update:

Everything works as expected,
version 3 ( named telepush) removed the /api/messages endpoint,
but rejects it with 405 == "method not allowed", which is technically correct ( more or less ).
Unfortunately this isn't interpreted as failure by curl --fail

scripts used were changed to try v3 (/api/messages/YOUR-Tic-Tac-Token) first ,
but some clients ( not physically reachable ) might still try to use the old method
which does not work unless a frontend proxy is used like this ( nginx example)

    location ~ ^/api/messages$ {
         return 404;
         access_log /dev/null;
    }

so the request is:
Please return 404 on /api/messages and /api/inlets/default ( exact path),
which would be technically correct as well ( endpoint does not exist ) and make "curl --fail" work

Pull Request here: #54

Thank You

---

OLD TEXT:

(Original Text 2)

* the router seems to fail on 
* * `/api/messages/YOUR-Tic-Tac-Token`
* * `/api/inlets/default/YOUR-Tic-Tac-Token`


( solution/problem: 
in the below "original ticket" text , the endpoint `/api/messages` was blocked by nginx
to speed up the detection of v2/v3 telepush,

however , the "current" docker image does not seem to accept the urls including tokens..

so if you need to send something , the "recipient_token" has to be in json ,
then both "non-tokenized" endpoints `/api/inlets/default` and `api/messages` will work


)


(Original Text 1)
## Situation:

* had the thingy running in longpoll mode , logs filled up with "destination unreachable" and other warnings
* in a specific chat "sending links" was forbidden for a certain time
* the script used to send the messages is https://gitlab.com/the-foundation/pam-notify-telegram/
* thingy did not send to specific chat anymore , but to all the others
* thingy did still respond to "/start" command
* started thingy in webhook mode 
* manually set webhook ( will document in another ticket)
* created a new token ( for the not-working chat )
* new token works
* as many clients ( some physically hard to reach ) use the "damaged" token , some approaches were tried to make the old location work again: 

* * 301/302 redirect  /api/messages/old-token -> /api/messages/new-token : client follows to new location , sends exact same byte size , message is not sent
* * nginx rewrite with extra location /api/messages/old-token /api/messages/new-token ( verified with extra host/port and rewrite_log)


## Your Question was ?
is there any chance to 

* implement a "token re-write" easily 
* debug the internal storage 
* "reset" any error marker for that chat ( if there is smth like that )
*  delete the old token and change a new one
* make it accept 301 requests 
* or "anything that works" ?

Idea: Kubernetes example

Hey! May be useful for someone... I can share ready k8s configs to deploy.

You need to change just 2 things:

• pvc.yaml: spec.storageClassName
• kubernetes.yaml: Secret.data.token (in base64)
  ...else
• kubernetes.yaml: Deployment.spec,args (add/change as you wish)

I can create a Helm-chart for Telepush if it is needed. But everything is working well with this configs so far so good.

kubernetes.yaml

apiVersion: v1
kind: Secret
metadata:
  name: telepush-secret
  namespace: default
type: Opaque
data:
  token: XXXX
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: telepush
  namespace: default
  labels:
    app: telepush
spec:
  replicas: 1
  selector:
    matchLabels:
      app: telepush
  template:
    metadata:
      labels:
        app: telepush
    spec:
      containers:
      - name: telepush
        image: ghcr.io/muety/telepush
        args:
        - -mode
        - poll
        - -truncateMsgs
        - "true"
        volumeMounts:
        - name: telepush-data
          mountPath: /srv/data
        ports:
        - containerPort: 8080
        env:
        - name: APP_TOKEN
          valueFrom:
            secretKeyRef:
              name: telepush-secret
              key: token
      volumes:
      - name: telepush-data
        persistentVolumeClaim:
          claimName: telepush-pvc
---
apiVersion: v1
kind: Service
metadata:
  name: telepush
  namespace: default
spec:
  selector:
    app: telepush
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080
  type: ClusterIP

pvc.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: telepush-pvc
  namespace: default
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: microk8s-hostpath
  resources:
    requests:
      storage: 100Mi

telepush.zip

Attach file to messages

How hard could be to implement the possibility to attach a file to the message sent to the bot?

Allow to pass payload as query parameters of GET request

Is it possible to achieve this form:
https://domain.com/api/messages/?data={"recipient_token":"f9e1-bb79-4db2-8fff-cfbb8","type":"TEXT","origin":"somedevice"}
Because not all software supports all values, such as origin.

Invalid character response using requests in Python

I'm really excited to message myself via MiddleManBot to alert me when trailhead spots open for the John Muir Trail.

I'm calling the public bot in python as follows:

url = "https://middleman.ferdinand-muetsch.de/api/messages"
payload = {
    'recipient_token': my_token,
    'text': my_text,
    'origin': my_app,
}
import requests
response = requests.post(url, data=payload)

I'm getting a response with status code 400, whose text is:

invalid character 'r' looking for beginning of value

Any idea what the issue could be?

Idea: Stripe webhook inlet

Listen for certain Stripe events, e.g. customer.subscription.created.

Program does not exit when bind fails

Refactor store implementation

https://github.com/etcd-io/bbolt

failed to read store from /srv/data/store.gob

Hi!
I'm getting this error when run telepush with docker:

docker run --rm \
        -p 8080:8080 \
        -v /opt/telepush:/srv \
        --name telepush \
        telepush \
        -mode webhook \
        -token my-token \
        -metrics \
        -dataDir /srv/data \
        -urlSecret secret \
        -whitelist /srv/whitelist.txt

any advice?

Store not thread safe

StorePut put is executed from messageHandler, we might have a situation with a race condition.

Switch to config-based inlet adapter format

I want ease the process of adding new inlets without having to modify the actual code and recompile the app. Instead, come up with a JSON- / YAML- / INI-based config format for inlets, that can be switched out at runtime.

Move from DockerHub to ghcr

https://github.com/muety/wakapi/blob/master/.github/workflows/docker.yml

Asynchronously process incoming updates

Similar to what #17 did for messages sent via respective inlets.

Feature Request: Support arm64 architecture

Hi, I built telepush on arm64 (aarch64) locally and everything seems to work fine. The build doesnt take long, but of course would be nice to just pull a published image from the project. For arm64, I did not need to make any changes to build it locally, so hopefully should not be hard. I can help testing if needed.

PS: support for e.g. arm32 (armhf) would be also nice. Both of these would open the project for the fairly numerous raspberryPi's user-base

Whitelist not working with group chat_id

Hi,

Whitelist doesn't work with group chat id (with negative sign), something like -123456789
It seems to work well with users chat_id that doesn't have the minus sign.

I haven't tested with Blacklist.

I'm using self-hosted bot with docker image version 3.3.0

I have tested a whitelist file with:
user chat id + group chat id (only user chat is permitted)
group chat id (all chats are permitted)

Request rate limit by IP

High-entropy tokens.

I tried to start TelepushBot and it issued me a token that consist of just six hex characters.
Does it mean that one can guess the token or just annoy random people by sending random messages though TelepushBot?
Should TelepushBot issue longer, more random tokens?

Unit tests

Telegram API size limitation 4096?

Hey there! We try to use telepush for alertmanager notifications from our clusters. But we encounter situation, when not all notifications are delivered. After some investigations:

ts=XXXXXXXXXXXXX caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=21 err="telepush/webhook[0]: notify retry canceled due to unrecoverable error after 1 attempts: unexpected status code 400: https://telepush.dev/api/inlets/alertmanager/XXXXXX: message too long (max is 4096 characters)"

Some software deployed with Helm have multiply labels, and it`s easy-to-go to step over this limit. How this issue can be sold? Maybe force Telepush to short notifications size for Telegram eats it?

Refrain from sending messages to chats that have blocked the bot

Also, don't accept requests from them anymore.

Make `origin` parameter optional

... and, if not set, just assume a default value of Webhook2Telegram instead.

Grafana alerts inlet

Idea: whitelist mode

Also known as "allowlist" mode.

Currently the blacklist (also known as "blocklist") allows any user to register a key and use this service as long as they are not blocked. I'm interested in an "allowlist" mode which only allows a group of authorized users to use this service.