Giter Site home page Giter Site logo

td's Introduction

td Go Reference codecov beta

Telegram MTProto API client in Go for users and bots.

Telegram: English chat Telegram: Russian chat Telegram: Chinese chat Telegram: Online count

Before using this library, read How To Not Get Banned guide.

Due to limitations of pkg.go.dev, documentation for tg package is not shown, but there is hosted version.

Higher level libraries

The gotd itself is a pretty low-level library, and you may want to use higher level libraries instead:

  • GoTGProto is a helper package for gotd library, It aims to make td's raw functions easy-to-use with the help of features like using session strings, custom helper functions, storing peers and extracting chat or user ids through it etc.

Usage

go get github.com/gotd/td
package main

import (
	"context"

	"github.com/gotd/td/telegram"
)

func main() {
	// https://core.telegram.org/api/obtaining_api_id
	client := telegram.NewClient(appID, appHash, telegram.Options{})
	if err := client.Run(context.Background(), func(ctx context.Context) error {
		// It is only valid to use client while this function is not returned
		// and ctx is not cancelled.
		api := client.API()

		// Now you can invoke MTProto RPC requests by calling the API.
		// ...

		// Return to close client connection and free up resources.
		return nil
	}); err != nil {
		panic(err)
	}
	// Client is closed.
}

See examples for more info.

Features

  • Full MTProto 2.0 implementation in Golang, directly access any MTProto method with telegram.Client.API()
  • Highly optimized, low memory (150kb per idle client) and CPU overhead, can handle thousands concurrent clients
  • Code for Telegram types generated by ./cmd/gotdgen (based on gotd/tl parser) with embedded official documentation
  • Pluggable session storage
  • Automatic re-connects with keepalive
  • Vendored Telegram public keys that are kept up-to-date
  • Rigorously tested
    • End-to-end with real Telegram server in CI
    • End-to-end with gotd Telegram server (in pure Go)
    • Lots of unit testing
    • Fuzzing
    • 24/7 canary bot in production that tests reconnects, update handling, memory leaks and performance
  • No runtime reflection overhead
  • Conforms to Security guidelines for Telegram client software developers
    • Secure PRNG used for crypto
    • Replay attack protection
  • 2FA support
  • MTProxy support
  • Various helpers that lighten the complexity of the Telegram API
  • Connection pooling
  • Automatic datacenter migration and redirects handling
  • Graceful request cancellation via context
  • WebSocket transport support (works in WASM)

Status

The goal of this project is to implement a stable, performant and safe client for Telegram in pure Go while having a simple and convenient API and a feature parity with TDLib.

This project is fully non-commercial and not affiliated with any commercial organization (including Telegram LLC).

Examples

See examples directory.

Also take a look at

  • go-faster/bot as example of sophisticated telegram bot integration with GitHub
  • gotd/cli, command line interface for subset of telegram methods.

Auth

User

You can use td/telegram/auth.Flow to simplify user authentications.

codePrompt := func(ctx context.Context, sentCode *tg.AuthSentCode) (string, error) {
    // NB: Use "golang.org/x/crypto/ssh/terminal" to prompt password.
    fmt.Print("Enter code: ")
    code, err := bufio.NewReader(os.Stdin).ReadString('\n')
    if err != nil {
        return "", err
    }
    return strings.TrimSpace(code), nil
}
// This will setup and perform authentication flow.
// If account does not require 2FA password, use telegram.CodeOnlyAuth
// instead of telegram.ConstantAuth.
if err := auth.NewFlow(
    auth.Constant(phone, password, auth.CodeAuthenticatorFunc(codePrompt)),
    auth.SendCodeOptions{},
).Run(ctx, client.Auth()); err != nil {
    panic(err)
}

Bot

Use bot token from @BotFather.

if err := client.Auth().Bot(ctx, "token:12345"); err != nil {
    panic(err)
}

Calling MTProto directly

You can use the generated tg.Client that allows calling any MTProto method directly.

// Grab these from https://my.telegram.org/apps.
// Never share it or hardcode!
client := telegram.NewClient(appID, appHash, telegram.Options{})
client.Run(ctx, func(ctx context.Context) error {
  // Grab token from @BotFather.
  if _, err := client.Auth().Bot(ctx, "token:12345"); err != nil {
    return err
  }
  state, err := client.API().UpdatesGetState(ctx)
  if err != nil {
    return err
  }
  // Got state: &{Pts:197 Qts:0 Date:1606855030 Seq:1 UnreadCount:106}
  // This will close client and cleanup resources.
  return nil
})

Generated code

The code output of gotdgen contains references to TL types, examples, URL to official documentation and extracted comments from it.

For example, the auth.Authorization type in tg/tl_auth_authorization_gen.go:

// AuthAuthorizationClass represents auth.Authorization generic type.
//
// See https://core.telegram.org/type/auth.Authorization for reference.
//
// Example:
//  g, err := DecodeAuthAuthorization(buf)
//  if err != nil {
//      panic(err)
//  }
//  switch v := g.(type) {
//  case *AuthAuthorization: // auth.authorization#cd050916
//  case *AuthAuthorizationSignUpRequired: // auth.authorizationSignUpRequired#44747e9a
//  default: panic(v)
//  }
type AuthAuthorizationClass interface {
	bin.Encoder
	bin.Decoder
	construct() AuthAuthorizationClass
}

Also, the corresponding auth.signIn method:

// AuthSignIn invokes method auth.signIn#bcd51581 returning error if any.
// Signs in a user with a validated phone number.
//
// See https://core.telegram.org/method/auth.signIn for reference.
func (c *Client) AuthSignIn(ctx context.Context, request *AuthSignInRequest) (AuthAuthorizationClass, error) {}

The generated constructors contain detailed official documentation, including links:

// FoldersDeleteFolderRequest represents TL type `folders.deleteFolder#1c295881`.
// Delete a peer folder¹
//
// Links:
//  1) https://core.telegram.org/api/folders#peer-folders
//
// See https://core.telegram.org/method/folders.deleteFolder for reference.
type FoldersDeleteFolderRequest struct {
    // Peer folder ID, for more info click here¹
    //
    // Links:
    //  1) https://core.telegram.org/api/folders#peer-folders
    FolderID int
}

Contributions

Huge thanks to all contributors. Dealing with a project of this scale alone is impossible.

Special thanks:

  • tdakkota
    • Two-factor authentication (SRP)
    • Proxy support
    • Update dispatcher
    • Complete transport support (abridged, padded intermediate and full)
    • Telegram server for end-to-end testing
    • Multiple major refactorings, including critical cryptographical scope reduction
    • Code generation improvements (vector support, multiple modes for pretty-print)
    • And many other cool things and performance improvements
  • shadowspore
    • Background pings
    • Links in generated documentation
    • Message acknowledgements
    • Retries
    • RPC Engine
    • Gap (Updates) engine

Reference

The MTProto protocol description is hosted by Telegram.

Most important parts for client implementations:

Current implementation mostly conforms to security guidelines, but no formal security audit were performed.

Prior art

Who is using gotd?

  • The iyear/tdl, 📥 Telegram Downloader, but more than a downloader

Drop a comment here to add your project.

License

MIT License

Created by Aleksandr (ernado) Razumov

2020

Links

td's People

Contributors

abserari avatar actions-user avatar dependabot-preview[bot] avatar dependabot[bot] avatar doflatango avatar emmitrin avatar ernado avatar iyear avatar koenigskraut avatar lazarenkoa avatar luk0y avatar mendelmaleh avatar nnqq avatar roj1512 avatar savely-krasovsky avatar shadowspore avatar shivakumargn avatar snimshchikov avatar tdakkota avatar tie avatar zorinarsenij avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

td's Issues

client: reconnect on `read: connection reset by peer`

v0.13.1

telegram/read_loop.go:187        Read returned error        {"dc": 4, "error": "read: read intermediate: failed to read length: read tcp 195.201.99.97:52436->149.154.167.91:443: read: connection reset by peer

Probably we should not print such error and just reconnect.
This error means that connection was closed by remote side and should be handled by reconnection.

client: implement tracing using go-macro

Currently tracer relies on manual code:

type tracer struct {
	// OnMessage is called on every incoming message if set.
	OnMessage func(b *bin.Buffer)
}

func (t tracer) Message(b *bin.Buffer) {
	if t.OnMessage == nil {
		return
	}
	t.OnMessage(b)
}

It is too error-prone and I want to automate it.

Ref: tdakkota-legacy/gomacro#8

gen: reduce binary size

We have lots of mostly duplicating string literals that bloats binary size:

return fmt.Errorf("unable to decode destroy_session#e7512126: field session_id: %w", err)

This can be reduced, probably as following:

type decodingErr {
   Type string
   TypeID int32
   // Field string?
}
$ go run github.com/jondot/goweight | head -n 10
   45 MB github.com/gotd/td/tg
  6.0 MB runtime
  5.3 MB net/http
  2.4 MB net
  2.4 MB crypto/tls
  2.0 MB reflect
  1.4 MB github.com/gotd/td/internal/mt
  1.3 MB math/big
  1.2 MB go.uber.org/zap/zapcore
  1.2 MB syscall

client: ensure updates after reconnect

Telegram expects client to call some API method so server can mark session active.
Only last active session receives updates.

Currently, we don't call any method on reconnect event.

Probably we can use callback?

gen: automate pulling schema

Probably the most convenient way is to pull tdesktop, write mtproto and telegram schema to corresponding files and run codgen.

Later this should be done dependabot-like with automatic merge requests.

dispatcher's handlers not called after connection restarts

At first start dispatcher's OnNewChannelMessage is being called and messages are received.
When I disconnect the wifi and connect again, I can see that the connection restarted
But the dispatcher's OnNewChannelMessage never receive messages again until restart the process.

Here's the log

2021-01-14T22:41:35.195+0800    DEBUG   mtproto/handle_message.go:18    Handle message  {"type_id": "0x73f1f8dc", "type_name": "message_container"}
2021-01-14T22:41:35.195+0800    DEBUG   mtproto/handle_message.go:18    Handle message  {"type_id": "0x78d4dec1", "type_name": "updateShort#78d4dec1"}
2021-01-14T22:41:35.195+0800    DEBUG   mtproto/handle_message.go:18    Handle message  {"type_id": "0x74ae4240", "type_name": "updates#74ae4240"}
2021/01/14 22:41:35 GOT message from  1159455328
2021-01-14T22:42:11.827+0800    DEBUG   mtproto/new_encrypted_msg.go:14 Request {"type_id": "0xf3427b8c", "type_name": "ping_delay_disconnect#f3427b8c", "msg_id": 6917626073254531976}
2021-01-14T22:42:11.827+0800    DEBUG   group   tdsync/log_group.go:52  Task stopped    {"task": "pingLoop", "error": "disconnect (pong missed): write: write: write intermediate: write tcp 10.0.2.24:50878->91.108.56.175:443: write: can't assign requested address", "errorVerbose": "disconnect (pong missed):\n    github.com/gotd/td/mtproto.(*Conn).pingLoop\n        /home/darren/tbottgbot/td/mtproto/ping.go:114\n  - write:\n    github.com/gotd/td/mtproto.(*Conn).pingDelayDisconnect\n        /home/darren/tbottgbot/td/mtproto/ping.go:59\n  - write:\n    github.com/gotd/td/transport.(*connection).Send\n        /home/darren/tbottgbot/td/transport/connection.go:39\n  - write intermediate:\n    github.com/gotd/td/internal/proto/codec.Intermediate.Write\n        /home/darren/tbottgbot/td/internal/proto/codec/intermediate.go:57\n  - write tcp 10.0.2.24:50878->91.108.56.175:443: write: can't assign requested address", "elapsed": "1m0.170718158s"}
2021-01-14T22:42:11.827+0800    DEBUG   mtproto/conn.go:133     Closing
2021-01-14T22:42:11.827+0800    INFO    rpc     rpc/engine.go:240       Close called
2021-01-14T22:42:11.827+0800    DEBUG   group   tdsync/log_group.go:52  Task stopped    {"task": "ackLoop", "error": "acl: context canceled", "errorVerbose": "acl:\n    github.com/gotd/td/mtproto.(*Conn).ackLoop\n        /home/darren/tbottgbot/td/mtproto/ack.go:32\n  - context canceled", "elapsed": "1m0.170858717s"}
2021-01-14T22:42:11.828+0800    DEBUG   group   tdsync/log_group.go:55  Task complete   {"task": "handleClose", "elapsed": "1m0.171243565s"}
2021-01-14T22:42:11.828+0800    DEBUG   group   tdsync/log_group.go:55  Task complete   {"task": "readEncryptedMessages-1", "elapsed": "1m0.171149302s"}
2021-01-14T22:42:11.828+0800    DEBUG   read    mtproto/read.go:112     Read loop done  {"reason": "context canceled"}
2021-01-14T22:42:11.828+0800    DEBUG   group   tdsync/log_group.go:55  Task complete   {"task": "readEncryptedMessages-4", "elapsed": "1m0.171251912s"}
2021-01-14T22:42:11.828+0800    DEBUG   group   tdsync/log_group.go:52  Task stopped    {"task": "readLoop", "error": "context canceled", "elapsed": "1m0.171207591s"}
2021-01-14T22:42:11.828+0800    DEBUG   group   tdsync/log_group.go:55  Task complete   {"task": "readEncryptedMessages-5", "elapsed": "1m0.171257187s"}
2021-01-14T22:42:11.828+0800    DEBUG   group   tdsync/log_group.go:55  Task complete   {"task": "readEncryptedMessages-2", "elapsed": "1m0.171242276s"}
2021-01-14T22:42:11.828+0800    DEBUG   group   tdsync/log_group.go:55  Task complete   {"task": "readEncryptedMessages-7", "elapsed": "1m0.171186247s"}
2021-01-14T22:42:11.828+0800    DEBUG   group   tdsync/log_group.go:55  Task complete   {"task": "readEncryptedMessages-8", "elapsed": "1m0.171213614s"}
2021-01-14T22:42:11.828+0800    DEBUG   group   tdsync/log_group.go:55  Task complete   {"task": "readEncryptedMessages-6", "elapsed": "1m0.171342079s"}
2021-01-14T22:42:11.828+0800    DEBUG   group   tdsync/log_group.go:55  Task complete   {"task": "readEncryptedMessages-9", "elapsed": "1m0.171367265s"}
2021-01-14T22:42:11.828+0800    DEBUG   group   tdsync/log_group.go:55  Task complete   {"task": "readEncryptedMessages-3", "elapsed": "1m0.171305902s"}
2021-01-14T22:42:11.828+0800    DEBUG   group   tdsync/log_group.go:55  Task complete   {"task": "readEncryptedMessages-0", "elapsed": "1m0.171217868s"}
2021-01-14T22:42:11.828+0800    DEBUG   mtproto/conn.go:175     Run: end
2021-01-14T22:42:12.384+0800    INFO    telegram/client.go:252  Restarting connection   {"v": "v0.20.1-0.20210110102221-42ee6165d055", "error": "group: task pingLoop: disconnect (pong missed): write: write: write intermediate: write tcp 10.0.2.24:50878->91.108.56.175:443: write: can't assign requested address", "errorVerbose": "group:\n    github.com/gotd/td/mtproto.(*Conn).Run\n        /home/darren/tbottgbot/td/mtproto/conn.go:175\n  - task pingLoop: disconnect (pong missed): write: write: write intermediate: write tcp 10.0.2.24:50878->91.108.56.175:443: write: can't assign requested address"}
2021-01-14T22:42:12.384+0800    INFO    rpc     rpc/engine.go:44        Initialized     {"retry_interval": "5s", "max_retries": 5}
2021-01-14T22:42:12.384+0800    DEBUG   mtproto/conn.go:157     Run: start
2021-01-14T22:42:12.385+0800    INFO    mtproto/conn.go:188     Dialed transport        {"addr": "91.108.56.175:443"}
2021-01-14T22:42:12.385+0800    INFO    mtproto/conn.go:206     Key already exists
2021-01-14T22:42:12.385+0800    DEBUG   mtproto/conn.go:211     Generating new session id
2021-01-14T22:42:12.385+0800    DEBUG   group   tdsync/log_group.go:49  Task started    {"task": "handleClose"}
2021-01-14T22:42:12.385+0800    DEBUG   group   tdsync/log_group.go:49  Task started    {"task": "readEncryptedMessages-0"}
2021-01-14T22:42:12.385+0800    DEBUG   group   tdsync/log_group.go:49  Task started    {"task": "readEncryptedMessages-9"}
2021-01-14T22:42:12.385+0800    DEBUG   group   tdsync/log_group.go:49  Task started    {"task": "readEncryptedMessages-4"}
2021-01-14T22:42:12.385+0800    DEBUG   group   tdsync/log_group.go:49  Task started    {"task": "readEncryptedMessages-5"}
2021-01-14T22:42:12.385+0800    DEBUG   group   tdsync/log_group.go:49  Task started    {"task": "readEncryptedMessages-1"}
2021-01-14T22:42:12.385+0800    DEBUG   group   tdsync/log_group.go:49  Task started    {"task": "readEncryptedMessages-8"}
2021-01-14T22:42:12.385+0800    DEBUG   group   tdsync/log_group.go:49  Task started    {"task": "readEncryptedMessages-2"}
2021-01-14T22:42:12.385+0800    DEBUG   group   tdsync/log_group.go:49  Task started    {"task": "readLoop"}
2021-01-14T22:42:12.385+0800    DEBUG   group   tdsync/log_group.go:49  Task started    {"task": "readEncryptedMessages-3"}
2021-01-14T22:42:12.385+0800    DEBUG   group   tdsync/log_group.go:49  Task started    {"task": "userCallback"}
2021-01-14T22:42:12.385+0800    DEBUG   read    mtproto/read.go:106     Read loop started
2021-01-14T22:42:12.405+0800    DEBUG   conn    telegram/conn.go:133    Initializing
2021-01-14T22:42:12.385+0800    DEBUG   group   tdsync/log_group.go:49  Task started    {"task": "ackLoop"}
2021-01-14T22:42:12.385+0800    DEBUG   group   tdsync/log_group.go:49  Task started    {"task": "readEncryptedMessages-7"}
2021-01-14T22:42:12.385+0800    DEBUG   group   tdsync/log_group.go:49  Task started    {"task": "readEncryptedMessages-6"}
2021-01-14T22:42:12.385+0800    DEBUG   group   tdsync/log_group.go:49  Task started    {"task": "pingLoop"}
2021-01-14T22:42:12.405+0800    DEBUG   mtproto/rpc.go:26       rpcDo start     {"content_msg": true, "msg_id": 6917626077127709272}
2021-01-14T22:42:12.405+0800    DEBUG   rpc     rpc/engine.go:84        Do called       {"msg_id": 6917626077127709272}
2021-01-14T22:42:12.405+0800    DEBUG   mtproto/new_encrypted_msg.go:14 Request {"type_id": "0xda9b0d0d", "type_name": "invokeWithLayer#da9b0d0d", "msg_id": 6917626077127709272}
2021-01-14T22:42:12.405+0800    DEBUG   rpc     rpc/ack.go:29   Waiting for acknowledge {"msg_id": 6917626077127709272}
2021-01-14T22:42:12.867+0800    DEBUG   mtproto/handle_message.go:18    Handle message  {"type_id": "0x73f1f8dc", "type_name": "message_container"}
2021-01-14T22:42:12.867+0800    DEBUG   mtproto/handle_message.go:18    Handle message  {"type_id": "0x9ec20908", "type_name": "new_session_created#9ec20908"}
2021-01-14T22:42:12.867+0800    DEBUG   mtproto/handle_session_created.go:16    Session created {"unique_id": -8939959683178286869, "first_msg_id": 6917626077127709272}
2021-01-14T22:42:12.867+0800    INFO    conn    telegram/conn.go:67     SessionInit
2021-01-14T22:42:12.950+0800    DEBUG   mtproto/handle_message.go:18    Handle message  {"type_id": "0xf35c6d01", "type_name": "rpc_result"}
2021-01-14T22:42:12.950+0800    DEBUG   mtproto/handle_result.go:18     Handle result   {"type_id": "0x3072cfa1", "type_name": "gzip", "msg_id": 6917626077127709272}
2021-01-14T22:42:12.950+0800    DEBUG   mtproto/handle_result.go:35     Decompressed    {"type_id": "0x330b4067", "type_name": "config#330b4067", "msg_id": 6917626077127709272}
2021-01-14T22:42:12.950+0800    DEBUG   rpc     rpc/engine.go:96        Handler called  {"msg_id": 6917626077127709272}
2021-01-14T22:42:12.950+0800    DEBUG   rpc     rpc/ack.go:50   Acknowledge canceled    {"msg_id": 6917626077127709272}
2021-01-14T22:42:12.950+0800    DEBUG   mtproto/rpc.go:50       rpcDo end       {"content_msg": true, "msg_id": 6917626077127709272}
2021-01-14T22:42:12.950+0800    DEBUG   group   tdsync/log_group.go:55  Task complete   {"task": "userCallback", "elapsed": "564.578224ms"}
2021-01-14T22:42:12.951+0800    DEBUG   telegram/client.go:335  Data saved      {"v": "v0.20.1-0.20210110102221-42ee6165d055", "key_id": "599a808d43d784fc"}
2021-01-14T22:42:12.951+0800    DEBUG   telegram/client.go:262  Ready   {"v": "v0.20.1-0.20210110102221-42ee6165d055"}
2021-01-14T22:42:12.951+0800    DEBUG   mtproto/handle_message.go:18    Handle message  {"type_id": "0x62d6b459", "type_name": "msgs_ack#62d6b459"}
2021-01-14T22:42:12.951+0800    DEBUG   mtproto/handle_ack.go:17        Ack     {"msg_ids": [6917626077127709272]}
2021-01-14T22:42:12.951+0800    DEBUG   rpc     rpc/ack.go:18   Acknowledge callback not set    {"msg_id": 6917626077127709272}
2021-01-14T22:42:27.409+0800    DEBUG   mtproto/new_encrypted_msg.go:14 Request {"type_id": "0x62d6b459", "type_name": "msgs_ack#62d6b459", "msg_id": 6917626141555715712}
2021-01-14T22:42:27.409+0800    DEBUG   ack     mtproto/ack.go:25       ACK     {"msg_ids": [6917626081457987585]}
2021-01-14T22:43:12.408+0800    DEBUG   mtproto/new_encrypted_msg.go:14 Request {"type_id": "0xf3427b8c", "type_name": "ping_delay_disconnect#f3427b8c", "msg_id": 6917626334828287032}
2021-01-14T22:43:12.648+0800    DEBUG   mtproto/handle_message.go:18    Handle message  {"type_id": "0x347773c5", "type_name": "pong#347773c5"}
2021-01-14T22:43:12.648+0800    DEBUG   mtproto/ping.go:36      Pong

gen: incorrect bool decoder generation for some objects

For example: https://core.telegram.org/constructor/peerNotifySettings.

Should be:

  func (p *PeerNotifySettings) Decode(b *bin.Buffer) error {
	  if p == nil {
		  return fmt.Errorf("can't decode peerNotifySettings#af509d20 to nil")
	  }
	  if err := b.ConsumeID(PeerNotifySettingsTypeID); err != nil {
		  return fmt.Errorf("unable to decode peerNotifySettings#af509d20: %w", err)
	  }
	  {
		  if err := p.Flags.Decode(b); err != nil {
			  return fmt.Errorf("unable to decode peerNotifySettings#af509d20: field flags: %w", err)
		  }
	  }
	  if p.Flags.Has(0) {
		  value, err := b.Bool()
		  if err != nil {
			  return fmt.Errorf("unable to decode peerNotifySettings#af509d20: field show_previews: %w", err)
		  }
		  r.ShowPreviews = value
	  }
	  if p.Flags.Has(1) {
		  value, err := b.Bool()
		  if err != nil {
			  return fmt.Errorf("unable to decode peerNotifySettings#af509d20: field silent: %w", err)
		  }
		  r.Silent = value
	  }
	  if p.Flags.Has(2) {
		  value, err := b.Int()
		  if err != nil {
			  return fmt.Errorf("unable to decode peerNotifySettings#af509d20: field mute_until: %w", err)
		  }
		  p.MuteUntil = value
	  }
	  if p.Flags.Has(3) {
		  value, err := b.String()
		  if err != nil {
			  return fmt.Errorf("unable to decode peerNotifySettings#af509d20: field sound: %w", err)
		  }
		  p.Sound = value
	  }
	  return nil
  }

Now:

func (p *PeerNotifySettings) Decode(b *bin.Buffer) error {
if p == nil {
return fmt.Errorf("can't decode peerNotifySettings#af509d20 to nil")
}
if err := b.ConsumeID(PeerNotifySettingsTypeID); err != nil {
return fmt.Errorf("unable to decode peerNotifySettings#af509d20: %w", err)
}
{
if err := p.Flags.Decode(b); err != nil {
return fmt.Errorf("unable to decode peerNotifySettings#af509d20: field flags: %w", err)
}
}
p.ShowPreviews = p.Flags.Has(0)
p.Silent = p.Flags.Has(1)
if p.Flags.Has(2) {
value, err := b.Int()
if err != nil {
return fmt.Errorf("unable to decode peerNotifySettings#af509d20: field mute_until: %w", err)
}
p.MuteUntil = value
}
if p.Flags.Has(3) {
value, err := b.String()
if err != nil {
return fmt.Errorf("unable to decode peerNotifySettings#af509d20: field sound: %w", err)
}
p.Sound = value
}
return nil
}

gen: rework String() derive

For now, generator generates a bit incorrect String() method without correct indentation.
Due to we want to support not only fmt.Stringer but also, for example, JSON encoding, I propose to add interfaces like

type Object interface {
	Format(f Formatter) error
}

type ArrayFormatter interface {
	AppendInt8(v int8) error
	AppendInt16(v int16) error
	AppendInt32(v int32) error
	AppendInt64(v int64) error
	AppendInt(v int) error

	AppendUint8(v uint8) error
	AppendUint16(v uint16) error
	AppendUint32(v uint32) error
	AppendUint64(v uint64) error
	AppendUint(v uint) error

	AppendBool(v bool) error
	AppendString(v string) error
	AppendBytes(v []byte) error

	AppendObject(v Object) error
}

type Formatter interface {
	FormatInt8(name string, v int8) error
	FormatInt16(name string, v int16) error
	FormatInt32(name string, v int32) error
	FormatInt64(name string, v int64) error
	FormatInt(name string, v int) error

	FormatUint8(name string, v uint8) error
	FormatUint16(name string, v uint16) error
	FormatUint32(name string, v uint32) error
	FormatUint64(name string, v uint64) error
	FormatUint(name string, v uint) error

	FormatBool(name string, v bool) error
	FormatString(name string, v string) error
	FormatBytes(name string, v []byte) error

	FormatArray(name string, v func(f ArrayFormatter) error) error
	FormatSub(name string, v func(f Formatter) error) error
}

gen: automatic optional fields

Automatically set optional (#) fields if they are not blank

Currently we should use setters to set optional fields:

// SetPrivacyPolicyURL sets value of PrivacyPolicyURL conditional field.
func (a *AccountAuthorizationForm) SetPrivacyPolicyURL(value string) {
	a.Flags.Set(0)
	a.PrivacyPolicyURL = value
}

So if one just set PrivacyPolicyURL directly, not via SetPrivacyPolicyURL, value will be never encoded.

This is annoying and suprising. We can make those fields private and expose only methods, but I think there should be better approach.

So, we should update gen/make_field.go and main.tmpl.

Zero values

We can't just use zero value, becuase some stucts contain slices and == operation is not defined on them.
So, probably we can generate Blank() bool method for each structure

Bitfields

We can't just change

if a.Flags.Has(0) {
	b.PutString(a.PrivacyPolicyURL)
}

To

if a.Flags.Has(0) || a.PrivacyPolicyURL != "" {
	b.PutString(a.PrivacyPolicyURL)
}

Because we should update a.Flags value, so probable solution is 2-step encoding, like

// For each conditional field, automatically set flags:
if a.PrivacyPolicyURL != "" {
	a.Flags.Set(0)
}

// THEN encode a.Flags
if err := a.Flags.Encode(b); err != nil {
	return fmt.Errorf("unable to encode account.authorizationForm#ad2e1cd8: field flags: %w", err)
}

// THEN just unchanged flow
if a.Flags.Has(0) {
	b.PutString(a.PrivacyPolicyURL)
}

client: support all constructors of Updates type

For now, read loop does not handle all constructors(e.g. updateShort) of TL type Updates.

td/telegram/read_loop.go

Lines 58 to 76 in 8d4b634

switch id {
case mt.BadMsgNotificationTypeID, mt.BadServerSaltTypeID:
return c.handleBadMsg(b)
case proto.MessageContainerTypeID:
return c.processBatch(b)
case mt.NewSessionCreatedTypeID:
return c.handleSessionCreated(b)
case proto.ResultTypeID:
return c.handleResult(b)
case mt.PongTypeID:
return c.handlePong(b)
case mt.MsgsAckTypeID:
return c.handleAck(b)
case proto.GZIPTypeID:
return c.handleGZIP(b)
case tg.UpdatesTypeID:
return c.handleUpdates(b)
default:
return c.handleUnknown(b)

gen: type instantiation in registry

Should be in two parts: tmap and generated code.

Possible API:

// in bin package:

// Object wraps Decoder and Encoder interface.
type Object interface {
	Decoder
	Encoder
}

// Stringer probably should be optional.


// example:

types := tmap.New(
	mt.TypesMap(),
	tg.TypesMap(),
	proto.TypesMap(),
)

// v is bin.Object or nil otherwise
v := types.New(id)

Should be possible to instantiation every type (including requests and responses) by type id.

client: migration on MTProxy

For now, DC migration on MTProxy does not work correctly due to MTProxy uses DC ID as address, not TCP/IP address.

client: optimize encryption

Currently encryption of every message requires allocation, because each message has different encryption key.

We can't reuse AES buffer due to crypto/cipher design.

Related issue in go:
golang/go#39365

client: implement updates handling

This is epic for client updates dispatching feature.

Description

Client should provide exactly once (or at least once, TBD) guarantees for updates handling via dispatcher.

Also there should be (auto-generated) helpers that deal with Telegram entities, like InputPeer

Reference

TODO

  • Support all Update types (including short ones)
  • Support "fetch-only" connections (i.e. invokeWithoutUpdates)
  • Define interface for session storage
  • Trigger state check on reconnect
  • Trigger state check on long update absence (1 minute+? docs stayts 15min)
  • TBD

Handling state

  1. For new state, generate pts=0(1?) and do not fetch difference, we want to process only new events starting for this moment
  2. If fresh, get state and check for difference
  3. If we detect gap, call getDifference (or wait for ~0.5s, incoming updates may fix the gap)
  4. If we get channelDifferenceTooLong, we should act same as in (1)
    1. If we are bot, that means that all unprocessed messages for bot are lost at this moment
    2. If we are user, fetch latest history from chats

Accept interface instead of *zap.Logger

Right now the td library doesn't accept any other logging engine except the concrete zap logger, and the whole situation around the logging seems not enough scalable. For instance, if somebody wants to use a standard logger, it will be upset by the absence of abstraction around the logger.

client: How send media

Hi, please share an example of how to send a media (picture) to a specific recipient?
tg. Client has a method MessagesSendMedia, it is not clear how it works and whether it works at all (for example, MessagesSendMessage does not work)

gen: type-safe rpc errors

Currently, documentation contains information about possible errors.

We should generate code for them to reduce hard-coding things like comparing error type to "STATS_MIGRATE".

Probably we can even do following:

if tg.IsStatsMigrate(err) {
    // ...
}

rpc error code 400: PHONE_NUMBER_BANNED"

I use td to login in with my telegram account.

it can get some messages. but after a few seconds , I got PHONE_NUMBER_BANNED.
And my account is Banned.

just use these codes:
return client.Run(ctx, func(ctx context.Context) error {
codePrompt := func(ctx context.Context) (string, error) {
// NB: Use "golang.org/x/crypto/ssh/terminal" to prompt password.
fmt.Print("Enter code: ")
code, err := bufio.NewReader(os.Stdin).ReadString('\n')
if err != nil {
return "", err
}
return strings.TrimSpace(code), nil
}
if self, err := client.Self(ctx); err != nil {
if err := telegram.NewAuth(
telegram.CodeOnlyAuth("my_phone_number", telegram.CodeAuthenticatorFunc(codePrompt)),
telegram.SendCodeOptions{},
).Run(ctx, client); err != nil {
log.Error("auth: %s", err.Error())
return err
}
}

	c := tg.NewClient(client)
	for range time.NewTicker(time.Second * 5).C {
		chats, err := c.MessagesGetAllChats(ctx, nil)

		var rpcErr *mtproto.Error
		if errors.As(err, &rpcErr) && rpcErr.Type == "FLOOD_WAIT" {
			// Server told us to wait N seconds before sending next message.
			log.Infow("Sleeping", "seconds", rpcErr.Argument)
			time.Sleep(time.Second * time.Duration(rpcErr.Argument))
			continue
		}

		if err != nil {
			log.Error("failed to get chats: %s", err.Error())
			return err
		}

		switch chats.(type) {
		case *tg.MessagesChats: // messages.chats#64ff9fd5
			log.Info("Chats")
		case *tg.MessagesChatsSlice: // messages.chatsSlice#9cd81144
			log.Info("Slice")
		}
	}

	return nil
})

bin: invalid string decoding

Telegram docs says:

If L <= 253, the serialization contains one byte with the value of L, then L bytes of the string followed by 0 to 3 characters containing 0, such that the overall length of the value be divisible by 4, whereupon all of this is interpreted as a sequence of int(L/4)+1 32-bit numbers.
If L >= 254, the serialization contains byte 254, followed by 3 bytes with the string length L, followed by L bytes of the string, further followed by 0 to 3 null padding bytes.

bin.String():

if strLen >= maxSmallStringLength {

Same at bin.Bytes():

if vLen >= maxSmallStringLength {

Should be strLen > maxSmallStringLength.
In case when L is equal to 253, we get an unexpected error.

client: handle AUTH_KEY_UNREGISTERED

Failed to get state after reconnect	{"dc": 5, "error": "rpcDoRequest: rpc error code 401: AUTH_KEY_UNREGISTERED", "errorVerbose": "rpcDoRequest:\n    github.com/gotd/td/telegram.(*Client).rpcDo\n        /go/pkg/mod/github.com/gotd/[email protected]/telegram/rpc.go:45\n  - rpc error code 401: AUTH_KEY_UNREGISTERED"}

gen: sugared RPC request builder

For now use to make request we have to write something like:

	_, err = invoker.MessagesSendMedia(ctx, &tg.MessagesSendMediaRequest{
		Peer: &tg.InputPeerChannel{
			ChannelID:  ch.ID,
			AccessHash: ch.AccessHash,
		},
		Media: &tg.InputMediaUploadedDocument{
			File: f,
			Attributes: []tg.DocumentAttributeClass{
				&tg.DocumentAttributeFilename{FileName: "video.mp4"},
			},
		},
		RandomID: id,
	})

and there are some problems

  • You need to write MessagesSendMedia(ctx, ...) instead of

    messages := client.Messages(tgs.Ctx(ctx))
    messages.SendMedia(...).Do()

    or

    messages := client.Messages()
    messages.SendMedia(...).DoCtx(ctx)
  • You need to write types tg.InputMediaUploadedDocument or tg.MessagesSendMediaRequest instead of pretty methods like SendMedia or UploadedDocument.

  • You need to fill fields like RandomID manually, but it can be filled automatically using given RandomIDSource

  • Field Peer can be using different structures, so we can use mappers from #144 to provide multiple filling methods

  • Some classes in schema have a empty constructor which can be used by default, for example InputMessagesFilterEmpty

  • We can use zero values as valid fields in some cases even if these fields required by schema, for example message.sendMedia does not really require non-empty message field.

Finally, the sugared RPC call would be like

rpc.Messages(tgs.Ctx(ctx), tgs.ID(rand.Reader)).
  SendMedia().
  PeerChannel(id, hash).
  MediaDocument(f).Do()

mtproto: returning 404 error on zero session

Telegram server can return Error 404 (auth key not found) error when session id is equal to 0.

This corresponds to "CodeAuthKeyNotFound" (auth key not found) error.

Just creating this issue for myself and other folks that will be desperately googling this kekikus maximus condition.

JUST GENERATE RANDOM SESSION ID ON -404 ERROR

Server error code 404.

client: hang on reconnect

Hi, I have an application in which your client is taken as a basis, the application sends messages to the recipients on a schedule, the problem is that sometimes it stops sending, although the application itself continues to work. In the logs here's what:

{"level":"error","timestamp":"2021-01-13T08:51:38.589Z","logger":"rpc.retry","msg":"Retry failed","msg_id":6917164630320032880,"error":"write: write intermediate: write tcp 172.17.220.94:41784->149.154.167.50:443: write: broken pipe","errorVerbose":"write:\n    github.com/gotd/td/transport.(*connection).Send
        /tmp/codon/tmp/cache/go-path/pkg/mod/github.com/gotd/[email protected]/transport/connection.go:39
  - write intermediate:\n    github.com/gotd/td/internal/proto/codec.Intermediate.Write
        /tmp/codon/tmp/cache/go-path/pkg/mod/github.com/gotd/[email protected]/internal/proto/codec/intermediate.go:57
  - write tcp 172.17.220.94:41784->149.154.167.50:443: write: broken pipe"}
{"level":"error","timestamp":"2021-01-13T08:51:38.590Z","msg":"Failed to init connection after reconnect","error":"request: rpcDoRequest: retryUntilAck: write: write intermediate: write tcp 172.17.220.94:41784->149.154.167.50:443: write: broken pipe","errorVerbose":"request:\n    github.com/gotd/td/telegram.(*Client).initConnection
        /tmp/codon/tmp/cache/go-path/pkg/mod/github.com/gotd/[email protected]/telegram/init_connection.go:33
  - rpcDoRequest:\n    github.com/gotd/td/telegram.(*Client).rpcDo
        /tmp/codon/tmp/cache/go-path/pkg/mod/github.com/gotd/[email protected]/telegram/rpc.go:45
  - retryUntilAck:\n    github.com/gotd/td/telegram/internal/rpc.(*Engine).Do
        /tmp/codon/tmp/cache/go-path/pkg/mod/github.com/gotd/[email protected]/telegram/internal/rpc/engine.go:141
  - write:\n    github.com/gotd/td/transport.(*connection).Send
        /tmp/codon/tmp/cache/go-path/pkg/mod/github.com/gotd/[email protected]/transport/connection.go:39
  - write intermediate:\n    github.com/gotd/td/internal/proto/codec.Intermediate.Write
        /tmp/codon/tmp/cache/go-path/pkg/mod/github.com/gotd/[email protected]/internal/proto/codec/intermediate.go:57
  - write tcp 172.17.220.94:41784->149.154.167.50:443: write: broken pipe"}
{"level":"warn","timestamp":"2021-01-13T08:52:33.589Z","logger":"pinger","msg":"ping error","error":"write: write: write intermediate: write tcp 172.17.220.94:41784->149.154.167.50:443: write: broken pipe","errorVerbose":"write:\n    github.com/gotd/td/telegram.(*Client).pingDelayDisconnect
        /tmp/codon/tmp/cache/go-path/pkg/mod/github.com/gotd/[email protected]/telegram/ping.go:61
  - write:\n    github.com/gotd/td/transport.(*connection).Send
        /tmp/codon/tmp/cache/go-path/pkg/mod/github.com/gotd/[email protected]/transport/connection.go:39
  - write intermediate:\n    github.com/gotd/td/internal/proto/codec.Intermediate.Write
        /tmp/codon/tmp/cache/go-path/pkg/mod/github.com/gotd/[email protected]/internal/proto/codec/intermediate.go:57
  - write tcp 172.17.220.94:41784->149.154.167.50:443: write: broken pipe"}

gen: improve camel case to pascal

Upper version of cdnConfig should be CDNConfig, not CdnConfig.

In general, snake to pascal/camel is good, but lower camel to upper camel is bad.
Probably we should split words by upper case, like fooBarBaz to foo Bar Baz.

client: handle USER_MIGRATE

Incomplete code implementation
When I tested my code, I passed the proxy connection to the server, and I always received the 303 message at the rpcDo function.

According to my reading of the code https://github.com/Lonami/grammers,
https://github.com/Lonami/grammers/blob/master/lib/grammers-client/src/client/auth.rs
bot_sign_in function

Err(InvocationError::Rpc(RpcError { name, value, .. })) if name == "USER_MIGRATE" => {
                self.config.session.auth_key = None;
                self.sender = connect_sender(value.unwrap() as i32, &mut self.config).await?;
                self.invoke(&request).await?
            }

When the 303 message is received, the rpc connection should be switched to the parameter number dc server in the 303 message.
There is no such operation in the current code. Since I am not familiar with the code you wrote, I cannot complete this part of the code yet, so please add it yourself.

The echo example has worked, thank you for your contribution.

gotdecho: infinite recursion on self messages

td/cmd/gotdecho/main.go

Lines 58 to 77 in e271a8d

dispatcher.OnNewMessage(func(ctx tg.UpdateContext, u *tg.UpdateNewMessage) error {
switch m := u.Message.(type) {
case *tg.Message:
switch peer := m.PeerID.(type) {
case *tg.PeerUser:
user := ctx.Users[peer.UserID]
logger.Info("Got message", zap.String("text", m.Message),
zap.Int("user_id", user.ID),
zap.String("user_first_name", user.FirstName),
zap.String("username", user.Username))
return client.SendMessage(ctx, &tg.MessagesSendMessageRequest{
Message: m.Message,
Peer: &tg.InputPeerUser{
UserID: user.ID,
AccessHash: user.AccessHash,
},
})
}
}

updates, err := c.tg.MessagesSendMessage(ctx, req)
if err != nil {
return err
}
return c.processUpdates(updates)

Handler does not check that message update is Out.
Possible solutions:

  • Add a check.
        switch m := u.Message.(type) { 
        case *tg.Message: 
                   if m.Out {
                        return nil
                   }
  • Do not handle Updates from RPC calls.

client: pagination helpers

https://core.telegram.org/api/offsets

There are a lot pageable methods in Telegram API, so we can provide helper using one of these ways:

  • Define a common interfaces for every pagination method using generated getters and setters and write pager manually
  • Derive some methods for pageable RPC methods using gotdgen and write pager manually
  • Generate helpers using code generator

Telegram pagination is specific and hard to write, so maybe some pagers have to be written manually

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.