ail-project/ail-feeder-telegram

External telegram feeder for AIL framework

ail-feeder-telegram

External telegram feeder for AIL framework (with an automated user account)

• Forum channels
• Replies Threads
• Images Download
• Users + channels meta and profile picture
• Files download (Saved on disk)

Install

• Install python3 dependencies:

pip3 install -U -r requirements.txt

• Copy config file:

cp etc/conf.cfg.sample etc/conf.cfg

Configuration

Add your telegram account in etc/conf.cfg:

• Login in with your telegram phone number
• Click under API Development tools.
• Create new application
• Fill App title and Short name in your application details
• Create application and retrieve your API ID and Hash
• Add your API ID and Hash in etc/conf.cfg

⚠️ Telegram API hash is secret and can't be revoked ⚠️
You can use this API ID and hash with any phone number or even for bot accounts

Usage

feeder.py

• chats ( List all joined chats_ )
• join ( Join a chat by its name or invite hash )
  • --name [Chat name] ( chat name/username )
  • --invite [Invite Hash] ( chat invite hash )
• leave [Channel] ( Leave a Chat )
• check [Invite Hash] ( Check an invite hash/chat without joining )
• messages [Chat ID/username] ( Get all messages from a chat )
  • --min_id [Message ID] ( Filter: Message Minimal ID )
  • --max_id [Message ID] ( Filter: Message Maximum ID )
  • --replies ( Get replies )
  • --mark_as_read ( Mark messages as read )
  • --media ( Download medias )
  • --save_dir ( Directory to save downloaded medias )
• message [Chat ID/username] [Message ID] ( Get a message from a chat )
  • --replies ( Get replies )
  • --mark_as_read ( Mark message as read )
  • --media ( Download medias )
  • --save_dir ( Directory to save downloaded medias )
• monitor ( Monitor all joined chats )
• unread ( Get all unread messages from all chats and mark them as read )
• chat [Chat ID/username] ( Get a chat metadata, list of users/admins, ... )
  • --users ( Get a list of all the users of a chat )
  • --admins ( Get a list of all the admin users of a chat )
  • --similar ( Get a list of similar/recommended chats )
• entity [Entity ID/username] ( Get chat or user metadata )
• search [to_search] ( Search for public Chats/Users )

Joining Channels

python3 bin/feeder.py join -n CHANNEL_USERNAME 

python3 bin/feeder.py join -i INVITE_HASH 

Channels can also be joined from the mobile application on Apple or Android. Once the script is re-run, the newly joined channel will be added to the messages queue.

Leaving Channels

python3 bin/feeder.py leave CHANNEL_USERNAME/CHANNEL_ID 

Channels can also be left from the mobile application on Apple or Android. If you leave a channel whilst the script is running there will likely be an exception error.

Get all Channels

python3 bin/feeder.py chats 

Running this action will export a python list of channel IDs your Telegram account has joined.

If you join too many channels, too quickly, you might experience a waiting period before you can join a new one.

Get Channel Messages

python3 bin/feeder.py messages CHANNEL_USERNAME/CHANNEL_ID

MONITOR Messages from all Joined Channels

python3 bin/feeder.py monitor

ail-feeder-telegram's Issues

Implement better error message when DB is locked

When the sqlite DB is locked, exit graciously (and earlier)

catch: sqlite3.OperationalError: database is locked

./feeder.py chats
Traceback (most recent call last):
  File "/home/ail/ail-feeder-telegram/bin/./feeder.py", line 138, in <module>
    tg.connect()
  File "/home/ail/ail-feeder-telegram/bin/telegram.py", line 93, in connect
    self.client.start()
  File "/home/ail/ail-feeder-telegram/venv/lib/python3.11/site-packages/telethon/client/auth.py", line 128, in start
    else self.loop.run_until_complete(coro)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/ail/ail-feeder-telegram/venv/lib/python3.11/site-packages/telethon/client/auth.py", line 135, in _start
    await self.connect()
  File "/home/ail/ail-feeder-telegram/venv/lib/python3.11/site-packages/telethon/client/telegrambaseclient.py", line 555, in onnect
    self.session.auth_key = self._sender.auth_key
    ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ail/ail-feeder-telegram/venv/lib/python3.11/site-packages/telethon/sessions/sqlite.py", line 180, in auth_key
    self._update_session_table()
  File "/home/ail/ail-feeder-telegram/venv/lib/python3.11/site-packages/telethon/sessions/sqlite.py", line 194, in _update_session_table
    c.execute('delete from sessions')
sqlite3.OperationalError: database is locked
Task was destroyed but it is pending!
task: <Task pending name='Task-3' coro=<Connection._send_loop() running at /home/ail/ail-feeder-telegram/venv/lib/python3.11/site-packages/telethon/network/connection/connection.py:316> wait_for=<Future pending cb=[Task.task_wakeup()]>>
Task was destroyed but it is pending!
task: <Task pending name='Task-4' coro=<Connection._recv_loop() running at /home/ail/ail-feeder-telegram/venv/lib/python3.11/site-packages/telethon/network/connection/connection.py:335> wait_for=<Future pending cb=[Task.task_wakeup()]>>
Task was destroyed but it is pending!
task: <Task pending name='Task-5' coro=<MTProtoSender._send_loop() running at /home/ail/ail-feeder-telegram/venv/lib/python3.11/site-packages/telethon/network/mtprotosender.py:464> wait_for=<Future pending cb=[Task.task_wakeup()]>>
Task was destroyed but it is pending!
task: <Task pending name='Task-6' coro=<MTProtoSender._recv_loop() running at /home/ail/ail-feeder-telegram/venv/lib/python3.11/site-packages/telethon/network/mtprotosender.py:507> wait_for=<Future pending cb=[Task.task_wakeup()]>>
Exception ignored in: <coroutine object Connection._recv_loop at 0x7326305854e0>
RuntimeError: coroutine ignored GeneratorExit

Add possibility to monitor a specific subchannel only

Hello,

Is there a way to monitor a specific subchannel of a group of chat ?

How can I monitor 'subschan2' without moniroting other subchan ?
MyChannel
--> subchan1
--> subchan2
--> subchan3

Regards,

TG API Throttle

Try to run the command python3 feeder.py --getall
everything start, but after a while I start seeing error like that

ERROR [api.py:116 - _check_response() ] Something went wrong (401): {'reason': 'Max Connection Attempts reached, Please wait 299s', 'status': 'error'}

and it crash after saying this

-752095614
Traceback (most recent call last):
File "feeder.py", line 155, in
client.loop.run_until_complete(get_entity_messages(client, entity, min_id=min_id, max_id=max_id))
File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "feeder.py", line 77, in get_entity_messages
await telegram.get_all_channel_messages(client, entity, pyail, min_id=min_id, max_id=max_id, feeder_uuid=feeder_uuid)
File "/home/********/ail-feeder-telegram/bin/telegram.py", line 199, in get_all_channel_messages
dict_meta['channel_id'] = message.input_chat.channel_id
AttributeError: 'InputPeerChat' object has no attribute 'channel_id'

Please add detailed usage info

Please add detailed usage infos with examples.
Is it possible to refeed telegram links found by the AIL-framework?