joj0/synadm

Command line admin tool for Synapse (the Matrix reference homeserver)

synadm's Issues

Make it possible to change user type

It would be great to be able to modify the user_type field of users.

Reduce postgresql database size?

Hello,

My synapse postgres database is consuming quite a bit of space. Is there any way to find out which room is the reason and how can i trim it down?

Thanks

Config file/directory is created world-readable

The ~/.config/synadm.yaml file (and the ~/.config/directory, if it doesn't already exist) is created with default permissions, so with a typical umask ends up world-readable. But it contains the admin user's access token, so should be explicitly created with mode 0o600 (or 0o700 for the directory).

Add ability to call custom GET/POST command via synadm script

Will be good to use same synadm tool not only for pre-defined API calls, but for custom too, for not to create custom scripts for each call.
For example, allow something like this:

synadm custom-api-call --type POST --path "/_synapse/admin/v1/media/ru-matrix.org/delete?before_ts=1602745639000" --body "{}"

where make default argument value for type as GET, body = {}.

This way will be much more comfortable, than composing full curl command in cli manually and remember your token each time.

Especially this is useful at now, when some of popular API calls are not implemented in build-in commands.

Update to use new API for deleting a room

Currently, the old POST method is used: self.query("post", f"v1/rooms/{room_id}/delete", data=data) (https://github.com/JOJ0/synadm/blob/master/synadm/api.py#L384).

The new route is
DELETE /_synapse/admin/v1/rooms/<room_id>.

See matrix-org/synapse#9889.

Unable to delete room (status code 500)

I am opening a new issue as I think this might be a different problem to the other 'unable to delete room' issue that's currently open.

Using 'synadm room delete 'room-id'' I get the following from synadm:

WARNING Synapse returned status code 500
errcode  M_UNKNOWN
error    Internal server error

I can see the room details before that message, so the room details are accessible.

The HS log shows this:

Traceback (most recent call last):
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/synapse/http/server.py", line 366, in _async_render_wrapper
    callback_return = await self._async_render(request)
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/synapse/http/server.py", line 572, in _async_render
    callback_return = await raw_callback_return
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/synapse/rest/admin/rooms.py", line 311, in on_DELETE
    return await self._delete_room(
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/synapse/rest/admin/rooms.py", line 368, in _delete_room
    await pagination_handler.purge_room(room_id, force=force_purge)
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/synapse/handlers/pagination.py", line 417, in purge_room
    await self.storage.purge_events.purge_room(room_id)
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/synapse/storage/purge_events.py", line 36, in purge_room
    state_groups_to_delete = await self.stores.main.purge_room(room_id)
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/synapse/storage/databases/main/purge_events.py", line 320, in purge_room
    return await self.db_pool.runInteraction(
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/synapse/storage/database.py", line 834, in runInteraction
    return await delay_cancellation(_runInteraction())
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/twisted/internet/defer.py", line 1656, in _inlineCallbacks
    result = current_context.run(
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/twisted/python/failure.py", line 514, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/synapse/storage/database.py", line 806, in _runInteraction
    result = await self.runWithConnection(
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/synapse/storage/database.py", line 929, in runWithConnection
    return await make_deferred_yieldable(
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/twisted/python/threadpool.py", line 244, in inContext
    result = inContext.theWork()  # type: ignore[attr-defined]
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/twisted/python/threadpool.py", line 260, in <lambda>
    inContext.theWork = lambda: context.call(  # type: ignore[attr-defined]
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/twisted/python/context.py", line 117, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/twisted/python/context.py", line 82, in callWithContext
    return func(*args, **kw)
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/twisted/enterprise/adbapi.py", line 282, in _runWithConnection
    result = func(conn, *args, **kw)
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/synapse/storage/database.py", line 922, in inner_func
    return func(db_conn, *args, **kwargs)
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/synapse/storage/database.py", line 670, in new_transaction
    r = func(cursor, *args, **kwargs)
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/synapse/storage/databases/main/purge_events.py", line 328, in _purge_room_txn
    txn.execute("DELETE FROM rooms WHERE room_id = ?", (room_id,))
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/synapse/storage/database.py", line 352, in execute
    self._do_execute(self.txn.execute, sql, *args)
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/synapse/storage/database.py", line 394, in _do_execute
    return func(sql, *args, **kwargs)
psycopg2.errors.ForeignKeyViolation: update or delete on table "rooms" violates foreign key constraint "destination_rooms_room_id_fkey" on table "destination_rooms"
DETAIL:  Key (room_id)=(!zjYxZkVEqwWcQQhXxc:techlore.net) is still referenced from table "destination_rooms".

Add an option to filter out empty rooms in room list

as discussed in #synadm:peek-a-boo.at (specific message)

To be decicded:

• Should it really be an option in synadm room list that just shows empty rooms and purging has to be done using jq/shell magic/looping and invoking synadm room delete?
• or rather be a separate command named something like synadm room purge-empty (similar to synadm user prune-devices
• Or even both?

Add `-y` option for force answer "Yes" when using synadm tool in other scripts

Will be good to add -y option for force answer "Yes" without user interactions when using synadm tool in other scripts.

Reintroduce the --servername option to the `media` commands

...since matrix-docker-ansible-deploy users get failing servername fetches due to the server-server api being used to fetch is not enabled by default.

next steps: auto-fetch servername via different ways:

• .well-known
• dns srv record

0.42 not on PyPi

0.42 has been released a month ago and PyPi is still on 0.41.3 : https://pypi.org/project/synadm/#history

How to get more detailed help of some command?

There is exist a general help command, that shows the help text:

synadm -h

In script code I see more detailed help text for commands, for example for room delete command:

synadm/synadm.py

Line 900 in 45b1220

@click.option('--new-room-user-id', '-u', type=str,

But I can't understand how to show it via command line arguments?
I try those:

synadm room delete
synadm -h room
synadm -h room delete

But no one shows me the detailed help text with list of available arguments.

missing/undocumented dependency: setuptools

Following the installation process on my server (Debian 10) I get the following error:

# python3 setup.py install
Traceback (most recent call last):
  File "setup.py", line 18, in <module>
    from setuptools import setup, find_packages
ModuleNotFoundError: No module named 'setuptools'

The solution for me was apt install python3-setuptools.

Support new API's to query room delete status

Add one new subcommand supporting these two API's:

https://matrix-org.github.io/synapse/latest/admin_api/rooms.html#query-by-room_id
https://matrix-org.github.io/synapse/latest/admin_api/rooms.html#query-by-delete_id

Abbreviations of output formats

Abbreviating an output format should be possible

$ synadm -o pp user search test
Usage: synadm [OPTIONS] COMMAND [ARGS]...
Try "synadm -h" for help.

Error: Invalid value for "--output" / "-o": invalid choice: pp. (choose from yaml, json, human, pprint)

default output format not working

On first start it asks for a default output format (I put json) but subsequently it always outputs in yaml mode. Manually specifying the output format works correctly.

media list throws a KeyError in human output mode

Happens when synadm media list -u 'unknown/not valid user' is called.

(synadm) jojo@jum ~/git/govynnus_synadm (regtok) $ synadm -vv -o human media list -u '123'
DEBUG Config entry read. user: admin
DEBUG Config entry read. token: SECRET
DEBUG Config entry read. base_url: http://localhost:8008
DEBUG Config entry read. admin_path: /_synapse/admin
DEBUG Config entry read. matrix_path: /_matrix
DEBUG Config entry read. timeout: 3600
DEBUG Config entry read. format: yaml
DEBUG Formatter in use: human - <function humanize at 0x7f6c5260cea0>
INFO  Querying get on http://localhost:8008/_synapse/admin/v1/users/123/media
WARNING Synapse returned status code 400
Traceback (most recent call last):
  File "/home/jojo/.venvs/synadm/bin/synadm", line 11, in <module>
    load_entry_point('synadm', 'console_scripts', 'synadm')()
  File "/home/jojo/.venvs/synadm/lib/python3.6/site-packages/click-7.1.2-py3.6.egg/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/jojo/.venvs/synadm/lib/python3.6/site-packages/click-7.1.2-py3.6.egg/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/jojo/.venvs/synadm/lib/python3.6/site-packages/click-7.1.2-py3.6.egg/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/jojo/.venvs/synadm/lib/python3.6/site-packages/click-7.1.2-py3.6.egg/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/jojo/.venvs/synadm/lib/python3.6/site-packages/click-7.1.2-py3.6.egg/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/jojo/.venvs/synadm/lib/python3.6/site-packages/click-7.1.2-py3.6.egg/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/jojo/.venvs/synadm/lib/python3.6/site-packages/click-7.1.2-py3.6.egg/click/decorators.py", line 33, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/home/jojo/.venvs/synadm/lib/python3.6/site-packages/click-7.1.2-py3.6.egg/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/jojo/git/govynnus_synadm/synadm/cli/media.py", line 83, in media_list_cmd
    from_=from_, limit=limit, sort=sort, reverse=reverse)
  File "/home/jojo/.venvs/synadm/lib/python3.6/site-packages/click-7.1.2-py3.6.egg/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/jojo/.venvs/synadm/lib/python3.6/site-packages/click-7.1.2-py3.6.egg/click/decorators.py", line 33, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/home/jojo/git/govynnus_synadm/synadm/cli/user.py", line 394, in user_media_cmd
    .format(media["total"]))
KeyError: 'total'

auto discover own server name

The media delete apis need the own server name to be given as an argument.

I have solved this by instead grabbing it via server_name = requests.get(f"{self.base_url}/_matrix/key/v2/server").json()['server_name'] instead. If you want I can submit a patch for this.

User modify API requires a redesign

https://github.com/gergelypolonkai/synadm/blob/dba8132dd74fe4be1129a78e07dd26806a6331d8/synadm/api.py#L536-L561

None is not considered truthy, so using None as "default" would cause the values to never change. This makes for a poorly functional API that is also very poorly defined (the severe lack of docs is included).

Basically, we have a problem that has nothing to do with this PR, but the PR has the same problems.
We can merge this PR without fixing the issue, but we definitely need to fix the issue which is more than just user types (trying to remove admin doesn't work because of a similar issue (False will not pass an implicit if condition)).

The user modify API and command works together poorly, unable to remove admin rights (too lazy to write that word). This is because it's using implicit if conditions, where values of False and None is used as defined values, but is interpreted as non-defined values.

Example: you can give admin, not remove. you can set user type (#90), you can't revert to default user type.

Originally posted by @JacksonChen666 in #90 (comment)

add user media command

https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/user_admin_api.rst#list-media-of-an-user

I actually have a patch implementing this... need to clean it up a bit. 🎉

Add useful raw SQL database queries to synadm tool

Some useful data about Synapse statistics isn't available via API calls, but only via raw SQL queries to database.
Will be good to extend synadm tool for integrate some SQL queries into it, here is list of some popular queries: https://github.com/matrix-org/synapse/wiki/SQL-for-analyzing-Synapse-PostgreSQL-database-stats

From that list, especially useful are commands for check last room and user activity:

• Show top 20 rooms by new events count in last 1 day
• Show top 20 users on homeserver by sent events (messages) at last month

Support new API's to block and unblock rooms

One new subcommand to block and unblock rooms: https://matrix-org.github.io/synapse/latest/admin_api/rooms.html#block-or-unblock-a-room

One new subcommand to query blocking status: https://matrix-org.github.io/synapse/latest/admin_api/rooms.html#get-block-status

Make sure when issued on pre-1.48. Synapse, synadm prevents issuing these API calls.

Unable to delete room

I'm trying to use synadm to delete a room from my server, but I keep getting that the following:

$ synadm room delete OMITTED-ROOM_ID
WARNING Synapse returned status code 404
errcode  M_NOT_FOUND
error    Room not found
WARNING Synapse returned status code 404
Traceback (most recent call last):
  File "/usr/local/bin/synadm", line 33, in <module>
    sys.exit(load_entry_point('synadm==0.34', 'console_scripts', 'synadm')())
  File "/usr/local/lib/python3.9/dist-packages/click-7.1.2-py3.9.egg/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/click-7.1.2-py3.9.egg/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.9/dist-packages/click-7.1.2-py3.9.egg/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.9/dist-packages/click-7.1.2-py3.9.egg/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.9/dist-packages/click-7.1.2-py3.9.egg/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.9/dist-packages/click-7.1.2-py3.9.egg/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/click-7.1.2-py3.9.egg/click/decorators.py", line 33, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/click-7.1.2-py3.9.egg/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/dietpi/.local/lib/python3.9/site-packages/synadm/cli/room.py", line 249, in delete
    ctx.invoke(members, room_id=room_id)
  File "/usr/local/lib/python3.9/dist-packages/click-7.1.2-py3.9.egg/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/click-7.1.2-py3.9.egg/click/decorators.py", line 33, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/home/dietpi/.local/lib/python3.9/site-packages/synadm/cli/room.py", line 206, in members
    .format(room_members["total"]))
KeyError: 'total'

I then tried updating my synadm, but was still unable to delete the room.

I then tried creating a completely new room and synadm won't delete it either.

It doesn't matter what I use for ROOM_ID or if I use ROOM_ID:server-name or anything - it all fails.

Neither synadm room delete -h nor the Github README give me any useful information about what ROOM_ID needs to be syntax-wise. Is there special syntax for the ROOM_ID?

I figured my test case with deleting a newly created room would give me the info I need, but it's all failing.

synadm stopped functioning on server

Before today, I have successfully ran synadm on this Matrix Synapse server. I'm not sure the last time I checked the functionality on this server as it is a single-user server. It was upgraded to the latest Matrix Synapse, which was released today/yesterday, of version 1.67.0. This may have been working perfectly fine with Matrix Synapse v 1.66.0, but again not sure when I last used it on this server.

System OS: Debian 11 Bullseye

Today, it will not function, and this is the output I receive:

$ synadm user list
WARNING Synapse returned status code 401
Traceback (most recent call last):
  File "/usr/local/bin/synadm", line 33, in <module>
    sys.exit(load_entry_point('synadm==0.36', 'console_scripts', 'synadm')())
  File "/usr/local/lib/python3.9/dist-packages/click-7.1.2-py3.9.egg/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/click-7.1.2-py3.9.egg/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.9/dist-packages/click-7.1.2-py3.9.egg/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.9/dist-packages/click-7.1.2-py3.9.egg/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.9/dist-packages/click-7.1.2-py3.9.egg/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.9/dist-packages/click-7.1.2-py3.9.egg/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/click-7.1.2-py3.9.egg/click/decorators.py", line 33, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/synadm-0.36-py3.9.egg/synadm/cli/user.py", line 80, in list_user_cmd
KeyError: 'total'

I then followed the upgrade instructions in the README.md, but afterwards I receive the same error message.

list index errors on pagination with --from <number>

when paginating through results and --from number is too high index error should be catched:

jojo@jum ~/git/synadm (media) $ synadm -o human media list -u @testuser1:peek-a-boo.at -f 1019
User has uploaded 1019 media blobs.
Traceback (most recent call last):
  File "/usr/local/bin/synadm", line 11, in <module>
    load_entry_point('synadm', 'console_scripts', 'synadm')()
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/jojo/git/synadm/synadm/cli/media.py", line 83, in media_list_cmd
    from_=from_, limit=limit, sort=sort, reverse=reverse)
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/home/jojo/git/synadm/synadm/cli/user.py", line 396, in user_media_cmd
    helper.output(media["media"])
  File "/home/jojo/git/synadm/synadm/cli/__init__.py", line 172, in output
    click.echo(self.formatter(data))
  File "/home/jojo/git/synadm/synadm/cli/__init__.py", line 39, in humanize
    if isinstance(data, list) and isinstance(data[0], dict):
IndexError: list index out of range

applies to commands: media list, user list, room list and probably more....FIXME

Prepare for Read the Docs Configuration v2

Strict deadline of 2023-09-25 (September 25, 2023), after which docs will completely fail to build.

https://blog.readthedocs.com/migrate-configuration-v2/

Add possibility to pass token via environment variable

So it doesn't have to be stored on disk in plaintext but can be retrieved from eg. pass

synadm 0.38 requires Click<8.0,>=7.0, but you have click 8.1.3 which is incompatible.

Hello,
I'm getting this while trying to:

pip3 install synadm

I need Flask installed too and that requires click>=8.0

Force user logout option

There should be an option to invalidate all access tokens for particular user = force logout.

Room Id not correct

I got the room id using,

$ synadm room list -s state_events|head -3
room_id                                      name                                 canonical_alias                              joined_members    joined_local_members    version  creator                            encryption            federatable    public    join_rules    guest_access    history_visibility      state_events
-------------------------------------------  -----------------------------------  -----------------------------------------  ----------------  ----------------------  ---------  ---------------------------------  --------------------  -------------  --------  ------------  --------------  --------------------  --------------
!kbmCAHJXpbUYAMzNZc:matrix.org                                                                                                              1                       0          1                                                           True           False                                                                 277550

But when i try to get its details, it doesnt work.

$ synadm room details kbmCAHJXpbUYAMzNZc:matrix.org
WARNING Synapse returned status code 404
errcode  M_NOT_FOUND
error    Room not found

How do i get the room id?

Suppport the Room State API

E.g to request power levels of a room (Moderator, Default, Admin, ...)

https://matrix-org.github.io/synapse/latest/admin_api/rooms.html#room-state-api

delete all media files uploaded by specific user

there should be an option to delete all files uploaded by specific user to prevent spammers from flooding the servers with uploading files continuously

regtok delete is broken

I'm using 0.41.2 (git clone of the repo). Running synadm regtok delete <token> results in:

Traceback (most recent call last):
  File "/root/code/synadm/.venv/bin/synadm", line 33, in <module>
    sys.exit(load_entry_point('synadm', 'console_scripts', 'synadm')())
  File "/root/code/synadm/.venv/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/root/code/synadm/.venv/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/root/code/synadm/.venv/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/root/code/synadm/.venv/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/root/code/synadm/.venv/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/root/code/synadm/.venv/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/root/code/synadm/.venv/lib/python3.10/site-packages/click/decorators.py", line 38, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/root/code/synadm/synadm/cli/regtok.py", line 146, in regtok_delete
    response = helper.api.regtok_delete(token)
  File "/root/code/synadm/synadm/api.py", line 1261, in regtok_delete
    return self.query("delete", "v1/registration_tokens/{token}",
  File "/root/code/synadm/synadm/api.py", line 114, in query
    urlpart = urlpart.format(*args, **kwargs)
KeyError: 'token'

The rest of the commands seem to work.

Deactivating multiple users at once

Hello,

Out of convenience, I am missing a feature for deactivating multiple users at once, preferbly based on a given regular expression or any other kind of matching pattern.

For this to work, all users must be queried and for each it must be check if the MXIDs localpart matches the regular expression. I assume that the list command can not be used for this at its current implementation. The first problem is, that the API limits how many users are returned and as far as I know there is no way of telling the API ro return all users without limit in one request. Second, the current way of filtering for users during the list command is not exactly the same as a pattern matching, since the fitler option probably only checks wheter a user ID contains a given substring or not, which is not exactly what I am looking for.

After the query is finished, the deactivate command has to take the list of user IDs returned by the modified list command and deactivate each of the accounts.

Off course this feature is entirely optional and is not coverd directly by the API at all.

Improve README: Add detailed prerequisites for matrix-docker-ansible-deploy users

Catch non-existent rooms on room delete earlier

In interactive room delete synadm room delete <id> (without --batch flag): Even if a room is not found in the first place, it's still asked wheter it should be deleted. This could be catched earlier. If a synadm user would like to "try it anyway" the bach flag still can be used, since it omits the "room info queries".

~ $ synadm -v room delete '!aBXqGDWIxVYeYxVbRu:matrix.org'
INFO  Querying get on https://mysynapse.org/_synapse/admin/v1/rooms/!ABCqGDWIxVYeYxVxyz:example.org
WARNING Synapse returned status code 404
errcode: M_NOT_FOUND
error: Room not found

INFO  Querying get on https://mysynapse.org/_synapse/admin/v1/rooms/!ABCqGDWIxVYeYxVxyz:example.org/members
WARNING Synapse returned status code 404
errcode: M_NOT_FOUND
error: Room not found

Are you sure you want to delete this room? (y/N): y
INFO  Querying delete on https://mysynapse.org/_synapse/admin/v1/rooms/!ABCqGDWIxVYeYxVxyz:example.org
failed_to_kick_users: []
kicked_users: []
local_aliases: []
new_room_id: null

~ $ 

Investigate why we allow empty string as output format

Output format choices seem to make it possible to use -o '' that doesn't seem useful but let's investigate / ask gitlens/gitblame if there maybe was a reason once:

https://github.com/JOJ0/synadm/pull/92/files#r1116809427

synadm doesn't work after v0.38 with Click 7.0

Hi, there.

Thank you for creating and maintaining synadm. I have been using it for a while on Ubuntu 20.04 and after I updated today to version 0.41 it stopped working with this error:

Traceback (most recent call last):
  File "/home/ubuntu/.local/bin/synadm", line 5, in <module>
    from synadm.cli import root
  File "/home/ubuntu/.local/lib/python3.8/site-packages/synadm/cli/__init__.py", line 486, in <module>
    from synadm.cli import room, user, media, group, history, matrix, regtok, notice  # noqa: F401, E402, E501
  File "/home/ubuntu/.local/lib/python3.8/site-packages/synadm/cli/room.py", line 336, in <module>
    def block(helper, room_id, block):
  File "/usr/lib/python3/dist-packages/click/decorators.py", line 173, in decorator
    _param_memo(f, OptionClass(param_decls, **option_attrs))
  File "/usr/lib/python3/dist-packages/click/core.py", line 1601, in __init__
    raise TypeError('Got secondary option for non boolean flag.')
TypeError: Got secondary option for non boolean flag.

Then I noticed this when upgrading:

Requirement already satisfied: Click<8.0,>=7.0 in /usr/lib/python3/dist-packages (from synadm) (7.0)

And then tried to install only this package (newer version from pip), what worked:

píp install -U click

Now I have this for my user:

$ pip list --user      
Package            Version
------------------ -------
click              8.1.3  
click-option-group 0.5.5  
dnspython          2.3.0  
synadm             0.41 

I did not find click in logs, so I guess that was the version installed by default on Ubuntu 20.04. Maybe the version needs to be updated in the dependencies.

PS: while troubleshooting it seems that the builtin version works with synadm up to v0.38 and fails after that.

Regards

Delete join/leave messages from specific room

Hello

if you could add a feature to delete or purge join/leave messages from a specific room that will be helpful to big announcements rooms

Special internal room IDs breaks some commands

A user could not delete a room because the internal room ID was very special:

!test/v7 🐈️:maunium.net (alias thingy is #test/v7:maunium.net)

The / and the emoji should be URL encoded (or just everything special) so that synapse doesn't misinterpret the / and the emoji doesn't cause weird issues.
That way, even if the room ID contains special characters, it should "just work" without having to resort to a workaround (manually URL encoding the room ID, so it's %21test%2Fv7%20%F0%9F%90%88%EF%B8%8F%3Amaunium.net instead).

Solution for user ref, Handy URL encode code snippet
Given the code needed to URL encode in python, I would put this label an "easy issue" but I'm not sure what changes are needed to make sure room IDs are URL encoded (there probably is also multiple places to also handle it sooooo).
Related issues in another project: Awesome-Technologies/synapse-admin#111

synadm user membership: There should be an option to display room alias if applicable

As of now whenever I do synadm user membership @user:host.com, I got

joined_rooms:
- '!oBvIoUsLyNoTaNiD:host.com'
- '!HaHaHa:host.com'
total: 3

for yaml output. This can't be made sense of by humans. Better if there is a flag to display aliases (if applicable) instead.

Add an option to filter out appservice users when listing users

Thanks for the awesome tool. I have loads of AS users and a few "regular" users on my server, I rarely want to operate on the AS users in the same way I want to operate on the regular ones.

How to delete local media for specific user ?

How to delete local media for a specific user ?

catch error on incomplete config after "auto-reconfigure"

Catch error when user execs a regular command eg. synadm user list - config is found to be incomplete - configurator is launched autoatically - user again does not enter all requirded information.

Either error should be catched or configuator should force user to input all required information and don't accept input empty strings (eg by just hitting enter)

ERROR config entry user missing
Traceback (most recent call last):
  File "/Users/jojo/.venvs/synadm_after_pr/bin/synadm", line 11, in <module>
    load_entry_point('synadm==0.14', 'console_scripts', 'synadm')()
  File "/Users/jojo/.venvs/synadm_after_pr/lib/python3.7/site-packages/click-7.1.2-py3.7.egg/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Users/jojo/.venvs/synadm_after_pr/lib/python3.7/site-packages/click-7.1.2-py3.7.egg/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Users/jojo/.venvs/synadm_after_pr/lib/python3.7/site-packages/click-7.1.2-py3.7.egg/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/jojo/.venvs/synadm_after_pr/lib/python3.7/site-packages/click-7.1.2-py3.7.egg/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/jojo/.venvs/synadm_after_pr/lib/python3.7/site-packages/click-7.1.2-py3.7.egg/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/jojo/.venvs/synadm_after_pr/lib/python3.7/site-packages/click-7.1.2-py3.7.egg/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/jojo/.venvs/synadm_after_pr/lib/python3.7/site-packages/click-7.1.2-py3.7.egg/click/decorators.py", line 33, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/Users/jojo/.venvs/synadm_after_pr/lib/python3.7/site-packages/synadm-0.14-py3.7.egg/synadm/cli/user.py", line 47, in list_user_cmd
AttributeError: 'NoneType' object has no attribute 'user_list'

Integrate code-style checks into our CI

I'd like to introduce some code-style checks into the CI. I have learnt by contributing to github.com/beetbox/beets what a valuable feature this can be. They check using flake8 via tox, so this might not be the right approach for us: https://github.com/beetbox/beets/blob/master/.github/workflows/ci.yaml#L116-L117

I'd definitely want to use flake8 but it might be a simple solution like pip-installing during our gh-actions workflow run and then just executing flake8 from cli for all our files/the whole repo. Would require some trial and error approaches to get ir right so it makes sense and is informational to the contributor.

Furthermore, proper excludes of certain errors and warnings will most probably be required. Not every single pep8/flake8 thing might be suitable for our project.

Add a way to see synadm version

I was running synadm v 0.38 (as seen in my local setup.py) and followed the update git installation section (which has been successful in the past for me).

I get the following errors when running pip install .

$ pip install .
Defaulting to user installation because normal site-packages is not writeable
Processing /home/dietpi/synadm
  Preparing metadata (setup.py) ... done
Requirement already satisfied: Click<9.0,>=7.1 in /usr/local/lib/python3.9/dist-packages/click-7.1.2-py3.9.egg (from synadm==0.42) (7.1.2)
Requirement already satisfied: requests in /usr/local/lib/python3.9/dist-packages (from synadm==0.42) (2.26.0)
Requirement already satisfied: tabulate in /usr/local/lib/python3.9/dist-packages/tabulate-0.8.9-py3.9.egg (from synadm==0.42) (0.8.9)
Requirement already satisfied: PyYaml in /usr/local/lib/python3.9/dist-packages (from synadm==0.42) (5.4.1)
Requirement already satisfied: click-option-group>=0.5.2 in /usr/local/lib/python3.9/dist-packages/click_option_group-0.5.3-py3.9.egg (from synadm==0.42) (0.5.3)
Requirement already satisfied: dnspython in /usr/local/lib/python3.9/dist-packages/dnspython-2.3.0-py3.9.egg (from synadm==0.42) (2.3.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests->synadm==0.42) (1.26.7)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests->synadm==0.42) (2021.5.30)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/dist-packages (from requests->synadm==0.42) (3.2)
Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.9/dist-packages (from requests->synadm==0.42) (2.0.6)
Building wheels for collected packages: synadm
  Building wheel for synadm (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [5 lines of output]
      running bdist_wheel
      running build
      running build_py
      copying synadm/api.py -> build/lib/synadm
      error: could not delete 'build/lib/synadm/api.py': Permission denied
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for synadm
  Running setup.py clean for synadm
Failed to build synadm
Installing collected packages: synadm
  Attempting uninstall: synadm
    Found existing installation: synadm 0.33.1
    Uninstalling synadm-0.33.1:
      Successfully uninstalled synadm-0.33.1
  Running setup.py install for synadm ... error
  error: subprocess-exited-with-error
  
  × Running setup.py install for synadm did not run successfully.
  │ exit code: 1
  ╰─> [7 lines of output]
      running install
      /home/dietpi/.local/lib/python3.9/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_py
      copying synadm/api.py -> build/lib/synadm
      error: could not delete 'build/lib/synadm/api.py': Permission denied
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  Rolling back uninstall of synadm
  Moving to /home/dietpi/.local/bin/synadm
   from /tmp/pip-uninstall-onmzdjc2/synadm
  Moving to /home/dietpi/.local/lib/python3.9/site-packages/synadm-0.33.1.dist-info/
   from /home/dietpi/.local/lib/python3.9/site-packages/~ynadm-0.33.1.dist-info
  Moving to /home/dietpi/.local/lib/python3.9/site-packages/synadm/
   from /home/dietpi/.local/lib/python3.9/site-packages/~ynadm
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> synadm

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.
--- Logging error ---
Traceback (most recent call last):
  File "/home/dietpi/.local/lib/python3.9/site-packages/pip/_internal/utils/logging.py", line 177, in emit
    self.console.print(renderable, overflow="ignore", crop=False, style=style)
  File "/home/dietpi/.local/lib/python3.9/site-packages/pip/_vendor/rich/console.py", line 1673, in print
    extend(render(renderable, render_options))
  File "/home/dietpi/.local/lib/python3.9/site-packages/pip/_vendor/rich/console.py", line 1305, in render
    for render_output in iter_render:
  File "/home/dietpi/.local/lib/python3.9/site-packages/pip/_internal/utils/logging.py", line 134, in __rich_console__
    for line in lines:
  File "/home/dietpi/.local/lib/python3.9/site-packages/pip/_vendor/rich/segment.py", line 249, in split_lines
    for segment in segments:
  File "/home/dietpi/.local/lib/python3.9/site-packages/pip/_vendor/rich/console.py", line 1283, in render
    renderable = rich_cast(renderable)
  File "/home/dietpi/.local/lib/python3.9/site-packages/pip/_vendor/rich/protocol.py", line 36, in rich_cast
    renderable = cast_method()
  File "/home/dietpi/.local/lib/python3.9/site-packages/pip/_internal/self_outdated_check.py", line 130, in __rich__
    pip_cmd = get_best_invocation_for_this_pip()
  File "/home/dietpi/.local/lib/python3.9/site-packages/pip/_internal/utils/entrypoints.py", line 58, in get_best_invocation_for_this_pip
    if found_executable and os.path.samefile(
  File "/usr/lib/python3.9/genericpath.py", line 101, in samefile
    s2 = os.stat(f2)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/bin/pip'
Call stack:
  File "/usr/local/bin/pip", line 8, in <module>
    sys.exit(main())
  File "/home/dietpi/.local/lib/python3.9/site-packages/pip/_internal/cli/main.py", line 70, in main
    return command.main(cmd_args)
  File "/home/dietpi/.local/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 101, in main
    return self._main(args)
  File "/home/dietpi/.local/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 223, in _main
    self.handle_pip_version_check(options)
  File "/home/dietpi/.local/lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 190, in handle_pip_version_check
    pip_self_version_check(session, options)
  File "/home/dietpi/.local/lib/python3.9/site-packages/pip/_internal/self_outdated_check.py", line 236, in pip_self_version_check
    logger.warning("[present-rich] %s", upgrade_prompt)
  File "/usr/lib/python3.9/logging/__init__.py", line 1454, in warning
    self._log(WARNING, msg, args, **kwargs)
  File "/usr/lib/python3.9/logging/__init__.py", line 1585, in _log
    self.handle(record)
  File "/usr/lib/python3.9/logging/__init__.py", line 1595, in handle
    self.callHandlers(record)
  File "/usr/lib/python3.9/logging/__init__.py", line 1657, in callHandlers
    hdlr.handle(record)
  File "/usr/lib/python3.9/logging/__init__.py", line 948, in handle
    self.emit(record)
  File "/home/dietpi/.local/lib/python3.9/site-packages/pip/_internal/utils/logging.py", line 179, in emit
    self.handleError(record)
Message: '[present-rich] %s'
Arguments: (UpgradePrompt(old='22.2.2', new='23.2.1'),)

After looking at these errors, it makes me think I'm running v 0.33.1 but I thought I was running v 0.38 and I want to be running the latest v 0.42. When I run synadm -h it's not clear what version I'm actually running/using. Is there another way to check this or fix this?

Additional info
I know python & pip are functioning as I've installed Matrix Synapse using this method and have been upgrading it this way for 3 years - I get the same --- Logging error --- section when doing this, but that's a different unrelated issue.

<user id> arguments: Add option to accept only the username

For humans, having to type @rht:matrix.domain.com every time is tedious. Would be nice if users can just type rht.

Add a LICENSE file

This project currently has no license, thus packaging it somewhere isn't possible. Could you add a license file?

Support v2 of the room delete API

and make sure pre-1.48 Synapse is still using delete API v1: https://matrix-org.github.io/synapse/latest/admin_api/rooms.html#delete-room-api

Also the new v2 API changed some behaviour:

block - Optional. If set to true, this room will be added to a blocking list, preventing future attempts to join the room. Rooms can be blocked even if they're not yet known to the homeserver (only with Version 1 of the API). Defaults to false.

User modify command doesn't show difference in non-interactive output mode

When using the non-interactive/batch flag, the output of "what will be changed" and the final "spitting out" of "what has been changed" are stuck together, thus it's not obvious what has happened:

$ synadm --non-interactive user modify --user-type support testuser6 
Current user account settings:
admin: 0
appservice_id: null
avatar_url: null
consent_server_notice_sent: null
consent_version: null
creation_ts: 1605467620
deactivated: 1
displayname: testuser6
external_ids: []
is_guest: 0
name: '@testuser6:example.org'
password_hash: null
shadow_banned: false
threepids: []
user_type: null

User account settings to be modified:
user_type: support
admin: 0
appservice_id: null
avatar_url: null
consent_server_notice_sent: null
consent_version: null
creation_ts: 1605467620
deactivated: 1
displayname: testuser6
external_ids: []
is_guest: 0
name: '@testuser6:example.org'
password_hash: null
shadow_banned: false
threepids: []
user_type: support

Make timeout value configurable by user

When making queries very often I got the timeout error:

synadm room details '!room_id:ru-matrix.org'
ERROR Timeout: HTTPConnectionPool(host='localhost', port=8008): Read timed out. (read timeout=7)

Seems this is because my server is not so nimble as expected, but it want some love too from this script :)

At now timeout is hard-coded in tgose lines:

synadm/synadm.py

Line 60 in 45b1220

resp = requests.get(url, headers=self.headers, timeout=7)

synadm/synadm.py

Line 89 in 45b1220

resp = requests.post(url, headers=self.headers, timeout=7, data=post_data)

synadm/synadm.py

Line 116 in 45b1220

resp = requests.put(url, headers=self.headers, timeout=7, data=put_data)

So will be good to make timeout configurable via .config/synadm.yaml file.