matrix-org/matrix-spec

The Matrix protocol specification

Matrix Specification

This repository contains the Matrix Specification. The current release version is rendered at https://spec.matrix.org, while the latest available build of the main branch is at https://spec.matrix.org/unstable.

Developers looking to use Matrix should join #matrix-dev:matrix.org on Matrix for help.

Spec authors and proposal writers are welcome to join #matrix-spec:matrix.org. We welcome contributions! See CONTRIBUTING.rst for details.

Structure

The Matrix spec is compiled with Hugo (a static site generator) with the following structure:

• /assets: assets that need postprocessing using Hugo Pipes. For example, Sass files would go here.

• /content: files that will become pages in the site go here. Typically these are Markdown files with some YAML front matter indicating, among other things, what layout should be applied to this page. The organization of files under /content determines the organization of pages in the built site.

• /data: this can contain TOML, YAML, or JSON files. Files kept here are directly available to template code as data objects, so templates don't need to load them from a file and parse them. This is also where our OpenAPI definitions and schemas are.

• /layouts: this contains Hugo templates. Some templates define the overall layout of a page: for example, whether it has header, footer, sidebar, and so on.

  • /layouts/partials: these templates can be called from other templates, so they can be used to factor out template code that's used in more than one template. An obvious example here is something like a sidebar, where several different page layouts might all include the sidebar. But also, partial templates can return values: this means they can be used like functions, that can be called by multiple templates to do some common processing.
  • /layouts/shortcodes: these templates can be called directly from files in /content.

• /static: static files which don't need preprocessing. JS or CSS files could live here.

• /themes: you can use just Hugo or use it with a theme. Themes primarily provide additional templates, which are supplied in a /themes/$theme_name/layouts directory. You can use a theme but customise it by providing your own versions of any of the theme layouts in the base /layouts directory. That is, if a theme provides /themes/$theme_name/layouts/sidebar.html and you provide /layouts/sidebar.html, then your version of the template will be used.

It also has the following top-level file:

• config.toml: site-wide configuration settings. Some of these are built-in and you can add your own. Config settings defined here are available in templates. All these directories above are configurable via config.toml settings.

Additionally, the following directories may be of interest:

• /attic: Here contains historical sections of specification and legacy drafts for the specification.
• /changelogs: Various bits of changelog for the specification areas.
• /data-definitions: Bits of structured data consumable by Matrix implementations.
• /meta: Documentation relating to the spec's processes that are otherwise untracked (release instructions, etc).
• /scripts: Various scripts for generating the spec and validating its contents.
• /packages: Various packages for shipping spec files like OpenAPI bindings and data definitions.

Authoring changes to the spec

Please read CONTRIBUTING.rst before authoring a change to the spec. Note that spec authoring takes place after an MSC has been accepted, not as part of a proposal itself.

1. Install the extended version (often the OS default) of Hugo: https://gohugo.io/getting-started/installing. Note that at least Hugo v0.117.0 is required.

   Alternatively, use the Docker image at https://hub.docker.com/r/klakegg/hugo/. (The "extended edition" is required to process the SCSS.)

2. Run npm i to install the dependencies. Note that this will require NodeJS to be installed.

3. Run npm run get-proposals to seed proposal data. This is merely for populating the content of the "Spec Change Proposals" page and is not required.

4. Run hugo serve (or docker run --rm -it -v $(pwd):/src -p 1313:1313 klakegg/hugo:ext serve) to run a local webserver which builds whenever a file change is detected. If watching doesn't appear to be working for you, try adding --disableFastRender to the commandline.

5. Edit the specification 🙂

We use a highly customized Docsy theme for our generated site, which uses Bootstrap and Font Awesome. If you're looking at making design-related changes to the spec site, please coordinate with us in #matrix-docs:matrix.org before opening a PR.

Building the specification

If for some reason you're not a CI/CD system and want to render a static version of the spec for yourself, follow the above steps for authoring changes to the specification and instead of hugo serve run hugo -d "spec" - this will generate the spec to /spec. If you'd like to serve the spec off a path instead of a domain root (eg: /unstable), add --baseURL "/unstable" to the hugo -d "spec" command.

For building the OpenAPI definitions, create a python3 virtualenv and activate it. Then run pip install -r ./scripts/requirements.txt and finally python ./scripts/dump-openapi.py to generate it to ./scripts/openapi/api-docs.json. To make use of the generated file, there are a number of options:

• You can open ./scripts/openapi-preview.html in your browser, and then open the file by clicking on Local JSON File.
• You can run a local HTTP server by running ./scripts/openapi-http-server.py, and then view the documentation by opening ./scripts/openapi-preview.html in your browser.

Issue tracking

Specification issues are tracked on github at https://github.com/matrix-org/matrix-spec/issues.

See meta/github-labels.rst for information on what the labels mean.

matrix-spec's Issues

How do you choose what Identity Server to use as a client? (SPEC-22)

Is this still valid? You specify the identity server on login..?

(Imported from https://matrix.org/jira/browse/SPEC-22)

(Reported by @ara4n)

Specify sounds for notifications in a way that works between different apps (SPEC-117)

How do we give users some kind of choice over what sound messages that match a given rule should make in an open, extensible and non-prescriptive way?

(Imported from https://matrix.org/jira/browse/SPEC-117)

(Reported by @dbkr)

Default power levels should let randoms rename the room & topic (SPEC-89)

Currently if A invites B to a room and gives the room a silly name, there's nothing B can do about it, which can be very annoying especially if A is awol. We should do IRC-style semantics where room topics may be changed by unprivileged users (unless you deliberately lock the topic/name).

(Imported from https://matrix.org/jira/browse/SPEC-89)

(Reported by @ara4n)

Voice/Video Call Type Switching (SPEC-129)

(Imported from https://matrix.org/jira/browse/SPEC-129)

(Reported by @dbkr)

Should we trust remote thumbnails to avoid regenerating local ones? (SYN-201)

Currently the media repo implementation rethumbnails all content it receives.

So if A sends a video to B, B will not be able to see the message meaningfully (i.e. with a thumbnail) until the whole video has been replicated over to B's server where it can be thumbnailed.

I propose that we should trust A's thumbnails of the content and save us the pain and delay of re-thumbnailing it ourselves. This gives us a much better UX, as we can send the thumbnail in advance of the file itself having been replicated over to B (and indeed without ever needing to replicate the content to B, if B doesn't care to download it).

(Imported from https://matrix.org/jira/browse/SYN-201)

(Reported by @ara4n)

Image orientation metadata (SPEC-85)

Context:
Pictures aren't always represented the Right Way Up. It is common to have EXIF data embedded into the file which tells the app/viewer/program the correct orientation of the image, which it then uses to display the image the Right Way Up.

The problem:
How do we handle this in Matrix, which is being run on a range of devices (Android, iOS, Web, etc)?

Proposals:

#⁠1 : Matrix m.room.message metadata
Simply add the correct orientation information to the m.room.message event when sending, along with the existing w and h keys.

• Pro: All Matrix clients will be able to get at the orientation information even if they cannot read EXIF information.
• Con: How do you handle inconsistent metadata (EXIF says 90deg clockwise, JSON says 180deg)?
  • Con: At what point do you say "for X metadata (e.g. tags, colour spaces, pixel formats), look in the EXIF?" Do we support all EXIF? What about XMP? How do we avoid duplicating all this metadata in the JSON?

#⁠2 : Homeserver query parameter
Make this a homeserver problem and allow Matrix clients to add query parameters like ?right_way_up=true to both thumbnail and hires Content Repository HTTP GET requests.

• Pro: We can enforce that homeservers should be able to read EXIF data, so requesting clients can get the image the right way up (from the EXIF) without having to be able to read EXIF themselves.
• Con: Cannot specify a custom orientation if it isn't the right way up and the cilent cannot read EXIF.

#⁠3 : Homeserver query parameter with metadata overrides
This is identical to #⁠2 but with the ability for the client to specify custom orientation information on upload (e.g. when uploading the file they can add query parameters?)

• Pro: Same as #⁠2.
• Pro: Allows clients who cannot read EXIF but who know that the image is the Wrong Way Up to specify the Right Way Up.
• Con: Inconsistent metadata. Same as #⁠1 (since they cannot read the EXIF, they cannot make sure there are no conflicts).
• Con: At what point do you stop adding metadata? Same as #⁠1

#⁠4 - Do nothing.
Don't handle this at all.

• Pro: No risk of inconsistencies.
• Con: We rely on clients being able to read EXIF data.

#⁠5 - Server annotates the event.
{panel}
Uploading client specifies rotation info in the EXIF, fixing the EXIF as required.
Uploading HS precomputes thumbnails rotating the image to a display orientation the image based on the EXIF.
Uploading client sends an m.room.message event referencing the mxc:// returned by the upload.
Uploading HS modifies the event with W+H of the image in the display orientation as it is being sent.
Downloading client uses the W+H in the event content to compute the size of the thumbnail it needs. It requests that thumbnail from its HS and assumes that the HS will return the thumnail in the size it requests.
Downloading HSes can return the thumbnail size requested, but may choose to return a different size.
Downloading HSes should return the thumbnail in the display orientation.
Downloading clients can request that their HS returns the image itself in the display orientation.
{panel}

• Pro: Downloading clients don't need to understand EXIF
• Pro: Uploading clients don't need to understand EXIF unless their camera is recording broken EXIF.
• Pro: Downloading clients never need to rotate the thumbnail.
• Pro: Since the rotation is recorded in EXIF it will be preserved over mxc:// federation.
• Pro: Downloading clients can correctly estimate the dimensions of the thumbnail using the W+H in the event.
• Con: The image must be sufficiently uploaded for the server to read the EXIF metadata before the m.room.message event can be sent.
• Con: A lot of special casing for m.room.message event sending serverside.

#⁠6 - EXIF rewrite extension.
Same as option #⁠5 but the uploading server may rewrite the EXIF data with the correct rotation data if the client asks.

• Pro: If the client knows the EXIF is wrong but can't rewrite the EXIF itself then the server can rewrite the EXIF for it.

#⁠7 - Server returns W+H from the upload.
Same as option #⁠5 but the upload server returns the correct W+H to the uploading client. The uploading client uses the W+H to set the W+H in the m.room.message content.

• Pro: The homeserver doesn't need to special case the m.room.message events.
• Con: The client needs to take information it gets from the HS and send it right back again: clients may not support re-sending all info that the mxc server gives it?

A lot of this boils down to what the lowest common denominator is, and that I think is older versions of Android with buggy cameras. These cameras often don't set the EXIF correctly (!) - they either don't say or outright lie. In these cases, you can hardcode lookups from device build to the "Right Way Up". Critically though, Android has supported reading and writing EXIF data since Android 2.0 (!) so there is no reason why clients cannot set the right oritentation in EXIF.

(Imported from https://matrix.org/jira/browse/SPEC-85)

(Reported by @ara4n)

We need a way to hide given messages from specific room members (SPEC-84)

If you're trying to loadbalance messages between multiple users in the same room, you may want to make the message only visible to subsets of the room at any given time.

This is similar to ACLing off historical message visibility if you so desire.

(Imported from https://matrix.org/jira/browse/SPEC-84)

(Reported by @ara4n)

Surely we should recommend an algorithm to use for ratelimiting? (SPEC-19)

Also, I wonder if it's unnecessarily harsh on client implementors to expect them to implement rate limiting clientside. I'd rather the c-s API was trivial to use, rather than having to worry about queueing and retry schemes for as common a case as flooding the server?

(Imported from https://matrix.org/jira/browse/SPEC-19)

(Reported by @ara4n)

v2 login flow representation (SPEC-118)

Consider if a tree-only approach is going to be better than using string arrays.

Complete stages:
foo -> bar
  `--> baz

alice -> bob -> charlie

stages: [
  {
    type: "foo",
    stages: [
      { type: "bar" },
      { type: "baz" }
    ]
  },
  {
    type: "alice",
    stages: [
      {
        type: "bob",
        stages: [
          { type: "charlie" }
        ]
      }
    ]
  }
]

@​eternaleye for suggestion:

M-kegan: Well, if "next" can be a list, then the sensible thing IMO is to replace the 'stages' list with a 'stages' tree.
M-kegan: Basically, instead of info: [{type, stages}] where stages: [type], have info: [{type, stages}] where stages: info
M-kegan: And then the absence of 'stages' indicates bottoming out on the auth flow.
M-kegan: Plus, if the next key is also an info (for a subtree), then it can just be treated as recursion in clients.
M-kegan: Also, it being a tree would allow stuff like a client library pruning by the auth methods it supports before returning the info to its caller
M-kegan: I guess part of it is that I feel the different representations may be a source of implementation bugs (especially server-side)
M-kegan: Another benefit is that server-side implementations could actually represent the auth setup as a tree, and just have a serializer to generate the appropriate json

(Imported from https://matrix.org/jira/browse/SPEC-118)

(Reported by @kegsay)

How do clients avoid downloading the whole memberlist when viewing a moderated room where most of the participants are viewers? (SPEC-109)

The equivalent of 'conference' mode on IRC servers - where you can't see who else is listening in on the room; you can only see the handful of people with permission to talk.

This presumably means

• adding a power level below which participants aren't shown in the membership list
• adding a power level above which participants are shown in the membership list if desired

or, just a way to filter/unfilter the membership list based on powerlevel?

(Imported from https://matrix.org/jira/browse/SPEC-109)

(Reported by @ara4n)

We need an admin interface & API for provisioning ASes (SPEC-114)

(Imported from https://matrix.org/jira/browse/SPEC-114)

(Reported by @ara4n)

Implement ID forwarding (like email .forward files) (SPEC-156)

We want people to run their own homeservers.

If people get too attached to their @​foo:matrix.org IDs, they may never want to run their own homeserver.

So we need to specify and implement the ability for @​foo:matrix.org to point through to @​foo:foo.net so that the user can actually exist on their own HS.

(Imported from https://matrix.org/jira/browse/SPEC-156)

(Reported by @ara4n)

Ability for server admins to acquire privileges in arbitrary rooms to resolve power struggles (SPEC-159)

Server admins currently have access to:

• /whois users
• delete aliases.

However, there are use cases for at least:
a) Redacting messages (both locally and globally)
b) Acquiring powerlevel in rooms in order to sort out problems
c) Injecting messages into a room
d) Overriding state in a room (updating topics; kicking users; etc)

Possible solutions include:

1. Creating the idea of global admins. Needless to say this is totally against the decentralised philosophy of Matrix and is totally off the table.
2. Allow server admins to inject events on their own server into the client-server API. This would act as a local overlay on top of the canonical federated representation of a room, supporting local WALL use cases. Supports only use case c) from the above.
3. Let servers optionally insert admins into the permissions data for all new rooms created on them, thus allowing admins to help out in times of strife. This would need to be combined with only allowing public rooms to be advertised on the server if they were created on that server. This solves use case b, and by extension a, c and d. However, it makes the act of where you create a room slightly more significant. It's unclear whether you'd also want admins inserted into private rooms, so that users can petition admins to sort out powerstruggles in their private group chats. Perhaps this would be another config option for the HS.
4. Abuse our current lack of E2E crypto and spoof messages from the room creator (or whoever does have power) to give ops to an admin.

Right now, matrix-org/matrix-spec-proposals#4 is the most practical option for solving our current problem of cleaning up orphaned garbage rooms on matrix.org's HS.

In future, matrix-org/matrix-spec-proposals#3 seems like the best solution to make this more manageable in the longer term.
matrix-org/matrix-spec-proposals#2 is useful too, but doesn't really solve the actual problem we were considering here - it'd only help us inject a local message saying "you're in the wrong place" which is visible only to non-federated users, which is of questionable value for cleaning up security messes.

(Imported from https://matrix.org/jira/browse/SPEC-159)

(Reported by @ara4n)

AS rate limiting behaviour (SPEC-143)

I think we need to think more carefully about AS rate limiting behaviour - specifically:

• HSes should not ratelimit ASes which are trying to lazyload content into a room
• ASes should only be limited per-room, not globally per-HS
• ...

(Imported from https://matrix.org/jira/browse/SPEC-143)

(Reported by @ara4n)

Decide how to handle HSes which break the power level rules (SPEC-2)

If a HS starts issuing events which break the powerlevel rules, we need to specify and decide how to ostracise them (and then implement it on synapse)

(Imported from https://matrix.org/jira/browse/SPEC-2)

(Reported by @ara4n)

Specify direct client-to-client file transfers (SPEC-146)

(Imported from https://matrix.org/jira/browse/SPEC-146)

(Reported by @dbkr)

Spec for transferring VoIP Calls

Both sides will have to support this for it to work so it's going to suck if we get clients that don't support it: we should spec it sooner rather than later.

Requirements:

• Between different devices (same person)
• Same device moving networks
• Transferring a call switchboard style in-house
• Possibly: just redirecting a call somewhere else entirely?

Some of these would be just transferring the media (ie. ICE renegotiation). Others would involve moving signalling too.

For just moving media, it would be nice to keep this as simple as possible, possibly just telling the client to start ICE over again.

(Imported from https://matrix.org/jira/browse/SPEC-150)

(Reported by @dbkr)

Add more metadata to m.room.member events (for bots and ASes) (SPEC-113)

Lifted from AS API discussion and other linked issues:

Sometimes we want to identify bridged users (e.g. nick completion as per original issue). There is no way to reliably do this currently: you have to do user_id regex parsing.

We need to spec out where we can find this out (extra JSON keys on m.room.member, something in the profile, etc) and what this should look like (URIs?) so informed clients can make decisions for things like visual representations, auto-complete, etc.

Original issue:
bq. If a Matrix user tab-completes to bing someone on IRC, it will autocomplete as (IRC) Arathorn or @​irc_Arathorn:matrix.org. This looks horrid to people on the IRC side of the bridge; we should either support autocompleting their IRC name on Matrix (probably the best bet) or the bridge could autoconvert through to the correct local idiom on the IRC side of the bridge. Given how ugly @​irc_Arathorn:matrix.org is in a message, and that bingers are for humans rather than machines, I'd prefer that we made the clients smarter about how they autocomplete, and/or expose IRC nicknames more intelligently to Matrix.

.. TODO
  Sometimes application services need to create rooms (e.g. when lazy loading 
  from room aliases). Created rooms need to have a user that created them, so 
  federation works (as it relies on an entry existing in m.room.member). We 
  should be able to add metadata to m.room.member to state that this user is an 
  application service, a virtual user, etc.

(Imported from https://matrix.org/jira/browse/SPEC-113)

(Reported by @kegsay)

Prevent AS routing loops (SPEC-131)

Problem: Two application services A and B are on IRC channel #foo. They both echo messages when a new IRC user joins the room, and when new Matrix users join the room. Alice on IRC joins the room:

[ ] indicates the **sender**

         IRC Channel                            Matrix Room
~~--~~~~--~~~~--~~~~--~~~~--~~~~--~~~~--~~~~--~~~~--~~~~--~~~~--~~~~--~~~~--~~~~--~~~~--~~-
[IRC] Alice has joined #foo.
                                      [A] IRC-Alice joined.
                                      [B] IRC_Alice joined.
[B] IRC-Alice has joined #foo.
[A] IRC_Alice has joined #foo
                                      [A] IRC-IRC-Alice joined.
                                      [B] IRC_IRC_Alice joined.
[B] IRC-IRC-Alice has joined #foo.
[A] IRC_IRC_Alice has joined #foo.
                                      [A] IRC-IRC-IRC-Alice joined.
                                      [B] IRC_IRC_IRC_Alice joined.
...

We should advise that AS developers mark virtual users as 'bots' and ignore bot traffic to prevent this. However, we need stronger guarantees to prevent this to avoid flooding both the IRC channel and Matrix with events.

One technique would be to put a new state event which is enforced by home servers which determines which HS domains can use application services to monitor this room, e.g. the room pointed to by #irc_python:matrix.freenode.net may have a state event saying only application services registered to the HS domain matrix.freenode.net should be passed events in this room. This breaks the infinite loop on the Matrix end by preventing B from receiving the join events. This relies on home servers implementing these rules, which is deemed more probable than relying on ASes to implement these rules, given we expect there to be many more AS implementations than HS implementations.

Alternative suggestions welcome.

(Imported from https://matrix.org/jira/browse/SPEC-131)

(Reported by @kegsay)

Join passwords (SPEC-61)

AKA "join keys" on IRC. The idea that anyone can join an invite-only room if they present some shared secret.

Wanted because Matrix Internal is now invite-only, meaning the Matrix<->IRC bridge bot can no longer join, nor have its ghosts join the room.

(Imported from https://matrix.org/jira/browse/SPEC-61)

(Reported by @leonerd)

A way to mark room as special custom-protocol rooms (SPEC-111)

Arisen a result of wanting to hide the SMS account room from the UI because there's no useful way for the user to interact with it. Rooms that only contain custom events and no messages should be hidden by regular messaging clients which don't understand the protocol being spoken on them.

(Imported from https://matrix.org/jira/browse/SPEC-111)

(Reported by @dbkr)

Should we switch the namespace of msgtype? (SPEC-68)

Matthew:
bq. we should switch namespace of msgtype away from m.text etc somehow - people get too confused.

(Imported from https://matrix.org/jira/browse/SPEC-68)

(Reported by @kegsay)

Alternative identity server models (SPEC-23)

Our identity server model has some shortcomings:

• The trusted clique could be seen to act as an oligarchic point of control
• The clique replicates all state over itself; unlike DNS root servers which partition the dataset
• We don't have a way to map tokens to arbitrary 3rd party credentials unless one of our trusted clique validates said 3rd party credentials. So otherwise you're stuck using @​myapp-$uid:myapp.com style IDs, which is rather foul.

Could something like bitauth help us on this?

(Imported from https://matrix.org/jira/browse/SPEC-23)

(Reported by @ara4n)

Support for running "ad-hoc" servers (SPEC-122)

Lots of people have said they'd love to run a matrix server, but it'd cost too much to maintain a VPS.

It'd be fun to do the thought experiment to work out really what's needed to run a server clientside, e.g. on a phone. Obvious things that come to mind are:

• A dialect of the s-s API which lets a firewalled server always initiate connections (be it long-poll or websockets or TURN or whatever), whilst somehow handling new invites?
• .well-known URIs (#433) or dyndns SRV or whatever for discovering where the server currently is (if folks are discovering it)

(Imported from https://matrix.org/jira/browse/SPEC-122)

(Reported by @ara4n)

We need to re-consider splitting users into alias & underlying opaque ID (SPEC-152)

Back when we defined rooms as split between aliases and IDs, there was a discussion over also splitting users between aliases and IDs too. At the time, the argument for doing so was "for symmetry", which didn't cut it, but we're starting to see concrete reasons why this could be a good thing to do. This bug is intended to gather together reasons to do so.

• We can use alias->ID api to also handle canonicalising case as per SYN-109 rather than SPEC-62
• We get email-style aliasing for free, allowing easy consolidation of various accounts to point at One True Account, and avoid having to support multiple accounts in clients
• We could use it for horizontal scaling, HA, and decentralising user accounts in general - replicating a single user account over multiple servers
• We could use it as the basis for a portability/account migration system.

Obviously the risks include:

• Another waste-of-time holy war over how to present aliases to users in UI as they're a many:one mapping. Needless to say, I suggest we just present them like email addresses (which are after all also many:one, effectively)
• Lots of security challenges in terms of which accounts are allowed to reside on which HSes, and how you prevent phishing attacks, and how you encrypt user data to protect them sufficiently.

If this idea went ahead, I suggest we adopt a syntax like:

@​matthew:matrix.org <-- user alias
^opaque_id:matrix.org <-- user id

I suggest using ^ as the sigil, as it doesn't escape in URLs, doesn't mean anything else, and is easily visually recognisable.

Yes, this would be a major backwards-incompatible change to the event format if we start shoving ^'s everywhere - so we need to decide to do so sooner than later if we are.

I suggest we consider using public key fingerprint as the opaque ID for the user (if this makes any sense at all).

(Imported from https://matrix.org/jira/browse/SPEC-152)

(Reported by @ara4n)

The "unavailable" presence state should surely actually be "away" (SPEC-15)

We currently set a user as "unavailable" if they've been idle for 5 minutes or whatever in the webclient.

In practice, surely "unavailable" should mean "i am busy; go away". Whereas "idle" or "away" should mean "i don't seem to be using Matrix currently".

(Imported from https://matrix.org/jira/browse/SPEC-15)

(Reported by @ara4n)

Specify discovery semantics for AS bridges (SPEC-115)

How do I know how to contact lync users on the other side of a matrix<->lync bridge? Do i need to know that @​foo:lync.amdocs.com magically does the right thing, or can I discover it somehow?

(Imported from https://matrix.org/jira/browse/SPEC-115)

(Reported by @ara4n)

How to deal with large rooms. (SPEC-36)

For every user in the room we currently have a dedicated state event for them. For large rooms with many 1000s or more members this becomes less than ideal due to the overhead for both clients and servers.

The protocol itself only requires knowledge of which servers are in the room.

(Imported from https://matrix.org/jira/browse/SPEC-36)

(Reported by @erikjohnston)

User queriable /version, /ping, /time etc (SPEC-136)

We track basic whois data (SYN-48), but don't give any way to expose querying it and other metadata (local time, ping-time) by end-users.

It's debatable whether we want to turn this on by default (like XMPP and IRC) and have powerusers be smart enough to disable it for privacy reasons, or make it a server-admin only function, or make it opt-in.

Creating bug to aid debate.

(Imported from https://matrix.org/jira/browse/SPEC-136)

(Reported by @ara4n)

Supporting arbitrary small-scale 3rd party ID systems. (SPEC-149)

I'm concerned that we've accidentally designed out the scenario where Bob has a cool app with an existing account DB in it - and wants to Matrix-enable his users.

Do they all have to go and manually sign up for separate Matrix accounts on a separate HS? Does he run a HS himself? Does he autoprovision these new accounts? (If so, the idea of one-matrix-id-per-user gets alarmingly broken) Does he force the user to enter their matrix HS details in order to link their existing app account to Matrix? Or a combination of the two, where their @​username:bobsoft.com forwards through to @​matthew:matrix.org?

It's critical that we let existing communities easily build on Matrix and get this story straight. Suggestions welcome.

(Imported from https://matrix.org/jira/browse/SPEC-149)

(Reported by @ara4n)

We surely need a way to clear/ACL scrollback for a room with various different semantics (SPEC-134)

I can think of many flavours of wanting to clear scrollback:

• Temporarily on a particular device (to save memory or to simply get a fresh screen)
• To censor old history from being visible on a given device
• To prune old history on your homeserver (to save disk space) - both on a permenant basis ("never backfill this!") and temporarily ("just save some disk space")
• To redact the events forever ("let us never speak of this scrollback again!")

The only one we actually support currently is redacting events forever - but only by looping through redacting them one by one, which is horrific.

Any ideas?

(Imported from https://matrix.org/jira/browse/SPEC-134)

(Reported by @ara4n)

Should we let users specify room-id when creating room, to make bridging easier? (SPEC-155)

(Imported from https://matrix.org/jira/browse/SPEC-155)

(Reported by @ara4n)

We need a way to be able to expire data from a HS. (SPEC-141)

This could be implementation-specific, or via a standard admin API or per-room state configuration.

Eitherway, we need a way to tell HSes to:

• no longer store this room
• only keep N messages for this room
• expire more than N messages for this room
• only keep X days of messages for this room
• purge state from this room (as opposed to messages)?

(Imported from https://matrix.org/jira/browse/SPEC-141)

(Reported by @ara4n)

Push: Flag events from the events stream that must be notified to the end user (SYN-304)

Currently, every client platform must implement the push rules logic in order to manage in-app notifications.
So, there is N implementations of them which seems risky in term of bug.

Could we keep the logic in one place: the homeserver and make the homeserver notify clients with additional information in events coming down the events stream?

(Imported from https://matrix.org/jira/browse/SYN-304)

(Reported by @manuroe)

Consider if more protocol strings should be "m." prefixed (SPEC-44)

There's at least two places in the spec that contain strings that are not "m."-prefixed:

• Federation query names
• Profile keys

We might want to consider if either of these are likely to be sufficiently extensible that we should namespace them, to allow 3rd-party extensions to play nicely.

(Imported from https://matrix.org/jira/browse/SPEC-44)

(Reported by @leonerd)

Per-room Activity Times (SPEC-92)

As part of killing the bad idea that is the global "last_active_age" timer, we need a way for servers to send the activity age of each member of the room, to new clients that join/initialSync the room.

This is only needed on initialSync because thereafter, m.room.message or m.room.member(JOIN) events will bump the timer, so clients can track that number themselves with no extra message overhead.

What is needed then, is that the server store the timestamp of the most recent message (or JOIN membership event) for every member of every room. It then sends this as part of the initialSync data about that room.

(Imported from https://matrix.org/jira/browse/SPEC-92)

(Reported by @leonerd)

How to delete "state". (SPEC-8)

Motivation

Currently, when a user leaves a room they keep an entry in the current state table which is never deleted. For large, old, busy rooms this could easily lead to the current state table being huge as it would contain an event for every use that has ever been in the room.

Since these entries are not used for anything, it would be much more convenient if the act of leaving simply involved removing the corresponding membership entry from the current state.

Problems

The main problem with removing entries from current state is making it work with the state conflict resolution algorithm. The algorithm must have these properties:

• Not depend on server state.
• Not depend on order events were received in.

Currently, the algorithm works by comparing the current state with the newly proposed state. If they are on separate branches of the tree then the two separate branches are compared.

However, if we have removed the entry from the current state we have nothing to compare any new state events with, and so it is treated as if it is completely new state. Thus by property 2, a deletion can only take effect if there are no other undeleted branches of the state tree, which doesn't really work very well.

(Imported from https://matrix.org/jira/browse/SPEC-8)

(Reported by @erikjohnston)

Users should be able to hide messages from their own view (SPEC-108)

Somehow we've got this far without users being able to delete messages from their view of a conversation (n.b. this is different to redactions). We obviously need to support it - especially for 1:1 chat.

(Imported from https://matrix.org/jira/browse/SPEC-108)

(Reported by @ara4n)

We need a way to restrict the permissions specific matrix apps have to utilise your account. (SPEC-79)

Both Hugh NS and jercos on IRC have called out that if you are handing your matrix HS credentials to an increasingly large set of random matrix-enabled apps, you are trusting these apps with a lot of power. (See IRC convo with jercos below).

My suggestion is that we either need much better support for creating ephemeral IDs which somehow fwd to your official one, but only have the ability to access rooms & history that they are explicitly invited to... or we need a formal FB-permissions model which specifies ACLs for a given app when you hand over your matrix credentials.

Dec 8 21:37 (IRC jercos) wow, that what-is-the-matrix post sounds like a local security nightmare
Dec 8 21:37 send a private message to a friend? now every app on your phone knows about it. and then suddenly you get google ads for things you've PMed people about.
Dec 8 21:46 (IRC Arathorn) @​irc_Arathorn:matrix.org jercos: how would google get to see the msg to advertise it?
Dec 8 21:47 if you don't want ads, don't use spyware apps which upload your messages to google :)
Dec 8 21:47 or use end-to-end crypto
Dec 8 21:48 to contrast: "send an email to a friend? now every mail client on your phone knows about it."
Dec 8 21:48 except you don't get google ads because a) you don't use google for your server, you run it yourself or use someone who respects your privacy
Dec 8 21:49 b) you choose mail clients who don't randomly go and datamine your mail.
Dec 8 21:49 so i think it's a fairly major win for local security
Dec 8 21:49 (but then again i'm not exactly neutral :-)
Dec 8 21:50 (IRC jercos) @​irc_jercos:matrix.org Arathorn: I don't log into my email account from 20 different apps as a matter of policy
Dec 8 21:50 * (IRC jercos) shrugs
Dec 8 21:50 (IRC Arathorn) @​irc_Arathorn:matrix.org i wouldn't expect you would with matrix either
Dec 8 21:50 (IRC jercos) @​irc_jercos:matrix.org obviously an informed user would very carefully choose what apps to use
Dec 8 21:50 (IRC Arathorn) @​irc_Arathorn:matrix.org you'd just pick a few favourite apps
Dec 8 21:51 the one with the best video calling... the one with the sexy UX... the one with the bes tgroupchat
Dec 8 21:51 (IRC jercos) @​irc_jercos:matrix.org an uninformed user can e.g., "sign in with google" without giving an app access to their mailbox
Dec 8 21:51 (IRC Arathorn) @​irc_Arathorn:matrix.org so it's no different to me having mailbox, k9, native imap and roundcube or similar for my mail accounts
Dec 8 21:52 (IRC jercos) @​irc_jercos:matrix.org specifically using the example in the article, what stops the football app from reading your IMs?
Dec 8 21:52 maybe it's a well-behaved football app, but if it can read your IMs, that breaks the implied security model around apps
Dec 8 21:52 (IRC Arathorn) @​irc_Arathorn:matrix.org jercos: you have to explictly pick your server. and even uninformed users can pick not-google (just as today with email)
Dec 8 21:53 in that example, yes - your football app could read your IMs. it's a good point i guess
Dec 8 21:53 either we need to support multiple personas more easily
Dec 8 21:53 (the equivalent of throwaway mail accounts)
Dec 8 21:53 or everything ACLed off by default
Dec 8 21:54 or per-app ACLs
Dec 8 21:54 will have a think.
Dec 8 21:54 in retrospect it is a good point :)
Dec 8 21:54 (IRC jercos) @​irc_jercos:matrix.org :D plenty of room to add that security
Dec 8 21:55 and to boot, that specific example I would see being more useful as a matrix-powered service than a matrix app
Dec 8 21:55 like text message updates, subscribe and get them without installing anything
Dec 8 21:56 but the genericism that would allow it to be used as a carrier of data to an app could just as easily wind up with a dedicated account being made automatically for that app
Dec 8 22:39 (IRC Arathorn) @​irc_Arathorn:matrix.org yup. you could implement the security either through throwaway accounts or per-app ACLs or some other model
Dec 8 22:39 but agreed that we need it.

(Imported from https://matrix.org/jira/browse/SPEC-79)

(Reported by @ara4n)

Spec for transferring VoIP Calls (SPEC-150)

Both sides will have to support this for it to work so it's going to suck if we get clients that don't support it: we should spec it sooner rather than later.

Requirements:

• Between different devices (same person)
• Same device moving networks
• Transferring a call switchboard style in-house
• Possibly: just redirecting a call somewhere else entirely?

Some of these would be just transferring the media (ie. ICE renegotiation). Others would involve moving signalling too.

For just moving media, it would be nice to keep this as simple as possible, possibly just telling the client to start ICE over again.

(Imported from https://matrix.org/jira/browse/SPEC-150)

(Reported by @dbkr)

Finer granularity on M_FORBIDDEN responses (SPEC-73)

It would be nice if we got more detail than the very generic M_FORBIDDEN when, for example, attempting to send a message to a room fails because you're not a member of it. Currently the IRC<->Matrix bridge is having to perform evil hackery of regexp matches against the intended-for-humans message string, in order to work out if that was the cause:

tm604/Matrix-IRCBridge@cf1c372#diff-fa4384ff1c710e597aba99c57e19eb6R361

(Imported from https://matrix.org/jira/browse/SPEC-73)

(Reported by @leonerd)

We should mandate homeservers declare a postmaster@ equivalent so we can talk to their admins if needed (SPEC-87)

(Imported from https://matrix.org/jira/browse/SPEC-87)

(Reported by @ara4n)

Concept of blocking IP users (ban by IP). (SPEC-82)

I as a user XMPP, appreciate it for including anonymity, but XMPP has a problem with blocking.
I do not understand why they were not smart enough to ban by ip but without disclosing it.

I suggest in Matrix make so:

1. We can to ban the already known IP (obtained outside the Matrix).
2. We can ban by IP, but the IP is not issued! You can show hash ­­– IP + salt. Salt should be become attached to the server, so that it was impossible to make some rainbow tables, and built not on the basis of the domain name, but rather on server hardware.
3. Subnet ban as for me a very bad decision, but it can be done without disclosing IP (you can do a hash on every part of IP ­­– "solthash.solthash.solthash.solthash").
4. Can make settings the server and/or rooms where specify display the IP or not, but when entering in such chatroom (which shows IP) should be a warning with confirmation for enter to the chatroom (this is important!). In addition possible in the account settings, add checkbox "not to enter the room where the displayed IP".

(Imported from https://matrix.org/jira/browse/SPEC-82)

(Reported by NightFox)

We should mandate a postmaster@ style admin account for HSes so admins are contactable when needed (SYN-224)

(Imported from https://matrix.org/jira/browse/SYN-224)

(Reported by @ara4n)

Index content repository content consistently by hash (SPEC-140)

In pre-Matrix stuff we always indexed and referred to binary content by its hashcode. This magically factored out duplicate content in server-side content repos. If we did this in Matrix we'd also avoid filling up diskspace with hundreds of identical cat GIFs; it's unclear why we don't?

(Imported from https://matrix.org/jira/browse/SPEC-140)

(Reported by @ara4n)

Can servers easily "rejoin" a room? (SPEC-148)

The problem being that it needs to find a server that has that room again.

(Imported from https://matrix.org/jira/browse/SPEC-148)

(Reported by @erikjohnston)

Create a mechanism to propagate profile changes over federation (SPEC-56)

We used to abuse m.presence to send updates as EDUs when users' profile information changed. Now we don't do that any more.

This therefore means the only way you'll know of someone's profile change is if you share at least one room with them, so you can observe an updated m.room.member event. If you have no rooms in common but still care about the user (e.g. because a client implements a generic "contact list") we still need a different way to handle this.

Add a new EDU/event type of m.profile and use that to handle profile changes.

(Imported from https://matrix.org/jira/browse/SPEC-56)

(Reported by @leonerd)

We should support revealing whether the user is currently typing on mobile/web/etc (SPEC-128)

It'd be fun and useful to optionally show whether a user is currently interacting with matrix via a phone, browser or whatever. a bit like fb does. it'd make people more sympathetic to typos and response time etc

(Imported from https://matrix.org/jira/browse/SPEC-128)

(Reported by @ara4n)

Alternative room transports to full mesh (SPEC-45)

We shouldn't assume that every room will use full mesh. Ideally rooms should be have configuration along with the policy server explaining how to distribute events amongst the participating servers.

We could have a list of distribution protocols in the room config with two transports that servers MUST support:

1. The current full mesh transport.
2. A simple list of endpoints to pass events to.

Rooms SHOULD support at least one of these two transports. A server can then always join the room by falling back to the simple list of endpoints if it doesn't understand the other federation protocols in the room.

(Imported from https://matrix.org/jira/browse/SPEC-45)

(Reported by @NegativeMjark)

We need to specify a way to flag whether a user is a bot or not (to allow bots to ignore each other when mirroring) (SPEC-12)

(Imported from https://matrix.org/jira/browse/SPEC-12)

(Reported by @ara4n)