The "📞 Call back"-Attack

Many probably have noticed relatively quickly that something went horrible wrong on May 06, 2023 around 08 AM (UTC), when the Python room made a call to all of it’s users and hundreds of 📞 Call back widgets popped up in seconds.

So, what happened? At first, everything started out like a normal day in Matrix. Then a few messages popped up: “Uhh”, “There’s 12k people here”, “wtf”, “?”, “Can we not?”, “Someone pressed the wrong button?”.
What happened was that five users created a specific event in the Python room that lead to hundreds of calls to emerge, from the room. In just a few minutes 85 Users left the room and the chaos widened. Though, whether you have seen the actual calls or just the messages of the perplexed users, depends on your client.
We immediately started investigating the incident. When we noticed that what ever happened continued, and servers of community members where crashing we immediately closed down all of our rooms with the message “This room is temporarily closed.” and continued with our investigation. It was also at that moment when we really became aware of the magnitude of the attack. It took roughly 45 minutes from changing the permissions until the room was visible closed down. After the load on our servers went down, we re-opened our rooms again later in the evening.

If you like to see a visual representation of what happened, check out the graph below. It shows the events in the room in different colors depending on their type during the attack.

Since the incident is related to a not yet publicly disclosed matrix security issue NVT#1548355, we cannot go into more detail about what happened and how users were instrumented to amplified the attack. At least not yet. But we will, after the bug has been fixed and an appropriate time has passed so users can update.

With a little help, we found a solution to protect our community rooms from this attack for the future. But nevertheless, you might see two things still happening as an aftermath:

• If you get a call from the Python room, please reject the call (it doesn’t happen anything bad if you try to answer the call. You just get an error message).
• If you see a widget telling you, you missed a call, please ignore it.

After re-investigating and re-creating the incident later on, we found out that in four of the five cases the possibility exists, that those users accidentally created this kind of event and that a scenario with only one attacker was more plausible. The reason for that is that the event the attacker initially created made it possible for the others to create additional events of the same kind, by accident.

Since there was no sufficient evidence that they had the desire to cause harm, there was no reason for them to be banned from the community anymore.

For the four users we banned during the attack, who have been involved by accident, we would like to formally apologize for banning them. They have been notified and we would appreciate, if they still want to come back.

🤙 The moderation Team 🤙