On the jabber.ru MITM attack

This post is about a recent security incident on a public XMPP service, which provides jabber.ru and xmpp.ru. We have received a few questions from Snikket users about whether they should be concerned about the security of their own servers (Snikket also uses XMPP).

The good news is that Snikket was not affected by this incident - this was a targeted attack against the jabber.ru/xmpp.ru service specifically. Later in the post we’ll share more information about what we’ve done, and what we have planned, to ensure our systems are secure from such attacks.

What happened with jabber.ru?

It transpired yesterday that jabber.ru and xmpp.ru public XMPP services have likely been subjected to interception of their encrypted traffic for at least 90 days, and possibly up to 6 months. It is not clear who performed the interception, or why. Possibilities include law enforcement, or a compromise of the infrastructure of two hosting providers (Hetzner and Linode) used by the services.

This post won’t go into too many technical details, for which we would refer to the original write-up published here.

The “machine in the middle” (MITM) attack targeted the jabber.ru and xmpp.ru domains, and ended shortly after its discovery. We’re not aware of any documented attacks of this nature on the XMPP network before, nor indeed other internet services at this time.

Why is this news?

Although traffic interception is nothing new, previously it has mostly been performed by passively observing traffic as it passed through network devices on the internet. Snowden revealed how widespread this practice was back in 2013, prompting a large shift towards TLS encryption by default across the internet. TLS protects traffic from observers, and today it is used to protect everything you do online, from chatting, to checking your email, to online banking.

What makes this attack notable, is that it was an “active” attack - not just passing traffic through, but modifying it. Specifically, they were decrypting and re-encrypting traffic as it passed through a network device (the “machine in the middle”) that had been placed between the jabber.ru server and the rest of the internet.

Usually TLS prevents such an attack from succeeding, as long as you verify certificates. However in this case the attacker was able to obtain valid certificates for the targeted domains, making all connections look like they were genuine.

With the advent of ACME-based certificate authorities such as Let’s Encrypt, obtaining certificates is not at all hard for someone able to intercept and respond to traffic that is sent to your server, and in this case that’s exactly what happened.

How is this related to Snikket?

It’s not, mostly. However, one thing we have in common is our hosting provider - we use Hetzner for a number of our servers, including those that power Snikket Hosting.

In response to the news of this attack, we have audited all our servers and verified that none demonstrate the anomalous characteristics reported by the jabber.ru team, confirming our belief that this was targeted only at their services.

We will be taking additional steps to safeguard our systems from similar attacks, as a preventative measure. This includes:

• Deploying a strict CAA record, ensuring only our Let’s Encrypt account will be authorized to issue certificates for our hosted domains (we already have DNSSEC in place to help secure this). Update: The CAA record is now deployed.

• Setting up monitoring to alert on any suspicious certificates issued for our hosted domains. We are not currently aware of suitable tooling that would meet our needs (though there are some existing efforts in this area, such as certspotter). If we develop anything new, we will share it with the community.

We are also currently working hard on the next Snikket Server release, which coincidentally supports the “channel binding” security feature. Channel binding is the ultimate protection against these kinds of attacks, and works even if the attacker is able to obtain a valid TLS certificate. The protection will be enabled by default on both hosted and self-hosted instances. This feature was part of our modern authentication and account security work in Prosody, funded by NGI Assure via NLnet.

Custom domains with Snikket

If you use a custom domain with your hosted Snikket instance, or if you are entirely self-hosting Snikket, you can also add a CAA record to increase security. You need to do this with your DNS provider - we cannot do it for you. We recommend using a relatively low TTL in case you make any mistakes.

Note that although it helps improve the security of your instance, setting a CAA record is entirely optional.

Custom domains on our hosting platform

If you are using our hosting platform, your CAA record contents should look like this exactly:

128 issue "letsencrypt.org;accounturi=https://acme-v02.api.letsencrypt.org/acme/acct/75887657"

You only need to set it for the main domain of your Snikket instance. If you use your domain for other things, do not add this CAA record on your root domain! E.g. If your Snikket instance is at ‘chat.example.com’, your CAA record should also be at ‘chat.example.com’. Otherwise it will prevent you from obtaining certificates for your other services.

Self-hosted instances

If you are self-hosting Snikket, you can also set a CAA record, but you will need to use your own account URI. You can run the following command in your snikket directory to find the right URI to use:

docker-compose exec snikket_server find /snikket/letsencrypt -name regr.json -exec grep uri {} +

Remember to set the CAA only on the domain or subdomain you use for Snikket. Put the URI (the part beginning with https://) into the record, replacing URI-GOES-HERE in the example below:

128 issue "letsencrypt.org;accounturi=URI-GOES-HERE"

If you have a reverse proxy in front of your Snikket instance that obtains its own certificates independently from your Snikket setup, you should add an additional CAA record in the same format with the accounturi that your reverse proxy uses.

After putting the CAA record in place, keep a close watch on Snikket and any other services on your domain over the following weeks, to ensure they successfully renew their certificates.

We hope this post has been helpful! If you have any questions about your Snikket setup, we have a helpful community chat. If you are using our hosted platform, you can also contact us via email at support@snikket.org.