Applebot exhibits aggressive behavior towards websites using Let’s Encrypt

Applebot, the RSS fetcher for the Apple News service, faces problems when it visits websites that uses encryption certificates from the Let’s Encrypt certificate authority, notes Slight Future.

As a relatively new certificate authority (CA), the Mozilla-backed Let’s Encrypt CA isn’t expected to work everywhere on day one. Most issues have been resolved, however, and everyone and their grandmother can now access websites with these encryption certificates like any other website. However, the Applebot gets stuck in a loop and swarm-behavior that resembles a small-scale-denial-of-service-attack when visiting websites that deploy certificates from Let’s Encrypt.

Applebot is the web crawler for Apple, used by products including Siri and Spotlight suggestions. Bots can be thought of as simple, contextual ways of accessing services without changing the context of where you already were and what you were doing. Bots reduce and often eliminate the need for downloading and using traditional mobile apps.