EchoboxBot

EchoboxBot is the web fetcher operated by Echobox Ltd. It retrieves feeds and page metadata from publisher websites so that those publishers can schedule and automate their own social media and newsletter posts.

If EchoboxBot is visiting your site, it is because you — or someone in your organisation — connected that site to an Echobox account, or because one of your readers shared a link to it. EchoboxBot does not crawl the web at large, and it does not discover sites on its own.

Classification#

Under Cloudflare's verified bot classifications, EchoboxBot's activity falls into two categories:

• Feed Fetching — polling RSS, Atom and sitemap feeds that publishers have explicitly registered with us.
• Social / Link Preview — reading Open Graph and Twitter Card metadata in order to compose link previews for the publisher's own social posts.

EchoboxBot is not a Training crawler, a Search crawler, or a Data Collection crawler. See What EchoboxBot does not do.

User agent#

EchoboxBot identifies itself on every request with a stable user agent:

Mozilla/5.0 (compatible; EchoboxBot/1.1; +https://bot.echobox.com)

Match on the substring EchoboxBot/1.1.

We use one user agent for all of this traffic. EchoboxBot does not present itself as a browser, and it does not present itself as another company's crawler.

Verifying that a request really is EchoboxBot#

A user agent string can be copied by anyone, so we do not ask you to trust it. Every EchoboxBot request is cryptographically signed using Web Bot Auth (HTTP Message Signatures, RFC 9421).

Signed requests carry these headers:

Our public keys are published as a JWKS at:

https://socialapi.service.echobox.com/.well-known/http-message-signatures-directory

You can verify a request by fetching that directory and checking the signature against the key identified in Signature-Input. Cloudflare and other bot management providers do this automatically.

We do not publish an IP allowlist. Our egress addresses change, and signature verification is both stronger and more stable than IP matching. If you currently allowlist Echobox by IP range, we recommend moving to signature verification or to user-agent matching.

What EchoboxBot fetches, and what triggers it#

There are two kinds of request, and they behave quite differently.

1. Feed fetches — periodic#

When a publisher registers an RSS, Atom or sitemap feed with Echobox, we poll that feed so that newly published articles appear in their Echobox account.

• Each feed is polled on its own interval, configured by the publisher who registered it. There is no global crawl schedule.
• If a feed starts failing, we automatically back off to a tenth of the configured rate. A feed that fails consistently is moved into an error state and polled far less often.
• Feed polling retrieves the feed URL only. It does not fetch the articles the feed links to.

2. Page fetches — event-driven#

EchoboxBot reads metadata from individual article pages. This is not a crawl. Pages are fetched in response to specific events:

• an article appears for the first time in a feed the publisher registered;
• a user adds an article to their Echobox account, or pastes a link into the compose box;
• a post is about to be published, and we re-read the page to make sure the preview reflects the current headline, description and image.

Results are cached against the article, so repeat requests for the same page are served from cache rather than re-fetched.

Alongside the page itself we may retrieve the assets needed to build a preview: the site's favicon, images referenced by the page's metadata, and enough of a video file to determine its dimensions.

EchoboxBot does not spider. It does not follow links to discover new pages. The only URLs it visits are the ones your feed gives it, the ones your users give it, and the assets those pages reference.

What we read from the page#

Open Graph tags (og:title, og:description, og:image, og:site_name, article:published_time, and related), Twitter Card tags (twitter:card, twitter:title, twitter:image, and related), the canonical link, and standard <title> / <meta name="description"> fallbacks.

Request volume#

We do not set a fixed crawl rate, because we are not running a crawl — the rate is a consequence of how often you publish and how you configured your own feeds. In practice a busy news site sees feed polls at the interval it chose, plus roughly one metadata fetch per article, plus a re-read shortly before each post goes out.

If EchoboxBot is generating more load than you expect, please get in touch — it usually means a feed is misconfigured, and we can fix it directly.

robots.txt#

We want to be straightforward about this, and about why.

robots.txt exists so that a site operator can tell automated crawlers which parts of their site to leave alone. EchoboxBot is not crawling your site to find content. Every request it makes originates from an explicit instruction by the site's own operator — you registered the feed, or connected the site, or asked us to publish a link. In that relationship EchoboxBot acts as your direct agent, carrying out a task you asked for on your own content, in the same way that your CMS or your scheduling tool does.

Applying robots.txt here would produce a perverse result: a publisher's own Disallow rules, written to keep search engines out of a section of their site, would silently stop their own articles from reaching their own social accounts — with no error, and no obvious cause.

How to block or limit EchoboxBot#

If you do not want EchoboxBot on your site, any of these will work, and none of them will be circumvented:

Turn off the integration. If you are an Echobox customer, removing the feed or disconnecting the site in your Echobox settings stops the traffic at source. This is the cleanest option, because it also stops your team wondering why posts have stopped appearing.

Block the user agent. Serve 403, 404 or 410 to requests matching EchoboxBot/1.1.

# Example: block EchoboxBot at the web server or WAF
User-Agent contains "EchoboxBot/1.1"  ->  block

Block by signature. If your bot management provider supports Web Bot Auth, you can block or rate-limit verified EchoboxBot traffic by its signature, which has the advantage of not catching anything merely claiming to be us.

Rate-limit rather than block. Applying a rate limit to the user agent is fine — we will back off rather than retry aggressively.

Please note what blocking will do: if you are an Echobox customer, blocking EchoboxBot will stop your articles being discovered and will stop link previews rendering correctly on your social posts. If you are blocking us because of request volume rather than because you want us gone, contact us first — that is almost always fixable.

We do not attempt to evade blocks. EchoboxBot does not rotate user agents, does not fall back to an unidentified or browser-like user agent, and does not retry through alternative infrastructure when it is refused.

What EchoboxBot does not do#

We are listed in some public bot directories as an AI scraper or a data collection crawler. Those entries are inaccurate, and we would like to correct them plainly.

We do not collect content to train or fine-tune models. Echobox does not build a training corpus, does not create embeddings stores from fetched pages, and runs no model training or fine-tuning on customer content. Our production systems are provisioned for model inference only and are technically incapable of launching a training job.

We do not gather competitive intelligence. EchoboxBot does not scrape competitors' content, pricing pages, or comparative datasets, and Echobox does not sell, license or redistribute the content it fetches.

We do not perform ad verification, brand safety scanning, or ad targeting. These activities are sometimes attributed to us in bot directories as a result of the name confusion with a similarly named bot from another company.

We do not circumvent paywalls. EchoboxBot sees exactly what any anonymous visitor sees. Where a publisher wants us to read metadata behind their paywall, that is something they configure on their own site, for their own content.

We do not follow links or index your site. EchoboxBot builds no index and performs no SEO analysis or site auditing.

We do not fetch content on behalf of third parties. EchoboxBot retrieves content only for the Echobox customer who owns the account it is acting for.

Data we retain#

• Page metadata (title, description, image URL, publication time and similar fields) is stored against the article record for as long as the publisher's account is active, so that posts can be composed and previews rendered without re-fetching the page. We keep a history of changes so publishers can see when their own metadata changed.
• Article text, where it is used, is held transiently to power product features such as suggested post copy and topic tagging. It is not accumulated into a corpus, and copies held in our analytics warehouse are deleted after seven days.
• Some product features pass article text to third-party services for per-request processing. These are inference calls, not data contributions.

Contact#

abuse@echobox.com

If EchoboxBot is behaving in a way this page does not describe, please tell us. That includes unexpected request volume, requests to parts of your site you did not expect, or traffic using our user agent that fails signature verification.

We will respond to reports of misbehaviour by our fetcher, and we will correct this page if it becomes inaccurate.