Our crawler

Last updated 18 October 2026

Sudwire Newsroom helps local news sites report what official bodies announce. To do that, it reads public news feeds and pages. If you run a website and saw our crawler in your logs, this page is for you.

How to recognise it

Its user agent is SudwireNewsroom/0.1.0 (+https://sudwire.com/bot).

What it reads

• RSS and Atom feeds, and YouTube channel feeds, that publishers chose to follow.
• The public article pages those feeds link to, to read the full text. Before reading a page it checks your robots.txt, and it follows the rules for SudwireNewsroom or, if there are none, for *.

How often

No more than one request per second to the same website, and each feed is checked at most every few minutes. It sends If-None-Match and If-Modified-Since headers, so unchanged feeds cost you almost nothing.

Credit

Articles written from your pages link back to them as the source.

Contact

To ask us to stop reading your site, or to report a problem, email support@sudwire.com with your website’s address. To block it yourself, add this to robots.txt:

User-agent: SudwireNewsroom
Disallow: /