HallAtlasBot

The crawler behind HallAtlas.

What it does

HallAtlasBot builds a weekly census of the permitting and agenda software each US county, city and township runs. It reads governments' own public pages and public DNS. It also reads two open datasets: the Census Bureau's list of US local governments and CISA's list of .gov domains.

User agent

Every request carries exactly this user agent:

HallAtlasBot/0.1 (+https://hallatlas.com/bot; bot@hallatlas.com)

How to opt out

HallAtlasBot follows robots.txt and answers to the token HallAtlasBot. To stop it crawling your site, add:

User-agent: HallAtlasBot
Disallow: /

Or email an opt-out request to bot@hallatlas.com. Opt-outs are honoured within 7 days.

Where requests come from

All requests come from one address: 20.98.74.217 (IPv4 only).

How fast

How it behaves

What it keeps

Non-personal link records, plus what it needs to follow these rules: DNS answers, robots.txt rules, its terms decisions and a log of its requests (kept 90 days). No page text, apart from short, name-scrubbed link text and short, name-scrubbed excerpts of a site's terms page.