# Crawling policy for ironfang.com (the website; formerly ironfang.uk).
#
# Everything public on this host may be crawled by every crawler. That includes
# AI search crawlers (OAI-SearchBot, ChatGPT-User, PerplexityBot,
# Claude-SearchBot, Googlebot, Bingbot) and model-training crawlers (GPTBot,
# ClaudeBot, Google-Extended, CCBot). It is deliberate: the documentation is
# meant to be found and quoted.
#
# Nothing private is served from this host. The portal and the identity service
# are separate hosts.
#
# Article previews and the staging site are kept out of indexes by an
# X-Robots-Tag response header, not by this file.
#
# Machine-readable entry points: /llms.txt and /.well-known/api-catalog

User-agent: *
Allow: /

Sitemap: https://ironfang.com/sitemap.xml
# Articles are published from the portal without a build, so their sitemap is
# made when it is asked for rather than written out with the rest.
Sitemap: https://ironfang.com/sitemap-articles.xml