HalalVeraBot is the crawler HalalVera uses to read a business's own website for contact details, opening hours and halal statements, so a listing can show that information together with its source.
1. Identity
- User-Agent
- HalalVeraBot/1.0 (+https://halalvera.com)
- Signature scheme
- Web Bot Auth (RFC 9421 HTTP Message Signatures), Ed25519
- Signature-Agent
- https://halalvera.com
- Public key directory
- https://halalvera.com/.well-known/http-message-signatures-directory
2. Behaviour
- Respects robots.txt (RFC 9309). A disallowed path is reported to HalalVera staff — never silently skipped and crawled anyway.
- At most 1 request per second to the same site; a batch run adds 500 ms between records.
- Never impersonates a browser, never changes its User-Agent, and never uses headless browsers, proxies or challenge-solving services.
- A temporary bot-protection refusal is retried once with the same identity, and never more than once.
3. When it visits
In two situations: an unattended batch run that refreshes existing records, and a single-page read triggered by a HalalVera staff member from the console while reviewing one business.
4. How to block or allow it
Add a rule for its user agent to your site's robots.txt:
User-agent: HalalVeraBot Disallow: /
Every request is signed (Web Bot Auth), so you can verify it really is HalalVeraBot against the public key directory above, rather than trusting the User-Agent header alone.
Questions about a specific request, or want us to stop crawling your site sooner than a robots.txt change takes effect? Contact us.