Site information
NovexaPilotBot
Novexa Pilot customers can ask the platform to crawl their own website to power site-audit and SEO-intelligence features (broken links, missing metadata, technology detection, and related on-page checks). When it does, requests carry the identity below so a site owner — whether or not they use Novexa Pilot — can recognise and, if they choose, control it.
DocumentsNovexaPilotBot
Crawler identity
- User agent
- NovexaPilotBot/1.0 (+https://novexapilot.com/bot)The exact string sent on the
User-Agentheader of every crawl request. It links back to this page. - Purpose
- Site-audit and SEO-intelligence checks — broken links, missing metadata, technology detection, and related on-page checks.
- What it crawls
- Only a domain that a Novexa Pilot customer has added and verified ownership of. It does not crawl arbitrary sites.
- Crawl rate
- Governed by your
robots.txt— the crawler respects it and any crawl-rate directives it declares. - Authentication
- None. It fetches publicly reachable pages over HTTP(S), does not submit forms, does not sign in, and does not attempt to reach non-public areas.
- IP ranges
- Not published yet — see below.
What the crawler does
- Only crawls a domain that a Novexa Pilot customer has added and verified ownership of — it does not crawl arbitrary sites.
- Respects
robots.txtand any crawl-rate directives it declares. - Fetches publicly reachable pages over HTTP(S) — it does not submit forms, does not authenticate, and does not attempt to access non-public areas.
- Runs with request-level safeguards against fetching internal/private network addresses (SSRF protection), so it stays confined to the public internet even if misconfigured.
IP ranges
Not published yet
The outbound IP range(s) the crawler runs from depend on production hosting infrastructure that has not been provisioned yet (see this product’s launch checklist). This section will list the real, stable range(s) once that infrastructure exists, so a site owner can allowlist or firewall by IP rather than only by user agent.
Blocking the crawler
Add a rule for NovexaPilotBot in your robots.txt, or block the IP ranges above once published, if you do not want your site crawled:
User-agent: NovexaPilotBot Disallow: /
If you are the site owner and use Novexa Pilot yourself, you can also just remove the domain from your account.