← All website monitoring tools
Dark Visitors
AI / agent crawler analytics + robots.txt generator — free list, paid agent log analytics from $49/mo.
Free tier: free AI-bot list and robots.txt generator
What it does well
- Provides a free, comprehensive, and regularly updated list of AI crawler user agents.
- Includes a simple robots.txt generator to block specific bots from the list.
- The paid service offers direct log file analysis for identifying bot traffic.
- Focuses specifically on the emerging problem of AI content scraping.
Where it falls short
- The tool is very new, founded in 2024, with a limited operational history.
- Log analysis is a paid feature, starting at a non-trivial price for small projects.
- It is not a real-time blocking or security tool, only an identification and reporting service.
- Effectiveness depends on bots honoring robots.txt, which is not always guaranteed.
Alternatives to Dark Visitors
Cloudflare AI Audit
freemiumCloudflare AI bot dashboard + bot fight mode — bundled with Cloudflare plans.
Done-for-you agent-readiness — llms.txt + AI robots.txt + MCP card + OpenAPI. From $79 one-time.
isitagentready.com
freeFree agent-readiness scanner from Cloudflare — score + AI-coding-agent fix prompts.
Free llms.txt content validator — checks spec compliance and crawlability.