- oen.de
- oenbot
oenbot
oenbot fetches web pages when someone uses a tool on OEN.de. There is no automatic background crawling.
User agent
Mozilla/5.0 (compatible; oenbot/1.0; +https://oen.de/oenbot)
Questions or problems?
If oenbot causes problems on your server, write to us via the contact form. We respond quickly.
ContactHow does oenbot work?
- Requests only happen when a person enters a URL into one of our tools. Like a browser visit.
- Per analysis we fetch the page itself, robots.txt, sitemap, host variants and a sample of up to 32 linked pages (mostly via HEAD requests).
- For the AI check the page is additionally fetched with the user agents of well-known AI crawlers to reveal blocks.
- Results are cached for up to 10 minutes; repeated checks in that time cause no new requests.
- A per-person limit prevents anyone from abusing OEN.de for mass requests.