robots.txt & Noindex
Catch an accidental deindex the moment it ships.
A one-line mistake in robots.txt or a stray noindex tag can remove pages from search for weeks. An HTTP monitor with a forbidden-keyword pattern catches both on every probe.
What it detects
- A robots.txt that disallows the whole site
- A noindex meta tag appearing on a page
How you're alerted
- The HTTP check fails as soon as the pattern appears, with the standard “HTTP Down” rule.
Notifications go to any of 8 channels — see notifications.
- Runs from
- Every probe region you assign — down only when all of them agree
- How
- HTTP monitor with a regex forbidden keyword
- Available on
- Professional · Enterprise and Custom on request
More in SEO & Content
View categorySitemap & URL Monitoring
Watch a sitemap for added and removed URLs and check that every listed URL still answers — alerting when a URL breaks.
Broken Link Monitoring
Crawl your site and find broken internal and external links, images, scripts and stylesheets — with the pages that link to them.
Content Change Monitoring
Alert when a page's visible text or detected technology stack changes — a plain HTTP fetch, not a headless browser.
Start monitoring everything in minutes
Create a free account, add your first check and get alerted the moment something breaks — no credit card required.