Dev Tools · 2h ago
Missing Googlebot UA pattern cost months of indexing
A developer discovered that Google's URL Inspector uses a different user agent (Google-InspectionTool) than the standard Googlebot, causing their site's geo-gate middleware to block indexing. The site's Next.js app returned a noindex tag to the inspection tool, preventing 252 URLs from being indexed for months. The fix requires explicitly matching the inspection tool's UA string in middleware rules.
Meridian48 take
This is a sharp reminder that Google's own tools don't always behave like its crawler, and developers should test against all documented UA strings.
Read the full reporting
I lost months of Google indexing to a single missing UA pattern →
DEV Community
google-indexinguser-agent