Welcome to Hidden Pages
Welcome to Hidden Pages. This post is a quick intro to who the tool is for, what it does, and how to get started.
Who is this for?
Hidden Pages is for anyone who needs to see what’s on a website beyond the obvious:
- SEO professionals — Find disallowed or noindex URLs, compare with what’s indexed, and spot crawl or indexing issues.
- Site owners and developers — See what’s blocked in robots.txt, what’s in sitemaps, and which paths might be hidden or forgotten.
- Security and compliance — Discover unlinked or hard-to-find pages that could need review.
If you’ve ever wondered “what URLs exist on this site that search engines or visitors aren’t supposed to see?”, Hidden Pages helps answer that.
What the tool does
You enter a website URL. Hidden Pages then:
- Reads robots.txt — Shows disallow rules and surfaces URLs that match those rules.
- Checks sitemaps — Follows sitemap references and lists URLs that are declared but may be disallowed or noindex.
- Looks for other “hidden” or unlinked URLs — Common paths (e.g. admin, login, backup), URLs that appear in the Wayback Machine, in JavaScript, or in HTML comments, and URLs that aren’t linked from the main crawl.
Results are split into Hidden pages (clearly disallowed or noindex) and Other pages to check (possible hidden or unlinked; worth verifying yourself). Everything is designed to run in seconds and give you a clear, actionable list.
Try it out
No signup required to start. Go to the home page, enter any website (e.g. your own or a client’s), and hit Scan site. You get a few free scans per month; you can log in to track usage and upgrade if you need more.
Thanks for being here — we hope Hidden Pages makes it easier to see what’s really on the web.