How to Find All Pages on a Website
Whether you are auditing your own site or researching a competitor, knowing how to find all pages on a website is a fundamental SEO skill. Here are the six most reliable methods, from fastest to most thorough.
1. Check the XML Sitemap
The fastest way to find all pages on a website is to check its XML sitemap. Most sites have one at /sitemap.xml. Open your browser and go to example.com/sitemap.xml — you will see every URL the site wants search engines to index. If the sitemap uses a sitemap index file, it will link to child sitemaps organized by section (blog, products, pages). This is the most complete and reliable method because site owners explicitly list the pages they consider important.
2. Use a Sitemap Finder Tool
Not every site has its sitemap at /sitemap.xml. Some use /sitemap_index.xml, /wp-sitemap.xml, or reference it only in robots.txt. A sitemap finder tool checks all 20+ common locations automatically. Enter any domain in SitemapFixer and we will discover the sitemap, parse every URL, and show you the complete list grouped by section — no manual searching required.
3. Google site: Operator
Type site:example.com into Google Search to see all indexed pages. This shows you what Google has found, which may differ from the sitemap. Pages that appear in the site: results but not the sitemap are discoverable through links. Pages in the sitemap but not in site: results may have indexing issues. Comparing these two lists is a powerful SEO diagnostic technique.
4. Google Search Console
If you own the site, Google Search Console provides the most accurate view. The Pages report shows every URL Google knows about, its indexing status, and any issues. The Sitemaps section shows which URLs were submitted versus indexed. This is the authoritative source — but it only works for sites you have verified ownership of.
5. Crawl the Website
Tools like Screaming Frog, Ahrefs Site Audit, or Sitebulb crawl a website by following every link from the homepage. This discovers pages that may not be in the sitemap and identifies orphan pages (pages with no internal links). Crawling is thorough but slow for large sites and requires desktop software. For a quick alternative, SitemapFixer parses your sitemap and identifies structural issues in 60 seconds.
6. Check robots.txt
The robots.txt file at example.com/robots.txt often references the sitemap URL directly. It also shows which sections of the site are blocked from crawling. If a section is disallowed in robots.txt, those pages will not appear in Google search results regardless of whether they are in the sitemap.