Force Google to Crawl Your Site: What Actually Works
You cannot literally force Googlebot to do anything — but you can send a stack of strong signals that almost always trigger a fresh crawl within hours to a few days. The catch: most of the "tricks" people search for (paid indexing services, mass URL submission, ping spam) either do nothing or actively hurt you. This guide is the actual playbook: which signals Google responds to, how to send them properly, the order to try them in, and when "force crawl" is the wrong question entirely because the page should not be indexed in the first place.
What "Forcing a Crawl" Actually Means
Google's crawl scheduler decides when to revisit a URL based on a few signals: PageRank-style importance, observed change frequency, sitemap lastmod dates, internal link discovery, and explicit user requests through Search Console. When people say "force Google to crawl," what they really want is to push a URL up that scheduler's queue from "maybe in a few weeks" to "within hours."
You have four levers with real effect: GSC URL Inspection "Request Indexing," an updated sitemap with a fresh lastmod, IndexNow (which Google does not officially consume but Bing and Yandex do, and there is reporting evidence Google reads the IndexNow signals via shared infrastructure with Bing), and inbound link signals from already-crawled pages. Everything else is noise. Submitting a URL 50 times to a third-party indexer does not stack — if anything, repeated submission of low-quality URLs trains Google's crawler to deprioritise your domain.
Step 1: Use GSC URL Inspection + Request Indexing
The most direct way to ask Google for a fresh crawl of a single URL. The flow:
Open Google Search Console, paste the full URL into the inspection bar at the top, and wait for the live data to load. If the page shows "URL is not on Google," click "Test Live URL" first. The live test fetches the page in real time as Googlebot, runs the rendering pipeline, and reports back any blocks (robots.txt, noindex, redirect loops, JavaScript errors, 5xx). Only when the live test passes — meaning Googlebot was able to fetch and render the URL successfully — should you click "Request Indexing."
Request Indexing is rate-limited per property, typically 10–12 successful requests per day. Do not waste them on low-priority pages. Use it for: brand-new pages that need to enter the index quickly, pages you just fixed (removed a noindex, fixed a 5xx, corrected a canonical), and high-priority commercial pages. Do not use it for: blog posts you just published if you have a healthy sitemap and crawl rate — the sitemap will pick them up within a day or two anyway.
Realistic outcome: a successful Request Indexing call typically results in a Googlebot crawl within 1–24 hours. Indexing (the page appearing in search) follows the crawl, but is not guaranteed — Google still applies quality filters. A crawl is not an index.
Step 2: Submit and Resubmit Your Sitemap
For bulk URL changes (a new section launch, post-migration, dozens of new pages), Request Indexing does not scale. Sitemaps do. The sitemap is Google's primary mechanism for batch URL discovery, and the <lastmod> tag is what tells Google "this URL changed since you last looked."
Three things must be true for a sitemap to actually accelerate crawl: every URL returns 200 (not 3xx, 4xx, or 5xx), every URL is canonical (no parameter-stripped duplicates), and every <lastmod> reflects the real last content change date in W3C datetime format. A sitemap full of stale lastmod values trains Google to ignore your sitemap timestamps entirely.
Submit the sitemap in GSC: Sitemaps → paste the path → Submit. Google will fetch it within minutes. If you have just updated existing URLs, also fire the legacy ping endpoint — Google officially deprecated it in June 2023, but it still returns 200 OK and there is community evidence it still influences crawl scheduling for some properties:
# Google sitemap ping (officially deprecated June 2023, still works) curl -s "https://www.google.com/ping?sitemap=https://example.com/sitemap.xml" # Bing sitemap ping (still officially supported) curl -s "https://www.bing.com/ping?sitemap=https://example.com/sitemap.xml" # Verify the sitemap itself returns 200 with correct content-type curl -I https://example.com/sitemap.xml # Expect: HTTP/2 200 + Content-Type: application/xml or text/xml # Confirm lastmod values are recent and in W3C format curl -s https://example.com/sitemap.xml | grep -oE '<lastmod>[^<]+' | head -5 # Expect: <lastmod>2026-04-28T10:30:00+00:00 (not <lastmod>2024-...)
Keep your sitemap reference in robots.txt as well. This is how Bing, Yandex, DuckDuckGo, and the long tail of crawlers discover it without manual submission, and it is still Google's fallback when GSC submission has not happened yet:
# /robots.txt User-agent: * Allow: / Sitemap: https://example.com/sitemap.xml Sitemap: https://example.com/sitemap-news.xml Sitemap: https://example.com/sitemap-products.xml
Step 3: Use IndexNow for Bing, Yandex, and Indirectly Google
IndexNow is an open protocol jointly run by Microsoft Bing and Yandex. You publish a key file at the root of your domain, then send a JSON POST listing changed URLs. Bing and Yandex begin crawling typically within minutes. Google has not officially adopted IndexNow, but Cloudflare, Wix, and other partners forward IndexNow signals to multiple endpoints, and there is operational evidence the signals influence Google's crawl scheduler indirectly via shared infrastructure.
To set up: generate a 32-character hex key (any UUID without dashes works), host it at https://example.com/<key>.txt with the key as the file's only contents, then post URL batches:
# Submit up to 10,000 URLs in a single IndexNow request
curl -X POST "https://api.indexnow.org/indexnow" \
-H "Content-Type: application/json" \
-d '{
"host": "example.com",
"key": "a1b2c3d4e5f6789012345678901234ab",
"keyLocation": "https://example.com/a1b2c3d4e5f6789012345678901234ab.txt",
"urlList": [
"https://example.com/new-product-page",
"https://example.com/updated-blog-post",
"https://example.com/category/refreshed-listing"
]
}'
# Expect HTTP 200 (accepted) or 202 (accepted, processing)
# 422 = key validation failed; 429 = rate-limited (slow down)Wire IndexNow into your CMS publish hook so every new or edited URL fires automatically. WordPress users have plugins (Rank Math, Yoast Premium, IndexNow) that handle this. Next.js, Nuxt, and custom stacks should add a server-side hook on content publish that POSTs to the endpoint.
Step 4: Internal Linking From High-Authority Pages
This is the most underrated lever and the only one that compounds over time. Googlebot rediscovers URLs primarily by following links from pages it already crawls frequently. If your homepage is crawled hourly and you add a link from the homepage to your new URL, that new URL gets discovered on the next homepage crawl — usually within hours, no GSC submission required.
Identify your highest-crawled pages by opening GSC → Settings → Crawl stats → By response (200) → Examples. The URLs at the top are crawled multiple times per day. These are your discovery accelerators. From a URL you are trying to get crawled, work backwards: is there a contextual link from any of those high-frequency pages? If not, add one.
Concrete tactics: add new posts to a "Latest" module on the homepage, link new product pages from category pages (which are typically crawled more often than products themselves), and cross-link new content from semantically related, already-indexed posts. Avoid link footers stuffed with every new URL — Google devalues sitewide link blocks, and they look like manipulation.
Fix the Crawl Blockers Before Anything Else
None of the techniques above work if Googlebot is structurally blocked from the URL. Before you fire pings or click Request Indexing, run a five-minute crawl-blocker check:
# 1. Confirm the URL returns 200 (not 3xx/4xx/5xx)
curl -I -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" \
https://example.com/your-url
# 2. Check robots.txt does not Disallow this path
curl -s https://example.com/robots.txt | grep -E "Disallow|Allow"
# 3. Confirm no noindex meta tag in HTML
curl -s -A "Googlebot" https://example.com/your-url | \
grep -iE 'name="robots"|name="googlebot"'
# Anything containing "noindex" = page is intentionally excluded
# 4. Confirm no X-Robots-Tag header
curl -I -A "Googlebot" https://example.com/your-url | grep -i x-robots-tag
# 5. Measure TTFB (Googlebot throttles slow servers)
curl -o /dev/null -s -w "TTFB: %{time_starttransfer}s\n" https://example.com/your-url
# Target < 0.6s; > 1.5s and crawl rate will be cappedThe most common "Google will not crawl my site" root causes I see in audits: a forgotten noindex from staging that shipped to production; a Disallow: / in robots.txt left over from a launch freeze; a misconfigured Cloudflare WAF rule blocking Googlebot's IPs (Google publishes its IP ranges so you can verify); intermittent 5xx during deploys; and TTFB above 2 seconds, which causes Google to reduce crawl frequency to protect your origin.
Verify Googlebot Can Actually Render the Page
For JavaScript-heavy sites, "the page returns 200" is not enough. Googlebot fetches the raw HTML, queues the URL for rendering with a Chrome instance, then runs your JavaScript — sometimes minutes, sometimes days later. If the content is only present after JS execution, the first-wave crawl sees an empty shell and may not requeue rendering for a while.
Use GSC's Live URL Inspection → View Tested Page → Screenshot to see what Googlebot actually rendered. If the screenshot is blank or shows a loading spinner, your content is not visible to the crawler at fetch time. The fix is server-side rendering or static generation. In Next.js, that means using the App Router with default server components, or Pages Router with getStaticProps/getServerSideProps. For SPAs without SSR, the only real option is a prerender service (Prerender.io, Rendertron) at the edge.
Also confirm structured data is server-rendered. JSON-LD injected via client JavaScript is processed by Google but with delay. Embedding the JSON-LD inline in the server response (the way every page on this site does it) ensures it is in the first-wave crawl:
<!-- Server-rendered JSON-LD: visible in first-wave crawl -->
<script type="application/ld+json">
{
"@context": "https://schema.org",
"@type": "Article",
"headline": "Force Google to Crawl Your Site",
"datePublished": "2026-04-29",
"author": { "@type": "Organization", "name": "SitemapFixer" }
}
</script>Realistic Timelines: Hours, Days, or Weeks
Set expectations correctly — otherwise you will keep firing pings thinking the first round did not work. Typical recrawl latencies for a healthy domain (DR 30+, regular content updates, clean technical baseline):
1–6 hours: homepage and top-level navigation pages, after any sitemap update or strong inbound link signal. 6–48 hours: URLs that have a successful Request Indexing call in GSC. 2–7 days: new URLs added to the sitemap with valid lastmod, with a homepage or category-page link. 1–3 weeks: deep URLs with no internal link signal and no GSC submission — these may sit in "Discovered, currently not indexed" until Google decides they are worth the crawl budget.
For new domains (under 6 months old, low DR, no backlinks), multiply all the above by 3–5. New domains are crawl-budget-poor by default. The fix is not more pings — it is acquiring even one or two real backlinks from already-indexed sites. A single inbound link from a frequently-crawled domain typically reduces time-to-first-crawl from weeks to hours.
What NOT to Do
Do not buy "indexing services." The cheap ones do nothing — they spin up backlink farms or hit deprecated ping endpoints in a loop. The expensive ones use Request Indexing API access (which is fine but limited to 200 URLs/day on the Indexing API, and only officially permitted for JobPosting and BroadcastEvent schema) or scrape GSC accounts at volume, which violates Google's ToS and risks deindexing the domain entirely.
Do not mass-submit thousands of URLs to IndexNow daily. IndexNow accepts up to 10,000 URLs per request, but submitting that volume daily on a small site signals a low-quality content farm and can get your domain throttled. Submit only URLs that genuinely changed.
Do not click Request Indexing on the same URL repeatedly. Google explicitly states that requesting indexing multiple times for the same URL does not speed anything up. Once per genuine content change is the correct cadence.
Do not use the Indexing API for non-job-posting pages. Google's Indexing API officially supports only JobPosting and BroadcastEvent schema URLs. Some SEO tools push it for general content; this works briefly, then Google reconciles and the URLs are dropped — sometimes with a manual action against the property.
Do not generate fake activity. Bot-driven traffic, fabricated social shares, or rapid content republication to bump lastmod dates do not fool the crawler and degrade Google's trust in your sitemap signals.
When "Force Crawl" Is Not the Right Question
If a page has been crawled multiple times already and shows up in GSC as "Crawled, currently not indexed" or "Discovered, currently not indexed" — forcing another crawl will not help. Google has already seen the page and chosen not to index it. Triggering more crawls just confirms the same decision faster.
The real question is why Google decided the page is not worth indexing. The usual answers: thin content (under 300 words of unique text), near-duplicate of another page on your site, near-duplicate of a higher-authority page on a competitor site, low domain quality signals (lots of other low-quality pages dragging the property down), or no demonstrated user demand for this specific topic.
The fix in those cases is content quality, consolidation, or pruning — not crawl manipulation. Merge thin pages into stronger pillar pages, redirect duplicates, and noindex pages you do not actually need indexed (tag archives, paginated category page 50+, parameter variants). A smaller site with 100 strong pages outperforms the same site with 100 strong pages plus 5,000 thin ones, because Google's site-wide quality assessment improves when the average page quality goes up.
The Order to Try Things In
For a single high-priority URL: run the crawl-blocker check, fix any blockers, click Request Indexing in GSC. Done in 10 minutes. Wait 24 hours before doing anything else.
For a batch of new URLs (e.g., 50 new product pages): update the sitemap with correct lastmod, resubmit it in GSC, ping the legacy endpoint, fire IndexNow, and add internal links from your homepage or relevant category pages. Wait a week before considering further action.
For an entire site that Google seems to have stopped crawling: this is a different problem — usually a manual action, a major TTFB regression, or accidental sitewide noindex/Disallow. Check Manual Actions in GSC, check Crawl Stats for a sudden drop, and re-run the crawl-blocker check at the property level. Forcing a crawl will not fix a structural issue.