Why your pages are not indexed, and how to fix it
Most unindexed pages are blocked, duplicated or judged not worth indexing yet. How to read Search Console's reasons and fix each one in the right order.
By Sohail Hussain2 November 20267 min read
A page is missing from Google for one of three reasons: Google is blocked from it, Google thinks it is a duplicate of another page, or Google has seen it and decided it is not worth indexing yet. Search Console tells you which one applies to each URL. Fix blocks first, then duplicates, then the pages Google is holding back, because the first two are mechanical and the third needs better content or better links.
Blocks are the cheapest of the three to fix once found. Indexing checks are the first step of the audit in our technical SEO and performance work and the first group in our technical SEO checklist.
How do you find out why a page is not indexed?
Use two tools in Search Console, for different jobs.
URL Inspection answers the question for one URL. Paste the address into the search bar at the top. "URL is on Google" means the page is indexed and eligible to appear. "URL is not on Google" means it cannot appear, and the Page indexing section underneath gives the reason, the last crawl date, the canonical you declared and the canonical Google chose. "Test live URL" fetches the page now, so you can confirm a fix before asking Google to look again.
The Page indexing report (Pages, under Indexing) shows the pattern across the site. The "Why pages aren't indexed" table groups URLs by reason. The Source column says whether the cause is the website or Google; in general, only "Website" rows are yours to fix. The sitemap filter above the chart lets you limit the view to URLs you submitted, which is where to start, because those are the pages you said you want indexed.
Google's own help for the report says that a site under about 500 pages can often skip it and search site:yourdomain.com plus a key term instead. That is fair. On a small site, inspect your ten most important URLs one by one and you will learn more than from the totals.
Which reasons mean something is blocking Google?
These are the reasons to fix the same day. Each one means Google could not fetch or was told not to index the page.
| Reason in Search Console | What it means | Usual fix |
|---|---|---|
| Excluded by 'noindex' tag | The HTML or an X-Robots-Tag header says noindex | Remove it from pages you want indexed. Check theme settings, SEO plugins and staging leftovers |
| Blocked by robots.txt | A Disallow rule matches the URL | Change the rule; test it with a robots.txt tester |
| Blocked due to unauthorized request (401) | The page asks for a login | Remove the password, or accept the page is private |
| Blocked due to access forbidden (403) | The server refuses Googlebot | Check firewall, CDN bot rules and security plugins |
| Server error (5xx) | The server failed when Google asked | Check hosting logs and the Crawl stats report |
| Redirect error | Chain too long, a loop, or a bad URL in the chain | Point the redirect straight at the final URL |
| Not found (404) | The URL no longer exists | Fine if the page is gone; 301 it if it moved |
| Soft 404 | The page returns 200 but looks empty or like an error | Return a real 404, or add real content |
Two traps are worth knowing. A page blocked by robots.txt can still be indexed from links elsewhere, without a description. And a noindex tag only works if Google can crawl the page to read it, so blocking a page in robots.txt and adding noindex at the same time means the noindex is never seen. Robots.txt: what to block and what not to covers both, and our robots.txt tester shows which rule matches a given URL before you publish a change.
Redirect errors usually come from chains that grew over several redesigns. How to fix redirect chains walks through finding and collapsing them.
What do the duplicate and canonical reasons mean?
Four reasons describe pages Google grouped with another URL and chose not to show separately:
- Page with redirect. The URL redirects, so the target is indexed instead. Normal after any migration.
- Alternate page with proper canonical tag. The page points to a canonical, and Google agreed. Nothing to do.
- Duplicate without user-selected canonical. Google found duplicates, no canonical was declared, and it picked one itself.
- Duplicate, Google chose different canonical than user. You declared a canonical, and Google disagreed.
The first two are working as intended. The last two need a look. Inspect the URL and compare "User-declared canonical" with "Google-selected canonical". If Google picked the version you would have picked anyway, add a canonical tag that says so and move on. If Google picked the wrong page, the usual causes are internal links pointing at the other version, a sitemap listing the other version, or two pages that are close enough that Google sees one topic. Canonical tags explained and duplicate content: causes and fixes cover the fixes.
Google's canonicalization troubleshooting guide adds a useful detail: after you make two pages clearly different, Google may keep them clustered for up to two weeks before splitting them.
What does "Discovered - currently not indexed" mean?
Google knows the URL exists but has not crawled it. Search Console's help says this typically happens when Google wanted to crawl the URL but expected it to overload the site, so it rescheduled. That is why the last crawl date is empty.
On a large shop with tens of thousands of filter URLs, that explanation is literal: Google is rationing requests, and the fix is crawl budget work. Google's crawl budget guide is written for sites of roughly 10,000 pages or more that change daily, or a million or more that change weekly, and for sites with a large share of URLs stuck in this state.
On a 40-page business site, the server is rarely the limit. The page is low priority. Common causes:
- The page is linked only from the sitemap, or from one deep page.
- The site is new and has few links from other sites.
- Thousands of low-value URLs (tags, parameters, calendar pages) compete for the same crawling.
- The server is slow, so Google crawls less of everything.
Link to the page from the homepage, a service hub or a well-visited post. Remove or block the low-value URLs. Then request indexing once. Site architecture and internal linking explains how to give important pages the links they need.
What does "Crawled - currently not indexed" mean?
Google fetched the page and decided not to add it to the index for now. Search Console's help says the page may or may not be indexed in the future and that there is no need to resubmit it.
This is the hardest reason, because nothing is technically broken. Google has judged that the page adds too little. The pages that typically land here are thin tag and category archives, location pages that differ only by the city name, near-identical service pages, short news items, and blog posts that say what a hundred other pages already say.
There are three honest options for each page:
- Improve it so it says something the other pages do not: real examples, prices, photos of your work, answers to the questions customers ask you.
- Merge it into a stronger page on the same topic and 301 the old URL.
- Accept it. Not every URL needs to be in Google. A tag archive or a thank-you page being left out costs you nothing.
Requesting indexing again without changing the page does not help. If the content is the same, the decision will be the same.
How do you get Google to check again after a fix?
Match the effort to the number of pages.
- For one important URL, run a live test in URL Inspection. If it passes, click Request indexing. There is a daily limit per property, so save it for pages that matter.
- For a group of URLs with the same reason, fix them all and click "Validate fix" on that reason in the Page indexing report. Google rechecks the affected URLs and reports progress over the following days or weeks.
- For many new or changed pages, update your XML sitemap with accurate
lastmoddates and let Google pick them up. XML sitemaps: what should be in yours covers what to list.
Then wait. Google's help for the Page indexing report says new content can take a few days to be indexed, and recrawls of existing pages follow Google's own schedule. Check again after a week, not after an hour. If the same reason comes back, the fix did not address the cause; go back to URL Inspection and compare the live test with the indexed version. How to use Google Search Console covers the rest of the reports worth checking each month.
Was this guide useful?