Why your pages are not indexed, and how to fix it

Most unindexed pages are blocked, duplicated or judged not worth indexing yet. How to read Search Console's reasons and fix each one in the right order.

By Sohail Hussain2 November 20267 min read

A page is missing from Google for one of three reasons: Google is blocked from it, Google thinks it is a duplicate of another page, or Google has seen it and decided it is not worth indexing yet. Search Console tells you which one applies to each URL. Fix blocks first, then duplicates, then the pages Google is holding back, because the first two are mechanical and the third needs better content or better links.

Blocks are the cheapest of the three to fix once found. Indexing checks are the first step of the audit in our technical SEO and performance work and the first group in our technical SEO checklist.

How do you find out why a page is not indexed?

Use two tools in Search Console, for different jobs.

URL Inspection answers the question for one URL. Paste the address into the search bar at the top. "URL is on Google" means the page is indexed and eligible to appear. "URL is not on Google" means it cannot appear, and the Page indexing section underneath gives the reason, the last crawl date, the canonical you declared and the canonical Google chose. "Test live URL" fetches the page now, so you can confirm a fix before asking Google to look again.

The Page indexing report (Pages, under Indexing) shows the pattern across the site. The "Why pages aren't indexed" table groups URLs by reason. The Source column says whether the cause is the website or Google; in general, only "Website" rows are yours to fix. The sitemap filter above the chart lets you limit the view to URLs you submitted, which is where to start, because those are the pages you said you want indexed.

Google's own help for the report says that a site under about 500 pages can often skip it and search site:yourdomain.com plus a key term instead. That is fair. On a small site, inspect your ten most important URLs one by one and you will learn more than from the totals.

Which reasons mean something is blocking Google?

These are the reasons to fix the same day. Each one means Google could not fetch or was told not to index the page.

Reason in Search ConsoleWhat it meansUsual fix
Excluded by 'noindex' tagThe HTML or an X-Robots-Tag header says noindexRemove it from pages you want indexed. Check theme settings, SEO plugins and staging leftovers
Blocked by robots.txtA Disallow rule matches the URLChange the rule; test it with a robots.txt tester
Blocked due to unauthorized request (401)The page asks for a loginRemove the password, or accept the page is private
Blocked due to access forbidden (403)The server refuses GooglebotCheck firewall, CDN bot rules and security plugins
Server error (5xx)The server failed when Google askedCheck hosting logs and the Crawl stats report
Redirect errorChain too long, a loop, or a bad URL in the chainPoint the redirect straight at the final URL
Not found (404)The URL no longer existsFine if the page is gone; 301 it if it moved
Soft 404The page returns 200 but looks empty or like an errorReturn a real 404, or add real content

Two traps are worth knowing. A page blocked by robots.txt can still be indexed from links elsewhere, without a description. And a noindex tag only works if Google can crawl the page to read it, so blocking a page in robots.txt and adding noindex at the same time means the noindex is never seen. Robots.txt: what to block and what not to covers both, and our robots.txt tester shows which rule matches a given URL before you publish a change.

Redirect errors usually come from chains that grew over several redesigns. How to fix redirect chains walks through finding and collapsing them.

What do the duplicate and canonical reasons mean?

Four reasons describe pages Google grouped with another URL and chose not to show separately:

  • Page with redirect. The URL redirects, so the target is indexed instead. Normal after any migration.
  • Alternate page with proper canonical tag. The page points to a canonical, and Google agreed. Nothing to do.
  • Duplicate without user-selected canonical. Google found duplicates, no canonical was declared, and it picked one itself.
  • Duplicate, Google chose different canonical than user. You declared a canonical, and Google disagreed.

The first two are working as intended. The last two need a look. Inspect the URL and compare "User-declared canonical" with "Google-selected canonical". If Google picked the version you would have picked anyway, add a canonical tag that says so and move on. If Google picked the wrong page, the usual causes are internal links pointing at the other version, a sitemap listing the other version, or two pages that are close enough that Google sees one topic. Canonical tags explained and duplicate content: causes and fixes cover the fixes.

Google's canonicalization troubleshooting guide adds a useful detail: after you make two pages clearly different, Google may keep them clustered for up to two weeks before splitting them.

What does "Discovered - currently not indexed" mean?

Google knows the URL exists but has not crawled it. Search Console's help says this typically happens when Google wanted to crawl the URL but expected it to overload the site, so it rescheduled. That is why the last crawl date is empty.

On a large shop with tens of thousands of filter URLs, that explanation is literal: Google is rationing requests, and the fix is crawl budget work. Google's crawl budget guide is written for sites of roughly 10,000 pages or more that change daily, or a million or more that change weekly, and for sites with a large share of URLs stuck in this state.

On a 40-page business site, the server is rarely the limit. The page is low priority. Common causes:

  • The page is linked only from the sitemap, or from one deep page.
  • The site is new and has few links from other sites.
  • Thousands of low-value URLs (tags, parameters, calendar pages) compete for the same crawling.
  • The server is slow, so Google crawls less of everything.

Link to the page from the homepage, a service hub or a well-visited post. Remove or block the low-value URLs. Then request indexing once. Site architecture and internal linking explains how to give important pages the links they need.

What does "Crawled - currently not indexed" mean?

Google fetched the page and decided not to add it to the index for now. Search Console's help says the page may or may not be indexed in the future and that there is no need to resubmit it.

This is the hardest reason, because nothing is technically broken. Google has judged that the page adds too little. The pages that typically land here are thin tag and category archives, location pages that differ only by the city name, near-identical service pages, short news items, and blog posts that say what a hundred other pages already say.

There are three honest options for each page:

  1. Improve it so it says something the other pages do not: real examples, prices, photos of your work, answers to the questions customers ask you.
  2. Merge it into a stronger page on the same topic and 301 the old URL.
  3. Accept it. Not every URL needs to be in Google. A tag archive or a thank-you page being left out costs you nothing.

Requesting indexing again without changing the page does not help. If the content is the same, the decision will be the same.

How do you get Google to check again after a fix?

Match the effort to the number of pages.

  • For one important URL, run a live test in URL Inspection. If it passes, click Request indexing. There is a daily limit per property, so save it for pages that matter.
  • For a group of URLs with the same reason, fix them all and click "Validate fix" on that reason in the Page indexing report. Google rechecks the affected URLs and reports progress over the following days or weeks.
  • For many new or changed pages, update your XML sitemap with accurate lastmod dates and let Google pick them up. XML sitemaps: what should be in yours covers what to list.

Then wait. Google's help for the Page indexing report says new content can take a few days to be indexed, and recrawls of existing pages follow Google's own schedule. Check again after a week, not after an hour. If the same reason comes back, the fix did not address the cause; go back to URL Inspection and compare the live test with the indexed version. How to use Google Search Console covers the rest of the reports worth checking each month.

Was this guide useful?

Questions

Frequently asked.

How long does Google take to index a new page?

Anything from a few hours to several weeks, and Google does not guarantee it. A page linked from your homepage or a busy section, listed in the sitemap and fast to load is usually picked up within days. A page on a new site with no links can wait much longer.

Should I keep clicking Request indexing until it works?

No. There is a daily quota per property, and a request only puts the URL in a queue. Clicking it again does not move the page up. Fix the reason the page was skipped, request once, and give it at least a week before looking again.

Is it bad if Search Console shows more non-indexed pages than indexed ones?

Not by itself. Redirects, canonicalised duplicates, filtered URLs and old 404s all count as not indexed, and Google says you should not expect 100% coverage. It is a problem only when pages you want in search appear under a reason you can fix.

Sohail Hussain

Founder, MSH Technologies

Sohail has designed and built websites, web apps and AI products for over 12 years, for clients from Singapore-based industrial groups to early-stage SaaS startups. He writes about what actually worked.

Keep reading

Related guides.

All posts

Your next move

Like what you see? Tell us about it.

A process that takes too long, a website that undersells you, or an idea ready to take shape. Tell us where you are and we will suggest a useful next step.

India · Working with clients worldwide

We reply within a working day. No spam, ever.