Published is not crawled, crawled is not indexed, indexed is not ranked: the six stages people confuse

A page goes through six stages before it earns you anything: published, crawled, indexed, ranked, clicked, converted. Each depends on the one before it, and none is guaranteed by the one before it. Google’s own description of Search has three stages, crawling, indexing and serving, and it says plainly that “not all pages make it through each stage.” Most of the bad SEO advice I hear comes from treating those six words as one word. Once you can tell them apart, Google Search Console will show you exactly which stage a page is stuck at, and the fix becomes obvious.

What are the six stages?

Here they are in order, with what each one depends on.

Published. The page exists on your server and returns when somebody requests it. This depends on you and nobody else. Nothing has happened yet in Google’s world.

Crawled. Google has found the URL and downloaded it. Google says it discovers pages by following links from pages it already knows about and from sitemaps you submit, and that during crawling it runs any JavaScript it finds so it can see the page as a visitor would. Crawling depends on the page being reachable: linked from somewhere, listed in a sitemap, not blocked by robots.txt, and served by a server that answers.

Indexed. Google has analysed the page and stored it in its index. This is the stage people assume follows automatically, and it does not. Google’s wording is blunt: “Indexing isn’t guaranteed; not every page that Google processes will be indexed.” Indexing depends on the page being allowed in (no noindex directive), being the version Google chooses to keep (canonicals, duplicates), and being worth keeping.

Ranked. Somebody searched, and Google’s systems decided the page was relevant enough to appear, and where. This depends on intent, content, links, trust, and everything else the rest of the video is about. An indexed page is eligible. That is all it is.

Clicked. The searcher chose your result over the others on the page. This depends on the title, the snippet, the position, and what else Google put in front of them.

Converted. The visitor did the thing the business needed: called, booked, bought, enquired. This depends on the page and the business, and Search Console cannot see it at all.

Each stage is a gate, not a step. Passing one earns you a place in the queue for the next, and nothing more. Google puts the same idea in one sentence: it “doesn’t guarantee that it will crawl, index, or serve your page, even if your page follows the Google Search Essentials.”

Why does confusing the stages produce bad advice?

Because the fix for each stage is a different job, and a fix applied at the wrong stage does nothing.

A page that is not indexed does not need more backlinks. It needs the reason it is not indexed found and removed. A page that is indexed but gets no impressions does not have a technical problem; it has a relevance problem, and no amount of request-indexing will change that. A page that gets impressions but few clicks has a presentation problem: the title and snippet. A page that gets clicks but no customers has a page problem or a business problem, and Search Console will never tell you which.

When somebody says “my SEO is not working”, the first useful question is: at which stage? Most people cannot answer, because they have never looked. The whole reason to learn these six words is so you can.

What happened to the JorTech booking page?

I know how invisible a stage failure can be, because I have lived one.

For far too long, a booking page on JorTech, my own repair business, carried a noindex directive. The page existed. The booking system worked. Customers could use it if they reached it. The page was not the problem, and the booking system was not the problem. Being found was.

Nobody complained. Nobody emailed to say the SEO was broken. They simply did not find the page. In six-stage terms, it was published and almost certainly crawled, and it stopped dead at indexed. It never ranked because it never could. No amount of rewriting it would have helped, because the rewriting was happening two stages too late.

Google’s Page indexing report describes this exact case. Its explanation for the reason “URL marked noindex” is that when Google tried to index the page it encountered a noindex directive and therefore did not index it, followed by advice that is almost cheerful: if you do not want this page indexed, congratulations; if you do, remove the directive. That is the whole fix. It is also the kind of fix nobody looks for, because nothing looks broken.

One related distinction, because it catches people: robots.txt is not noindex. Robots.txt tells crawlers which areas they should or should not request. Noindex tells a compliant search engine not to include the page in its index. Blocking crawling is not the same as removing a page from search, and you need to know which control you are reaching for.

Where do you see each stage in Search Console?

Google describes Search Console as a free service that helps you monitor, maintain and troubleshoot your site’s presence in Google Search results. In its own list, it lets you confirm that Google can find and crawl your site, fix indexing problems and request re-indexing, and view how often your site appears in Search, which queries show it, and how often searchers click through. Those three capabilities map neatly onto the middle four stages.

Crawled and indexed: the Pages report. The Page indexing report shows how many URLs on your site have been crawled and indexed, and sorts the rest by reason. Three reasons are worth learning by name, because they tell you the exact stage:

• “Discovered - currently not indexed” means Google found the URL but has not crawled it yet. Google says it typically wanted to crawl the URL but expected that to overload the site, so it rescheduled. That page is stuck between published and crawled.
• “Crawled - currently not indexed” means Google downloaded the page and chose not to keep it. Google’s note is that it “may or may not be indexed in the future; no need to resubmit this URL for crawling.” That page is stuck between crawled and indexed, and the answer is not to resubmit it.
• “URL marked noindex” is the JorTech case. Remove the directive.

Google also says, and this matters, that “non-indexed URLs can be fine” and that you should not expect every URL on your site to be indexed. Read the reason for each one, and confirm that the pages that actually matter to the business are not on the list.

One page at a time: URL Inspection. The URL Inspection tool shows you Google’s indexed version of a specific page and lets you test whether a URL might be indexable. If you want to know the state of a single page, this is where you ask: is it indexed, what does Google report, and are there issues?

Ranked and clicked: the Performance report. Google defines its four headline numbers like this. Impressions: “how many times your site appeared in Search results.” Clicks: “the number of times a user clicked your site from Google Search results.” CTR: “the click count divided by the impression count.” Average position: “the average position of the topmost result from your site.”

Read those with the stages in mind. Impressions without clicks means the page is ranking for something and the searcher is choosing somebody else, so look at the query, the position, and whether the title says what the page delivers. Clicks are one step further, and still not customers. CTR is evidence, not the objective; a lower click-through rate on a very specific query can still produce excellent customers. And average position is a measurement across a set of impressions, varying by query, location, device and search feature. It is not a fixed number engraved somewhere. “Across this data set, this is the reported average” is the accurate reading. “We are number four” is not.

Converted: not in Search Console. The sixth stage lives in your booking system, your phone log, your inbox. Search Console stops at the click. If your reporting stops there too, you are measuring the wrong end.

What does “request indexing” actually do?

Less than people hope, and Google is honest about it.

The URL Inspection help page says you can request that an inspected URL be indexed, and then qualifies it three ways. “Submitting a request does not guarantee that the page will appear in the Google Index.” Indexing “typically takes only a day or so, but can take much longer in some cases.” And “there is a daily limit to how many index requests you can submit.” For many pages, Google’s advice is to submit a sitemap with the updated pages marked by lastmod, rather than requesting them one at a time.

The live test has its own caveat. Google says it “can’t detect all page conditions” and does not check for duplicate or alternate page status, manual actions, quality issues or content removals, so “a positive result is no guarantee that it will appear in Search results.”

So the button does one thing: it asks Google to consider the URL for crawling and indexing. It is not a request for a ranking. If a page was refused at the indexing stage for a reason, fix the reason first, because asking again does not change the answer. And if a page is indexed and not ranking, the button has nothing to offer at all, because that is a different stage.

Which stage is your page stuck at?

Work down the list, and stop at the first stage that fails.

If the page is not in the Pages report at all, it is a discovery problem: link to it and put it in the sitemap. If it is “discovered, currently not indexed”, it is a crawl problem. If it is “crawled, currently not indexed” or marked noindex, it is an indexing problem, and the reason is written next to it. If it is indexed with no impressions, it is a relevance problem, which is content and intent. If it has impressions and few clicks, it is a title and snippet problem. If it has clicks and no customers, it is a page problem, and the page needs to earn the visit it was given.

The technical layer, the part that decides whether a page can pass the first three gates, is what the Site Score checks on this site: instant, on screen, no email, about twenty seconds, built on Google Lighthouse. Run it on your site and you will know whether the door is open before you spend a penny on what is behind it. The rest of the stages are covered across the twelve types of SEO in the playbook.

This is one part of the whole of SEO. The full picture, all five places a customer can find you and the six stages inside the third one, is in the masterclass video. Its companion page is at /watch/the-whole-of-seo, and the recording is coming.