Know About: Crawled But Not Indexed vs Discovered But Not Indexed

Outline

1. Introduction
2. Crawled But Not Indexed
3. Discovered But Not Indexed
4. FAQ

Introduction

When it comes to search engine optimization (SEO), understanding the difference between “crawled but not indexed” and “discovered but not indexed” is crucial. Both terms refer to web pages that Google has encountered but not included in its search index. However, there are key differences between these two states.

Crawled But Not Indexed

A “crawled but not indexed” page is a web page that Google’s search engine crawler has visited but has not added to its index. This means that the page is not eligible to appear in search results.

How Google Discovers Web Pages

Google uses a complex algorithm to discover new web pages. This process involves:

• Following links: Google follows links from indexed pages to discover new ones.
• Sitemaps: Website owners can submit sitemaps to Google, which provide a list of important pages on the site.
• User queries: When users search for something, Google may discover new pages that are relevant to the query.

Reasons Why a Page Might Be Crawled But Not Indexed

There are several reasons why a page might be crawled but not indexed:

• Technical issues: Errors in the page’s HTML code or site structure can prevent Google from indexing it.
• Duplicate content: If a page has identical content to another page on the site or elsewhere on the web, Google may only index one of them.
• Thin content: Pages with very little unique content may not be considered valuable enough to index.
• Lack of internal and external links: If a page is not linked to from other pages on the site or from external websites, it may be difficult for Google to find.
• Noindex tag: If the page has a <noindex> tag, it tells Google not to index it.

Tips for Getting Crawled But Not Indexed Pages Indexed

• Fix technical issues: Use tools like Google Search Console to identify and fix any technical problems with your website.
• Create unique content: Ensure that each page on your website has original, high-quality content.
• Improve site structure: Make sure your website is well-organized and easy for Google to navigate.
• Build internal and external links: Link to important pages on your website from other pages, and encourage other websites to link to your site.
• Remove the noindex tag: If you no longer want to prevent a page from being indexed, remove the <noindex> tag.

Discovered But Not Indexed

A “discovered but not indexed” page is a web page that Google has encountered but has not yet crawled. This means that the page has been discovered through links or sitemaps but has not been visited by Google’s crawler.

Reasons for “Discovered But Not Indexed” Pages

There are several reasons why a page might be “discovered but not indexed”:

• Insufficient crawling budget: Google has a limited crawling budget, and it may not have enough resources to crawl all of the pages that it discovers.
• Low-quality content: If a page is not considered valuable enough, Google may prioritize other pages for crawling.
• Site architecture issues: Problems with your website’s structure, such as slow loading times or excessive use of JavaScript, can hinder crawling.

Strategies for Getting “Discovered But Not Indexed” Pages Indexed

• Improve site speed: Optimize your website’s loading speed to make it easier for Google to crawl.
• Submit a sitemap: Provide Google with a sitemap that includes your important pages.
• Create high-quality content: Produce valuable, engaging content that is relevant to your target audience.
• Promote your content: Share your content on social media and other platforms to increase visibility.

FAQ

Q1.How do I know if a page is crawled but not indexed? 

You can use Google Search Console to check the index status of your pages.

Q2. Can I force Google to index a page? 

You cannot force Google to index a page, but you can take steps to improve its chances of being indexed.

Q3. What is the difference between “crawled but not indexed” and “discovered but not indexed”? 

A “crawled but not indexed” page has been visited by Google’s crawler but not added to the index, while a “discovered but not indexed” page has been found by Google but not yet crawled.

Q4.How long does it take for a page to be indexed after it is crawled? 

The time it takes for a page to be indexed can vary, but it typically takes a few days or weeks.

Q5. Can I submit a page to Google for indexing? 

You can submit a sitemap to Google, which includes a list of your important pages.