{"id":15767,"date":"2020-06-04T15:05:08","date_gmt":"2020-06-04T19:05:08","guid":{"rendered":"https:\/\/www.knownhost.com\/blog\/?p=15767"},"modified":"2020-10-16T07:43:13","modified_gmt":"2020-10-16T11:43:13","slug":"crawling-vs-indexing-how-to-fix","status":"publish","type":"post","link":"https:\/\/www.knownhost.com\/blog\/crawling-vs-indexing-how-to-fix\/","title":{"rendered":"Crawling vs. Indexing + How To Fix"},"content":{"rendered":"\n<h3 class=\"wp-block-heading\"><strong>Updated June 4, 2020<\/strong><\/h3>\n\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<p>Articles about SEO and how to improve site performance will frequently bounce terms about including crawling, indexing, robots.txt, meta robots, canonical and more.<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<p>It&#8217;s absolutely essential to understand the difference between crawling and indexing, plus why it matters to your site performance in search rankings.<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<p>Matt C. at Google explains the basics: <a href=\"https:\/\/www.google.com\/search\/howsearchworks\/crawling-indexing\/\">Here!<\/a><\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Crawling<\/strong><\/h3>\n\n\n<p>Search engines visit a site, often based on links from other sites, check the robots.txt file to see where they&#8217;re allowed to go on the site and begin exploring the various pages by following first one internal link and then another.  They&#8217;ll also use the sitemap provided to give them some indications as to what all they can find.<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<p>The robots.txt file is used to let search engines know which pages of the site they shouldn&#8217;t crawl and prevent images, video and other media files, as well as any script and CSS style files which you don&#8217;t want included in the index.<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<p>It does NOT prevent Google from indexing any particular pages.<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<p>Read that again\u2026.  robots.txt does not prevent Google from indexing any particular page.<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<p><a href=\"https:\/\/support.google.com\/webmasters\/answer\/6062608?hl=en\">Here&#8217;s the Google official explanation<\/a> about robots.txt.<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Indexing<\/strong><\/h3>\n\n\n<p>In the head of each page are a number of HTML tags such as the language used on the page, character set used, which CSS stylesheets to load, the page title and meta description.  It can also include a meta robots tag like:<br>&lt;meta name=&#8221;robots&#8221; content=&#8221;index, follow&#8221;&gt;<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<p>A page with the above tag will be a candidate for indexing and the links on the page will be checked by search engine crawlers.<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<p>The meta robots tag is covered by Google: <a href=\"https:\/\/developers.google.com\/search\/reference\/robots_meta_tag\">Here!<\/a><\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Preventing Indexing<\/strong><\/h3>\n\n\n<p>Keeping a page out of the search index means using a modified version of the above:<br>&lt;meta name=&#8221;robots&#8221; content=&#8221;noindex, follow&#8221;&gt;<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<p>In this case, search engines are instructed not to index the page content, but should follow links on the page to discover other pages of importance.<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<p>Blocking Crawling Means Indexing Directives Won&#8217;t Be Followed!<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<p>A search engine must be allowed to crawl a page in order to discover the meta robots instructions about indexing.<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<p>If a robots.txt file instructs search engines to not crawl a particular page that you don&#8217;t want in the index, then it won&#8217;t ever see the meta robots tag and won&#8217;t know to avoid indexing.<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<p>If a page gets in the index (such as when linked to externally), changing the meta robots to noindex won&#8217;t work, if search crawlers are blocked due to robots.txt.<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<p>The only way to get a page out of the index is to allow crawling (no blocks in robots.txt) and then adding the meta robots noindex tag.<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<p>Once you&#8217;ve unblocked Google from crawling, by editing any robots.txt entry, and added the meta robots noindex tag, you can request URL removal:<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<p><a href=\"https:\/\/support.google.com\/webmasters\/answer\/9689846?hl=en\">Google Search Console URL Removal tool<\/a><\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>How to Edit the robots.txt File<\/strong><\/h3>\n\n\n<p>Many site owners are using an SEO plugin of one type or another.  If this is the case, there&#8217;s a high likelihood that you&#8217;ll find a robots.txt editor within the SEO plugin options.<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<p>Otherwise, if you&#8217;re using cPanel:<br>\n1.\tlogin to cPanel<br>\n2.\topen File Manager<br>\n3.\tgo to the root directory of your site (\/public_html\/ is often the case)<br>\n4.\tuse either the editor or code editor to open the file<br>\n5.\tsave changes after performing any edits<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Block Design and Development Servers<\/strong><\/h3>\n\n\n<p>Many agencies will put new work online so that remote workers can collaborate and clients can view progress.  It&#8217;s important that these instances block search engines so that duplicate content doesn&#8217;t occur.<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<p>A blanket robots.txt entry can stop search crawlers from accessing any of the site:<br>\nUser-agent: *<br>\nDisallow: \/<\/p>\n<p>This entry tells all bots *  to ignore \/  all pages on site.<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<p>DO NOT USE THE ABOVE ON YOUR MAIN SITE!<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Double-Checking robots.txt<\/strong><\/h3>\n\n\n<p>There are two easy ways of checking the current robots.txt file, without logging in to cPanel.<\/p>\n<p>Visit the Site<br><a href=\"https:\/\/www.site.com\/robots.txt\">www.site.com\/robots.txt<\/a><\/p>\n<p>This will display the robots.txt file as seen by search crawlers.<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<p><a href=\"https:\/\/www.google.com\/webmasters\/tools\/robots-testing-tool\">Google Search Console robots.txt Tester<\/a><br>Google will report on what it has found and additional info on the impact.<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<p>There&#8217;s also a submit function where you can let Google know if you&#8217;ve updated your robots.txt and that the crawler needs to revisit.<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Best Practice for WordPress robots.txt<\/strong><\/h3>\n\n\n<p>Advice varies, but a generally accepted approach is to allow all search engines to crawl all pages, except the \/wp-admin\/ folder, yet still allow public access to admin-ajax.php (which is done because many themes and plugins rely on this, without which the site won&#8217;t render or function properly, possibly).<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<p>User-agent: *<br>\nDisallow: \/wp-admin\/<br>\nAllow: \/wp-admin\/admin-ajax.php<\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n<p>Looking for web hosting? KnownHost is industry leading in fast and reliable services for any hosting needs! <strong><a href=\"https:\/\/www.knownhost.com\">Learn More Here!<\/a><\/strong><\/p>\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<div style=\"height:20px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n","protected":false},"excerpt":{"rendered":"<p>Updated June 4, 2020 Articles about SEO and how to improve site performance will frequently bounce terms about including crawling, indexing, robots.txt, meta robots, canonical and more. It&#8217;s absolutely essential to understand the difference between crawling and indexing, plus why it matters to your site performance in search rankings. Matt C. at Google explains the [&hellip;]<\/p>\n","protected":false},"author":182,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[121],"tags":[2355,2357,2356,2354],"class_list":["post-15767","post","type-post","status-publish","format-standard","hentry","category-wordpress","tag-crawling","tag-editing","tag-indexing","tag-seo-performance"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.5 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Crawling vs. Indexing + How To Fix - KnownHost Crawling vs Indexing - SEO<\/title>\n<meta name=\"description\" content=\"Crawling vs. Indexing, which is better for your SEO rankings? Here&#039;s what we have to say about both, PLUS how to fix it!\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.knownhost.com\/blog\/crawling-vs-indexing-how-to-fix\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Crawling vs. Indexing + How To Fix - KnownHost Crawling vs Indexing - SEO\" \/>\n<meta property=\"og:description\" content=\"Crawling vs. Indexing, which is better for your SEO rankings? Here&#039;s what we have to say about both, PLUS how to fix it!\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.knownhost.com\/blog\/crawling-vs-indexing-how-to-fix\/\" \/>\n<meta property=\"og:site_name\" content=\"KnownHost\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/KnownHost\/\" \/>\n<meta property=\"article:published_time\" content=\"2020-06-04T19:05:08+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2020-10-16T11:43:13+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.knownhost.com\/blog\/wp-content\/uploads\/2020\/06\/photo-1498050108023-c5249f4df085.jpeg\" \/>\n\t<meta property=\"og:image:width\" content=\"1504\" \/>\n\t<meta property=\"og:image:height\" content=\"1001\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Emily Moore\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@KnownHost\" \/>\n<meta name=\"twitter:site\" content=\"@KnownHost\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Emily Moore\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.knownhost.com\\\/blog\\\/crawling-vs-indexing-how-to-fix\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.knownhost.com\\\/blog\\\/crawling-vs-indexing-how-to-fix\\\/\"},\"author\":{\"name\":\"Emily Moore\",\"@id\":\"https:\\\/\\\/www.knownhost.com\\\/blog\\\/#\\\/schema\\\/person\\\/b15716b5af83cfb7bfd5bd13ecf985fc\"},\"headline\":\"Crawling vs. Indexing + How To Fix\",\"datePublished\":\"2020-06-04T19:05:08+00:00\",\"dateModified\":\"2020-10-16T11:43:13+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.knownhost.com\\\/blog\\\/crawling-vs-indexing-how-to-fix\\\/\"},\"wordCount\":813,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/www.knownhost.com\\\/blog\\\/#organization\"},\"keywords\":[\"crawling\",\"editing\",\"indexing\",\"seo performance\"],\"articleSection\":[\"WordPress\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/www.knownhost.com\\\/blog\\\/crawling-vs-indexing-how-to-fix\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.knownhost.com\\\/blog\\\/crawling-vs-indexing-how-to-fix\\\/\",\"url\":\"https:\\\/\\\/www.knownhost.com\\\/blog\\\/crawling-vs-indexing-how-to-fix\\\/\",\"name\":\"Crawling vs. Indexing + How To Fix - KnownHost Crawling vs Indexing - SEO\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.knownhost.com\\\/blog\\\/#website\"},\"datePublished\":\"2020-06-04T19:05:08+00:00\",\"dateModified\":\"2020-10-16T11:43:13+00:00\",\"description\":\"Crawling vs. Indexing, which is better for your SEO rankings? Here's what we have to say about both, PLUS how to fix it!\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.knownhost.com\\\/blog\\\/crawling-vs-indexing-how-to-fix\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.knownhost.com\\\/blog\\\/crawling-vs-indexing-how-to-fix\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.knownhost.com\\\/blog\\\/crawling-vs-indexing-how-to-fix\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.knownhost.com\\\/blog\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Crawling vs. Indexing + How To Fix\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.knownhost.com\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/www.knownhost.com\\\/blog\\\/\",\"name\":\"KnownHost.com\",\"description\":\"Quality Managed Web Hosting by KnownHost\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.knownhost.com\\\/blog\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.knownhost.com\\\/blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.knownhost.com\\\/blog\\\/#organization\",\"name\":\"KnownHost, LLC\",\"url\":\"https:\\\/\\\/www.knownhost.com\\\/blog\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.knownhost.com\\\/blog\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.knownhost.com\\\/blog\\\/wp-content\\\/uploads\\\/2021\\\/04\\\/logo.png\",\"contentUrl\":\"https:\\\/\\\/www.knownhost.com\\\/blog\\\/wp-content\\\/uploads\\\/2021\\\/04\\\/logo.png\",\"width\":1200,\"height\":400,\"caption\":\"KnownHost, LLC\"},\"image\":{\"@id\":\"https:\\\/\\\/www.knownhost.com\\\/blog\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/KnownHost\\\/\",\"https:\\\/\\\/x.com\\\/KnownHost\",\"https:\\\/\\\/www.linkedin.com\\\/company\\\/knownhost\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.knownhost.com\\\/blog\\\/#\\\/schema\\\/person\\\/b15716b5af83cfb7bfd5bd13ecf985fc\",\"name\":\"Emily Moore\",\"sameAs\":[\"https:\\\/\\\/www.knownhost.com\"],\"url\":\"https:\\\/\\\/www.knownhost.com\\\/blog\\\/author\\\/emily-honeycutt\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Crawling vs. Indexing + How To Fix - KnownHost Crawling vs Indexing - SEO","description":"Crawling vs. Indexing, which is better for your SEO rankings? Here's what we have to say about both, PLUS how to fix it!","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.knownhost.com\/blog\/crawling-vs-indexing-how-to-fix\/","og_locale":"en_US","og_type":"article","og_title":"Crawling vs. Indexing + How To Fix - KnownHost Crawling vs Indexing - SEO","og_description":"Crawling vs. Indexing, which is better for your SEO rankings? Here's what we have to say about both, PLUS how to fix it!","og_url":"https:\/\/www.knownhost.com\/blog\/crawling-vs-indexing-how-to-fix\/","og_site_name":"KnownHost","article_publisher":"https:\/\/www.facebook.com\/KnownHost\/","article_published_time":"2020-06-04T19:05:08+00:00","article_modified_time":"2020-10-16T11:43:13+00:00","og_image":[{"width":1504,"height":1001,"url":"https:\/\/www.knownhost.com\/blog\/wp-content\/uploads\/2020\/06\/photo-1498050108023-c5249f4df085.jpeg","type":"image\/jpeg"}],"author":"Emily Moore","twitter_card":"summary_large_image","twitter_creator":"@KnownHost","twitter_site":"@KnownHost","twitter_misc":{"Written by":"Emily Moore","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.knownhost.com\/blog\/crawling-vs-indexing-how-to-fix\/#article","isPartOf":{"@id":"https:\/\/www.knownhost.com\/blog\/crawling-vs-indexing-how-to-fix\/"},"author":{"name":"Emily Moore","@id":"https:\/\/www.knownhost.com\/blog\/#\/schema\/person\/b15716b5af83cfb7bfd5bd13ecf985fc"},"headline":"Crawling vs. Indexing + How To Fix","datePublished":"2020-06-04T19:05:08+00:00","dateModified":"2020-10-16T11:43:13+00:00","mainEntityOfPage":{"@id":"https:\/\/www.knownhost.com\/blog\/crawling-vs-indexing-how-to-fix\/"},"wordCount":813,"commentCount":0,"publisher":{"@id":"https:\/\/www.knownhost.com\/blog\/#organization"},"keywords":["crawling","editing","indexing","seo performance"],"articleSection":["WordPress"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.knownhost.com\/blog\/crawling-vs-indexing-how-to-fix\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.knownhost.com\/blog\/crawling-vs-indexing-how-to-fix\/","url":"https:\/\/www.knownhost.com\/blog\/crawling-vs-indexing-how-to-fix\/","name":"Crawling vs. Indexing + How To Fix - KnownHost Crawling vs Indexing - SEO","isPartOf":{"@id":"https:\/\/www.knownhost.com\/blog\/#website"},"datePublished":"2020-06-04T19:05:08+00:00","dateModified":"2020-10-16T11:43:13+00:00","description":"Crawling vs. Indexing, which is better for your SEO rankings? Here's what we have to say about both, PLUS how to fix it!","breadcrumb":{"@id":"https:\/\/www.knownhost.com\/blog\/crawling-vs-indexing-how-to-fix\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.knownhost.com\/blog\/crawling-vs-indexing-how-to-fix\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.knownhost.com\/blog\/crawling-vs-indexing-how-to-fix\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.knownhost.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Crawling vs. Indexing + How To Fix"}]},{"@type":"WebSite","@id":"https:\/\/www.knownhost.com\/blog\/#website","url":"https:\/\/www.knownhost.com\/blog\/","name":"KnownHost.com","description":"Quality Managed Web Hosting by KnownHost","publisher":{"@id":"https:\/\/www.knownhost.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.knownhost.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.knownhost.com\/blog\/#organization","name":"KnownHost, LLC","url":"https:\/\/www.knownhost.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.knownhost.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.knownhost.com\/blog\/wp-content\/uploads\/2021\/04\/logo.png","contentUrl":"https:\/\/www.knownhost.com\/blog\/wp-content\/uploads\/2021\/04\/logo.png","width":1200,"height":400,"caption":"KnownHost, LLC"},"image":{"@id":"https:\/\/www.knownhost.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/KnownHost\/","https:\/\/x.com\/KnownHost","https:\/\/www.linkedin.com\/company\/knownhost"]},{"@type":"Person","@id":"https:\/\/www.knownhost.com\/blog\/#\/schema\/person\/b15716b5af83cfb7bfd5bd13ecf985fc","name":"Emily Moore","sameAs":["https:\/\/www.knownhost.com"],"url":"https:\/\/www.knownhost.com\/blog\/author\/emily-honeycutt\/"}]}},"_links":{"self":[{"href":"https:\/\/www.knownhost.com\/blog\/wp-json\/wp\/v2\/posts\/15767","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.knownhost.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.knownhost.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.knownhost.com\/blog\/wp-json\/wp\/v2\/users\/182"}],"replies":[{"embeddable":true,"href":"https:\/\/www.knownhost.com\/blog\/wp-json\/wp\/v2\/comments?post=15767"}],"version-history":[{"count":0,"href":"https:\/\/www.knownhost.com\/blog\/wp-json\/wp\/v2\/posts\/15767\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.knownhost.com\/blog\/wp-json\/wp\/v2\/media?parent=15767"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.knownhost.com\/blog\/wp-json\/wp\/v2\/categories?post=15767"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.knownhost.com\/blog\/wp-json\/wp\/v2\/tags?post=15767"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}