The author’s views are entirely his or her own (excluding the unlikely event of hypnosis) and may not always reflect the views of Allrss.
In this week’s episode of Whiteboard Friday, host Jes Scholz digs into the foundations of search engine crawling. She’ll show you why no indexing issues doesn’t necessarily mean no issues at all, and how — when it comes to crawling — quality is more important than quantity.
Click on the whiteboard image above to open a high resolution version in a new tab!
Video Transcription
Good day, Allrss fans, and welcome to another edition of Whiteboard Friday. My name is Jes Scholz, and today we’re going to be talking about all things crawling. What’s important to understand is that crawling is essential for every single website, because if your content is not being crawled, then you have no chance to get any real visibility within Google Search.
So when you really think about it, crawling is fundamental, and it’s all based on Googlebot’s somewhat fickle attentions. A lot of the time people say it’s really easy to understand if you have a crawling issue. You log in to Google Search Console, you go to the Exclusions Report, and you see do you have the status discovered, currently not indexed.
If you do, you have a crawling problem, and if you don’t, you don’t. To some extent, this is true, but it’s not quite that simple because what that’s telling you is if you have a crawling issue with your new content. But it’s not only about having your new content crawled. You also want to ensure that your content is crawled as it is significantly updated, and this is not something that you’re ever going to see within Google Search Console.
But say that you have refreshed an article or you’ve done a significant technical SEO update, you are only going to see the benefits of those optimizations after Google has crawled and processed the page. Or on the flip side, if you’ve done a big technical optimization and then it’s not been crawled and you’ve actually harmed your site, you’re not going to see the harm until Google crawls your site.
So, essentially, you can’t fail fast if Googlebot is crawling slow. So now we need to talk about measuring crawling in a really meaningful manner because, again, when you’re logging in to Google Search Console, you now go into the Crawl Stats Report. You see the total number of crawls.
I take big issue with anybody that says you need to maximize the amount of crawling, because the total…