This guy was MAD as spit on a griddle.
Despite months of creating content, building links, doing on-page SEO and keyword research on his website and blog — he was getting ZERO love from Google.
The only way he could describe it to me on the phone that day was to say…
“I don’t think I’m even IN Google!”
5 minutes later I spotted the problem. It wasn’t that he wasn’t IN Google — it was quite the opposite. More on this in a second…
First, let’s look at the simple way I found the problem…
Forget keyword research, content marketing and link building… for now.
None of it matters if you have problems in the Google Index.
Take 5 minutes to go through this exercise with me… your business will thank you.
Visit google.com and type the following:
Where domain.com is your domain. Do NOT include any spaces, the http:// or www. — it’s important to type it exactly as I show above.
Here’s what the site: operator query looks like for etsy.com,
Press ENTER to search and you’ll see Google search results that are restricted to your website only.
In other words, these are the pages that are eligible to receive search traffic. If it’s not indexed — it won’t receive traffic from Google.
At the top of the results page you’ll see the number of results.
Here’s what it looks like for etsy.com…
You’ll likely have much less than 52 million+ results in the Google index.
But here’s the question… are you surprised by what you see in this quick SEO audit? Do the numbers look about right? Are there a lot fewer pages indexed than you thought? A lot more?
Note: Particularly for larger sites, this number is an estimate of the number of pages Google has in its index. But the larger your site, the more important this exercise.
Ok, let’s look at some “What If’s”…
So, you see no results…
Don’t panic… yet.
First, make sure you typed the site: operator correctly into Google. (See Step 1)
Then, ask these questions:
Have I done anything to cause Google to ban my website from the Google index? Manipulative link building? Cloaking? Hiding text?
Action Items: Check Google Webmaster Tools for notifications under the MANUAL ACTIONS report. If you see a penalty, consider a Google reconsideration request.
Am I restricting access to Google and/or other search engines? It’s possible that you or someone else have taken actions to keep search engines out of your website.
Action Items: Check your robots.txt file by visiting… domain.com/robots.txt where domain.com is your domain. Ensure that you are not restricting access to search engines.
For example the following syntax in your robots.txt file will restrict access to all search engines…
Click here for more information about interpreting the robots.txt file.
Secondly, if you’re using WordPress, ensure that you have the following setting under SETTINGS > READING unchecked…
Why would search engines be restricted from the entire site? Usually because the developer restricted access while the site was being built and never took the restrictions out when they site was completed.
It happens more than you would think.
Remember that, especially for larger sites, the Google index is only an estimate.
That said, crawling through the search results from the site: operator can reveal severe problems with the crawlability of your website.
For example, you might find entire directories (folders) of your website are not in the index. Or, you might find that critical pages are not in the index.
You can check whether specific pages are in the Google index using the site: operator as well…
Action Items: Check your robots.txt file to ensure that you don’t have entire directories on your website restricted. For example, the following syntax in your robots.txt file will restrict Google’s access to all pages in the /products/ folder…
Second, use a tool like Copyscape to ensure that you don’t have issues with duplicate content. Google does not want to include multiple copies of the same content in their index. If you have duplicate content on your own website or on other websites across the web, Google may have chosen to drop your page(s) from the index.
Lastly, check the source code of the page(s) in question and ensure that you don’t see a ‘noindex’ directive on that page. It will look like this…
<META NAME=”ROBOTS” CONTENT=”NOINDEX”>
With this tag in the source code of the page, Google (and other search engines) will not index the page.
Having gobs of pages in the Google index that have no value can be just as damaging as the other scenarios we’ve talked about.
In fact, here’s the rest of the story from the opening of this article…
This clients website contained a plug-in that was used to build pages for events conducted by the business.
Sounds great, right?
Not so much. This plug-in had generated a page on his website for every day for the next 30 years!
When I did the site: operator I found over 10,000 event pages that contained zero content, the same title tag and meta description. We counted approximately 100 pages on the site that contained high quality content that we would want Google to index.
As a result… the site looked like this to Google…
Long story short — Google doesn’t like that. We shut off the plug-in (and did some 301 redirects) and their SEO improved almost immediately. Suddenly Google could clearly see the valuable pages on this website.
Action Items: First things first, make sure you aren’t hacked. Unfortunately this is the most common reason you’ll find a shockingly large amount of pages in the Google index.
Take the website for the West Virginia Symphony for example…
This site is hacked at the time of this writing and it only takes a quick site: operator query to see it…
There are nearly 1,400 spammy pages injected into the West Virginia Symphony website and only a couple dozen useful pages with content on them.
Google has indexed them all.
So… the West Virginia Symphony site looks like this to Google…
Not good for this organization or classical music fans in West Virginia.
Use a service like Sucuri to get your website cleaned up if you’ve been hacked.
Second, look for large quantities of pages that contribute no value to your business and don’t belong in the Google index. Use your robots.txt file to restrict access to these pages or delete them from your website being sure to 301 redirect deleted pages to a relevant page.
Perform the site: operator function and browse through the pages in the Google index.
Look for anything out of the ordinary and consult an SEO professional if you see anything that might be harmful. Or, if you’re feeling frisky… fix it yourself.
Whatever you do… take action.
Alright… time to make dinner for the kids. I wonder what the folks over at Food Network would have me do with some leftover spaghetti noodles and chicken…
What did you find in the Google index during your 5 minute SEO audit? Let me know in the comments below.