So you show up. Suddenly your traffic has taken a big drop. One thing I would do very early on is I would do site:mydomain.com and figure, out are you completely not showing up in Google, or do parts of your site show up in Google? That’s also a really good way to find out whether you are partially indexed. Or if you don’t see a snippet, then maybe you had a robots.txt that blocked us from crawling. So we might see a reference to that page, and we might return something that we were able to see when we saw a link to that page, but we weren’t able to see the page itself, or weren’t able to fetch it. You might also notice in the search results if we think that you’re hacked or have malware, then we might have a warning there. And that could, of course, lead to a drop in traffic if people see that and decide not to go to the hacked site or the site that has malware.
The next place I’d look is the webmaster console. Google.com/webmasters prove that you control or own the site in a variety of ways. And we’re doing even more messages than we used to do. Not just things like hidden text, park domains, and doorway pages. Actually, quite a few different types of messages that we’re publishing now, and when we think there’s been a violation of our quality guidelines. If you don’t see any particular issue or message listed there, then you might consider going to the Webmaster forum. That’s also linked to from google.com/webmasters. And there can be a lot of helpful people, including superusers, who might have ideas or suggestions on things to check out. And as part of that, you might end up asking yourself, is this affecting just my site or a lot of other people? If it’s just your site, then it might be that we thought that your site violated guidelines, or of course, it could be a server-related issue or an issue on your site, of course, on your side.
And so if a bunch of people are all seeing a particular change, then it might be more likely to be something due to an algorithm. You can also check other search engines, because if other search engines aren’t listing you, that’s a pretty good way to say, well, maybe the problem is on my side. So maybe I’ve deployed some test server, and maybe it had a robots.txt or a noindex so that people wouldn’t see the test server, and then you pushed it live and forgot to remove the noindex.
You can also do Fetch as Googlebot. That’s another method. That’s also in our Google Webmaster console. And what that lets you do is send out Googlebot and actually retrieve a page and show you what it fetched. And sometimes you’ll be surprised. It could be hacked or things along those lines, or people could have added a noindex tag, or a rel=canonical that pointed to a hacker’s page. So things along those lines are stuff to check into as such as fetch as Googlebot.
We’ve also seen a few people who, for whatever reason, were cloaking, and did it wrong, and shot themselves in the foot. And so they were trying to cloak, and instead, they returned normal content to users and completely empty content to Googlebot.
Certainly if you’ve changed your site, your hosting, if you’ve revamped your design, a lot of that can also cause things. So you want to look at if there’s any major thing you’ve changed on your side, whether it be DNS, hostname, anything along those lines, around the same time. That can definitely account for things. If you deployed something that’s really sophisticated Ajax, maybe the search engines are quite able to crawl that and figure things out. So by that time hopefully, you have an idea whether it affects just your site, all of your site, your site in different search engines, or just you, whether it affects multiple different people, so it might be an algorithmic change, those kinds of issues. And once you have that under your belt and you have an idea of what’s going on, then you can ask whether you should do a reconsideration request.
So if you think there might have been some violation, like you were cloaking, and you were returning an empty page to Google, or hidden text, or keyword stuffing, whatever, then you can do a reconsideration request. And that will actually tell you whether Google has taken manual action on your site. And if it hasn’t, then that’s good, because you might not be ranking where you want, but at least you know it’s an algorithmic issue. And so you can concentrate on what different things might be triggering from an algorithmic standpoint.
Whereas, if it is a manual issue, then we’ll do our best to tell you, it’s been corrected, things will be in better shape soon, or we still think that there’s an issue going on. And so that can give you a lot of visibility. That’s just in the last six months or so, since the last time we did a Webmaster video, which we’ve really started to show a lot more detail, not only in the messages that we’re doing but also in the answers to reconsideration requests. So that can be another great source of visibility.
So those are some of the issues that I would look at. Hopefully that gives you a few tools to do the diagnoses, and I hope things get better for whichever sites you’re looking at, because if you’ve got great content, to the extent possible, and to the extent that we think it’s appropriate, we’d like to have that good content in our index so that users can find it as well.
by Matt Cutts - Google's Head of Search Quality Team