Your website has gone from being in the Google search results for
several years to not even being included in Google’s index.
What happened?

I generally spend a few hours a week taking part in discussions and lending the occasional helping hand at the Google Webmaster Help group, a forum set up to help address the problems and technical queries of frustrated webmasters, designers and SEOs with regards to anything from site verification to indexing & crawling; in fact, just about any Google related issue imaginable.

Populated by a colourful mix of regular characters and part-time contributors from across the globe as well as more than a few of Google’s own staff, it acts as a kind of front-line support forum which has thus far attracted a membership of over 16,000.
Indeed, this article is the result of one particular discussion thread which caught my eye this afternoon.

What happened to my site?
As another user quite rightly pointed out, this is by far the most commonly asked question in the Google webmaster help group.
Which is to say; “My web site was ranking very well up until recently, but has now dropped in the results or disappeared from the Google index altogether.“

Although the comment about this question’s frequency was initially made as part of a post highlighting the need for a properly structured FAQ section, it eventually prompted a heated debate regarding Google’s spam report and penalty system which raised a few interesting points of its own.

Persecuted by Google?
The general stance taken by group members on one side of the discussion was that, if a web site simply disappears from the index without having previously employed ‘black hat’ tactics, the fault must be Google’s for persecuting it as a result of malicious spam reports made by competitors.

Having seen this type of allegation levelled against Google before, I followed the discussion with great interest wondering where things would end up.
After some to and fro involving the usual mix of paranoia, guesses and solid advice, one user pointed out that the issue had been briefly addressed only a short while ago by Google’s Adam Lasnik, stating:

“Spamfighting does not factor in a “popularity of the commons” scheme whereby if [x] people vote a site off the Google Island; it is ceremoniously dumped into /dev/null.
Put more directly: Having someone (or even 42 MILLION people) report a site as spam will not change how we view a site.

Our spam report, rather, helps us to become aware of pages violating our guidelines that we might not yet have crawled… enabling us to have another datapoint in our search quality efforts.” Full Post

To quote one user’s interpretation of the above statement:

“If it were really this easy to bump your competitor out of the rankings, every site in the world would now be banned and the Google index would be empty.
When a spam/abuse report is filed, Google looks at the spam techniques used and investigates ways to improve its algorithm to find these spam techniques in the future on all sites - not one site in particular.”

If a web site loses its rankings or is dropped from the Google index altogether, there is usually a reason; an issue which needs to be addressed in order to regain the lost ground or be re-included in the search results.
Despite the fact that Google’s Judge, Jury and Executioner approach of deleting sites from the index without notice may seem draconian at first glance, one should remember that it has become a search-leviathan, not only having to deal with an almost unfathomable number of indexed pages, but also having to differentiate between relevant and irrelevant resources whilst weeding out black-hat techniques in the process.

Speaking from personal experience, it seems to me that many webmasters are much quicker to level the finger at Google (and other engines) than they are to diagnose their own web sites in order to find out whether something is wrong.

Furthermore, a fair proportion of them still seem to be entirely unfamiliar with Google’s Webmaster Tools or even the Google Webmaster Guidelines.
The fact is that, love it or hate it; Google provides its users with a whole array of applications to aid the monitoring and ongoing promotion of their sites, as well as a host of other resources, including blogs, help pages and even the occasional piece of personal advice via the webmaster help group.

So, whilst Google-bashing is rapidly becoming a favourite pastime for some webmasters whose sites have been caught in the latest algorithm update or are struggling in the face of growing competition, I cannot help but wonder if their time and efforts would not be better spent redeveloping their web sites, addressing W3C Compliance issues and/or generating high-quality content and inbound links to aid their chances of future success.

In the immortal words of Henry Ford; “Don’t find fault. Find a remedy.“

If all else fails, take a look through the webmaster help group’s archives and, if you don’t find the answer to your problem there, post a question.
There’s usually someone at hand willing to give advice.

About the Author

With a writing career spanning well over a decade, Sasch Mayer currently lives in Larnaca, where he works under contract to IceGiant Web Design in Cyprus.
You will find further, non-solicited articles, tips and quality web and graphic design services at the IceGiant web site.