It’s official: Google is no God but just human ;-)

Last week we had some panic time at work. Trust me, it wasn’t fun. Few of our Oneindia.in channels started being shown as “infected”. If you are a Firefox user you would be shown a warning which said the site was infected with Malware and the reasons too were shown.

Google and Stopbadware work “together” to maintain a list of infected sites. This list is used by Firefox to warn its users from getting infected. In Firefox 3.x visit Tools->Options->Security Tab, you will see a checkbox option,

Tell me if the site I’m visiting is a suspected attack site.

Google may or may not send you an email about this warning.

We recently discovered that some of your pages can cause users to be infected with malicious software. We have begun showing a warning page to users who visit these pages by clicking a search result on Google.com. Below are some example URLs on your site which can cause users to be infected (space inserted to prevent accidental clicking in case your mail client auto-links URLs):

http://thatshindi.oneindia .in/
http://thatshindi.oneindia .in/2008/05/24/
http://thatshindi.oneindia .in/2008/07/19/news/

Here is a link to a sample warning page:
http://www.google.com/interstitial?url=http%3A//thatshindi.oneindia.in/

We strongly encourage you to investigate this immediately to protect your visitors. Although some sites intentionally distribute malicious software, in many cases the webmaster is unaware because:

1) the site was compromised
2) the site doesn’t monitor for malicious user-contributed content
3) the site displays content from an ad network that has a malicious advertiser

If your site was compromised, it’s important to not only remove the malicious (and usually hidden) content from your pages, but to also identify and fix the vulnerability. We suggest contacting your hosting provider if you are unsure of how to proceed. StopBadware also has a resource page for securing compromised sites:
https://www.stopbadware.org/my-site-has-badware

Once you’ve secured your site, you can request that the warning be removed by visiting http://www.google.com/support/webmasters/bin/answer.py?answer=45432 and requesting a review. If your site is no longer harmful to users, we will remove the warning.

Sincerely,

Google Search Quality Team

Google Webmaster Tools – Error in Error Reporting

But many more of our sites were blocked and we never received an email. Most site publishers use Google Webmaster Tools (GWT), if we had logged into our account we would have seen the warning, but then GWT informed site publishers in the 11th hour. It was too late. When you log in to GWT you don’t see an “Error” warning against your site name, so we assume the site has no errors and don’t bother clicking on it.

GWT does show an “Error” warning if there is a crawl error. Similarly, GWT should have warned us about this Malware in a similar way.

GWT did show the URLs which it suspected were infected. But we had an issue with those URLs – many of those were invalid URLs (404), a sub-domain it reported was not in use for over a year.

Our team did everything to please Google – I guess we even stood on one foot for hours to please God (Google)!

Earlier only Firefox users were informed about infected sites but now this error was being flashed on Google search results. That hurt us. Our page views went Southwards.

Google Talks About This Issue

Google has posted about this “error in error reporting” on their blog.

If you did a Google search between 6:30 a.m. PST and 7:25 a.m. PST this morning, you likely saw that the message “This site may harm your computer” accompanied each and every search result. This was clearly an error, and we are very sorry for the inconvenience caused to our users.

What happened? Very simply, human error. Google flags search results with the message “This site may harm your computer” if the site is known to install malicious software in the background or otherwise surreptitiously. We do this to protect our users against visiting sites that could harm their computers.

Stopbadware too clarified on this issue.

One must appreciate Google for acknowledging the error on their part.

What Publishers Would Have Preferred

• GWT reported URLs that were not existent. Google+Stopbadware should check if these URLs were infected “today” before flagging them instead of telling us “90 days ago these URLs were infected”.
• GWT should include an “Error” warning on the Dashboard to get the attention of the publisher. Yes, GWT is a free service from Google and they are not obligated to do anything for us, this is a feature “HUMBLE request”
• GWT needs to start sending emails when such errors occur.
• The reasons for infection are not very clear. For e.g. it said we had a URL childhe.com in our pages. We never did in the past and surely not now.

Anyways, the past is past. Let us hope Feb is a far better month than Jan (in terms of page views and revenues!)