Google got accidentally DDoSed by an unnamed European network carrier, sending its cloud service offline on the old continent for about an hour.
The incident occurred last Monday, but it is only now that Google has decided to explain exactly what happened.
According to a Google blog post, the accidental denial of service happened in Google’s Europe1-West region, sending it offline for around 70 minutes. The blog post says that Google’s engineers activated an additional link with an unnamed network carrier with whom the company already shares many peering links globally.
The majority of the addresses affected by the error were located in eastern Europe and the Middle East, and Google saw its traffic decline 13 percent during that time.
As the link was activated, the peer’s network signalled that it could route traffic to many more destinations than Google engineers had anticipated, and more than the link had capacity for, the blog post reads.
"Google’s network responded accordingly by routing a large volume of traffic to the link. At 11:55, the link saturated and began dropping the majority of its traffic".
This would not have happened if automation was operational, Google says, adding that it usually is.
"In normal operation, peering links are activated by automation whose safety checks would have detected and rectified this condition. In this case, the automation was not operational due to an unrelated failure, and the link was brought online manually, so the automation’s safety checks did not occur".
To stop this issue from flaring up again, Google’s network engineers have taken steps to disallow manual link activation.
Published under license from ITProPortal.com, a Net Communities Ltd Publication. All rights reserved.
Photo credit: sibgat / Shutterstock