According to a new survey from cloud security provider Distil Networks humans accounted for only 40.9 percent of web traffic in 2014 with the remainder accounted for by bots.
This compares with 50.8 percent human traffic in 2013, however the good news is that 'bad bot' traffic is down from 24.22 percent to 22.78. The rise in good bot traffic that makes up the difference is thought to be from more aggressive indexing by Bing and new search engines in 2014.
But although the percentage of bad bot traffic is down, Distil still saw 23 billion threats in 2014. The bad bots are also increasingly mimicking human behavior to evade detection, only 59 percent now behave like bots. Small sites suffer most from bad bots as they account for 32.04 percent of their traffic. Large sites see more good bots (43.65 percent), though as Distil points out this can still be undesirable due to the amount of traffic involved. Digital publishing and travel websites suffer the most bad bot traffic whilst the good bots account for more traffic on directory and real estate sites.
The rise of mobile has had an influence on the bot landscape too. There are more bots masking themselves as mobile users. The same characteristics that make mobile sites easier for humans to navigate also benefit the bots.
In 2013 Verizon Business accounted for almost 11 percent of all bad bot traffic, but the company has cleaned up its act with only 2.83 percent last year. Amazon Technologies is now top of the list originating 15.07 percent of all bad bot traffic thanks to its presence in the cheap hosting market.
The US with thousands of cheap hosts, dominates the rankings in bad bot origination. However, taken in isolation bad bot volume data can be somewhat misleading. Distil measures bad bots per online user to calculate a country's 'Bad Bot GDP'. On that measure Singapore tops the chart with 152.87 bad bots per user while the US slips to sixth with only 6.34.
The report concludes, "The bad bot landscape continues to evolve rapidly, particularly with the dramatic increase in mobile bot traffic, and an ever wider range of geographic and ISP points of origin. With the advent of cheap or free cloud computing resources, anyone with basic computer skills can download open source software and get into the game". But it notes that most companies still don't have visibility into or control over malicious traffic.
The full study is available to download from the Distil Networks site.
Image credit: Gunnar Assmy/Shutterstock