Sign up for our daily briefing

Make your busy days simpler with Axios AM/PM. Catch up on what's new and why it matters in just 5 minutes.

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Catch up on the day's biggest business stories

Subscribe to Axios Closer for insights into the day’s business news and trends and why they matter

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Stay on top of the latest market trends

Subscribe to Axios Markets for the latest market trends and economic insights. Sign up for free.

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Sports news worthy of your time

Binge on the stats and stories that drive the sports world with Axios Sports. Sign up for free.

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Tech news worthy of your time

Get our smart take on technology from the Valley and D.C. with Axios Login. Sign up for free.

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Get the inside stories

Get an insider's guide to the new White House with Axios Sneak Peek. Sign up for free.

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Catch up on coronavirus stories and special reports, curated by Mike Allen everyday

Catch up on coronavirus stories and special reports, curated by Mike Allen everyday

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Want a daily digest of the top Denver news?

Get a daily digest of the most important stories affecting your hometown with Axios Denver

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Want a daily digest of the top Des Moines news?

Get a daily digest of the most important stories affecting your hometown with Axios Des Moines

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Want a daily digest of the top Twin Cities news?

Get a daily digest of the most important stories affecting your hometown with Axios Twin Cities

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Want a daily digest of the top Tampa Bay news?

Get a daily digest of the most important stories affecting your hometown with Axios Tampa Bay

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Want a daily digest of the top Charlotte news?

Get a daily digest of the most important stories affecting your hometown with Axios Charlotte

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Sign up for Axios NW Arkansas

Stay up-to-date on the most important and interesting stories affecting NW Arkansas, authored by local reporters

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!

A new report shows that more than half of Internet traffic comes from bots — software applications that perform automated tasks for almost anything you can think of on the internet.

Bots are so prevalent in the digital media ecosystem that even the most sophisticated publishers can't avoid them. The good news is that the increase in bot traffic last year was mostly due to the addition of good bots in the ecosystem, while bad bot traffic remained nearly the same.

(function () { var attempt = 0, init = function(){ if (window.pym) { var pymParent = new pym.Parent("2017-02-04-internet-robots", "https://graphics.axios.com/2017-02-04-internet-robots/internet-robots-chart.html", {}); } else if (attempt++ < 40) { setTimeout(init, 50); } }; init(); })();

Data: Imperva Incapsula Bot Traffic Report. Based on 16.7+ billion visits to 100,000 randomly-selected domains on the Incapsula network; Chart: Lazaro Gamio / Axios

Good bots: Almost a quarter of internet traffic is good bots, which monitor or enhance the web. Common examples of good bots are "spiders," which Google uses to map the internet to expand their search results, and "feed fetchers,"which Facebook uses to refresh your newsfeed on mobile. Together, Google search "spiders" and Facebook "feed fetchers" make up over 8% of all web traffic.

Bad bots: Bad bots are typically used to steal content from websites, mimic websites to capture ad dollars, create spam pages to collect user data, or shut down websites by overwhelming them with traffic. According to the same study, commissioned by cybersecurity firm Imperva Inc., bad bots make up nearly 30 percent of all internet traffic, and the majority of them impersonate real websites or people to bypass security measures. In December, White Ops, a cybersecurity firm focused on malware and ad fraud, uncovered the largest bad bot criminal ring in history, called the "Methbot." Methbot stole millions of advertising dollars, mostly from ad networks, by recreating common websites and stealing their ad traffic and revenue. White Ops Senior Director Mark Schlosser tells Axios that cybercriminals in the ad space have become so advanced that the traffic they steal "is able to fake all key advertising metrics including clicks, viewability and even simple email signups."

Why it matters: The Association of National Advertisers and White Ops estimate that bots cost advertisers more than $7 billion in revenue annually. Most ad fraud occurs in the programmatic space, where ads are purchased through an automated bidding system. Sophisticated ad agencies will place trackers on ads that enable them to only bid on quality inventory, which saves their clients money and makes for more effective ad campaigns. One of Washington's largest advertising agencies tells Axios that they have dozens of team members working to ensure advertising is seen by real people, not bots. "The investment we have made in technology to ensure we aren't buying fraudulent traffic is extensive," says Ben Coffey Clark, a partner at Bully Pulpit Interactive. "It's a trust we take very seriously."

Go deeper

Dems' immigration plan hits major roadblock

Senate Majority Leader Sen. Chuck Schumer. Photo: Alex Wong/Getty Images

The Senate parliamentarian ruled Sunday that Democrats cannot include pathways to citizenship in the $3.5 trillion budget reconciliation package, per a copy of the ruling obtained by Axios.

Why it matters: It's a blow to Democrats who hoped to provide pathways for millions of undocumented immigrants in the U.S. Using reconciliations would have allowed them to pass politically contentious immigration changes with only 50 votes, as opposed to the usual 60 required.

FBI says human remains found in Wyoming likely Gabby Petito

Gabby Petito. Photo: FBI

Human remains found in Teton County, Wyoming, are "consistent with the description of" missing 22-year-old Gabby Petito, said FBI Denver official Charles Jones at a news conference Sunday.

Details: The cause of death had yet to be determined, but Jones said: "Full forensic identification has not been completed to confirm 100% that we found Gabby, but her family has been notified of this discovery." Authorities said they're continuing the search for her fiancé, Brian Laundrie.

6 hours ago - Politics & Policy

Beto not even best Dem against Abbott

Beto O'Rourke speaks at a rally at the Texas State Capitol in June. Photo: Sergio Flores/Getty Images

Actor Matthew McConaughey’s nine-point lead in a theoretical matchup against Greg Abbott shows just how vulnerable the hard-right Texas governor could be in a general election.

Why it matters: Abbott has won conservative accolades for his abortion, mask and vaccine bans. Axios reported Sunday that former Rep. Beto O’Rourke is preparing to announce a gubernatorial challenge — but a recent poll shows he’s not even the most popular Democrat in the state.

You’ve caught up. Now what?

Sign up for Mike Allen’s daily Axios AM and PM newsletters to get smarter, faster on the news that matters.

Please enter a valid email.

Please enter a valid email.

Subscription failed
Thank you for subscribing!