How to find Bot traffic and segment it out in Google Analytics

By Beacon News | Published February 1, 2016 | Categories: Google Analytics

Recently, I had a client come to me with this question (and mind you, we don’t manage Google Analytics for them. They were consulting with us on this problem):

“My traffic for Q1 has grown 86% YOY but they are not buying on my site. Is there something wrong?” 

My first thought: 86% – Whoa! – that’s some serious growth! My second thought: No way that can be real traffic growth (knowing this client doesn’t do much SEO for their site). So, I wondered if this issue is bot related. I jump into the data and start digging around...

Here are the steps I took to figure out the problem and fix it.

1st - Figure out where the increase in Traffic came from

To do this, I pulled session data by medium for Q1 YOY and right away I see a huge increase in traffic related to one medium….Direct. To be exact it was an increase of 376%, almost 134,000 visits! Now I’m definitely thinking my assumption about it being bot traffic is right.

2nd – Find the Bot Traffic

Now that we know the increase is related to direct, I start looking at ways to figure out if its bot traffic. I move on to look at Browser traffic. Right away I see Chrome had a significant increase in sessions and new users.

So now I know I need to look deeper into Chrome and click to see the browser versions.

Low and behold, the very first browser version 39.0.2171.95 shows an increase in session by 2,528% and new users by 3,690%. At this point I was totally getting all giddy and excited. You know that feeling you get on Christmas morning when you’ve just woke up and you can’t wait to see what presents you got but you have to wait just a little while longer on the rest of the family to wake up. Well that was me. I was so excited and anxious, knowing I was so close to getting the answer and fixing the problem but wasn’t there quite yet! (Insert childish grin here.)

Ok - so back on track now. We know the issue is coming from Chrome and specifically version 39.0.2171.95. And we know this is the issue because sessions drastically increase, bounce rate is 99.85%, time on site is 00:00:01 and there is no revenue. (I mean come on, I know the site needs improvements but it’s not that bad!) And remember this is for all traffic, no filter has been applied yet.
Now I want to see if any of the traffic coming from this browser is good traffic or not. So I add a secondary dimension for medium. I can see that there is some good traffic tied to other mediums but majority of the bad traffic is direct.

So now I turn on an advanced segment for direct traffic and turn off all sessions. After doing that, I change the secondary dimension from medium to landing page to see if I can narrow down a page that is being hit.

As luck would have it (and as I had assumed), the homepage showed a huge increase in sessions. So given all the information, I now had I could safely make the assumption: It was a bot causing the increase in sessions to the site and the problem was coming from Chrome and the homepage was involved. Using all that info, I can now set up an advanced segment to filter out the bot traffic.

 3rd – Set up an Advanced Segment to Filter out Bot Traffic

To set up the advance segment, click the + sign then click on Conditions in the left navigation. Now fill in the field using this information:

But because I’m super nice, here is a direct link to open and save the advanced segment in your Google analytics account: http://bit.ly/1nD00GH (Keep in mind you’ll need to tweak some of the fields based on your sites data such as landing page.)

You’re welcome ;)

 4th – Review the Data

Now that the advance segment has been saved, I turn off the direct traffic advanced segment and add back the all sessions segment. Once that is done loading, I return to the all traffic>source medium report and switch the primary dimension to medium. I now can see that only direct traffic shows a change in sessions and the behavior stats look a lot better. I also double check no other mediums were affected by the advance segment and make sure sessions numbers match for exclude and all sessions…which they do. So we’re good to go.

As a disclaimer – looking at the browser report I could see it looked like some bot traffic might still be coming through by the number was so low that it wouldn’t be a problem so I decided to leave the advanced segment as is instead of adding anything to it.