Bot Detection

The owntag Bot Detection analyzes incoming requests and identifies whether the request comes from a bot, such as a search engine crawler or an SEO tool spider.
Generally, you want to block such bots, especially for web analytics, to prevent them from distorting the data measured from real users.

Whether the request comes from a bot is made available in your SGTM via the HTTP header x-owntag-is-bot.
If the request comes from a bot, the header value is true; otherwise, it’s false.

How Does Bot Detection Work?

Bot Detection analyzes the User-Agent header of incoming requests. A request from a normal user looks like this:

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3

A request from a bot looks like this:

CCBot/2.0 (https://commoncrawl.org/contact/)

Bot Detection analyzes the User-Agent string and compares it with a list of over 500 known bot User-Agent strings. Since "CCBot" is part of this list, in this case, the header x-owntag-is-bot has the value true.

How Do I Use This Information in SGTM?

Reading the header

To read the bot status, you need to extract the header from the incoming tracking request. To do this, create a new variable of type “Request Header” (or “Request Header” in the English UI) named x-owntag-is-bot:

Sending bot status to GA4

If you don’t want to completely exclude bot requests, you can pass the bot status to GA4 (or similarly to other tools) and then filter it during analysis. This gives you more control over the data included in your tracking and allows you to understand the impact of bots on your data.
To send the bot status to GA4, you can add an additional parameter to your GA4 tags. In this example, we’ve named it bot_status:

To display the true or false value of the bot status in your GA4 reports, you need to add the parameter as a custom dimension in the GA4 configuration:

From then on, the value is available as a dimension in the GA4 interface, and you can, for example, filter data where the dimension has the value true.

Alternative: Exclude bot requests completely

If you prefer to completely exclude bot requests, you can use the bot status as a criterion for triggering triggers and have your triggers fire only when x-owntag-is-bot has the value false: