> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.pixalate.com/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.pixalate.com/_mcp/server.

# User-agent feed

> Pixalate's user-agent block list, with daily and weekly files, file names, schema, polling and how to combine it with the IP block lists using OR logic.

The user-agent feed lists user-agent (UA) strings used by non-human traffic. Bots often use UA strings that reveal what they are, so blocking those strings is another way to keep non-human traffic out.

## What's in the feed

The list starts from the IAB's list of valid UA strings and its list of known crawlers and spiders (the IAB/ABC Spiders & Bots List). Pixalate adds UA strings that its own data shows performing fraud, so it can catch new bot UA strings before they reach the IAB list.

Every UA on the list is a deterministic, high-risk entry (equivalent to a probability of 1.0). Entries generally fall into three groups:

* Known bots, spiders and crawlers, including those on the IAB/ABC Spiders & Bots List.
* UAs linked to pervasive sophisticated invalid traffic (SIVT), often from complex bot networks (botnets).
* UAs linked to hyperactive users, with prolonged activity and outlier traffic beyond Pixalate's thresholds.

In practice that covers:

* Search engine crawlers that declare who they are, such as Googlebot.
* Custom crawlers, for example UA strings containing keywords such as `grabber` or `python`.
* UA strings that are missing the "must-have" keywords that, per IAB guidelines, characterize a valid UA string.

UA strings are easy to spoof or self-declare, so use the most recent file you have, and combine the feed with post-bid measurement to tune it for your traffic.

## At a glance

|                | Daily list                | Weekly list            |
| -------------- | ------------------------- | ---------------------- |
| Update cadence | Daily                     | Weekly, on Fridays     |
| File format    | CSV                       | CSV                    |
| File name      | `UAblacklisting_YYYYMMDD` | `UAblacklistingWeekXX` |
| Columns        | `browserAgent`            | `browserAgent`         |
| File size      | Up to 5 MB                | Up to 5 MB             |

The feed comes in a daily and a weekly version. To confirm which one you receive, ask your Pixalate account manager.

## Schema

The file has a single column. There's no probability or fraud type column, because every entry is deterministic.

| Column         | Type   | Description                   |
| -------------- | ------ | ----------------------------- |
| `browserAgent` | string | A user-agent string to block. |

Example rows (illustrative):

```csv
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
python-requests/2.31.0
```

## Apply it pre-bid with the IP lists

The UA feed and the [IP block lists](/data-feeds/ip-block-lists) are separate lists. Use them together as a two-step filter: each makes the other more effective.

**Block a request if its user agent is on the UA list OR its IP is on the IP list** (at or above your IP threshold). Don't block only when both match, and don't build a combined list of UA and IP pairs.

Here's why. Say Googlebot crawls from one of Google's IP addresses, 198.51.100.66 in this example:

| UA string                                                                  | IP address    |
| -------------------------------------------------------------------------- | ------------- |
| `Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)` | 198.51.100.66 |

If you match on the pair, the crawler gets through as soon as it changes either its UA or its IP. With two separate lists in OR logic, the UA list catches Googlebot from any IP in the world, and the IP list catches any crawler from that IP, even with a UA string it hasn't used before.

The order of the two checks doesn't matter, as long as you combine them with OR.

## Check for new files

* **Daily list:** check once an hour. Download only when a new file is available. Keep checking hourly after you download, because more than one file is sometimes uploaded in a day.
* **Weekly list:** on Fridays, check once an hour. Download only when a new file is available, then stop checking until the next week.

For access and file locations, see [Delivery and integration](/data-feeds/delivery).

## Related

#### [IP block lists (IPv4 and IPv6)](/data-feeds/ip-block-lists)

Use with the UA feed in OR logic.

#### [Data center feeds](/data-feeds/data-center)

Weekly IPv4 and IPv6 CIDR ranges of hosting and cloud providers.

#### [Delivery and integration](/data-feeds/delivery)

Access, file locations and polling for every feed.