Microsoft Clarity · Status at source date: Newsletter archive
Microsoft Clarity shows which AI bots ignore robots.txt
Microsoft has added a Violations card to the Bot Analytics dashboard in Microsoft Clarity. For the first time, site owners can see which automated crawlers are accessing pages explicitly disallowed in robots.txt, shown as a percentage of total bot traffic with trends over time and filters by operator, bot name, and activity type.
What changed
It gives publishers visibility into which AI crawlers are ignoring their stated crawl rules.
As AI crawlers proliferate and content owners try to control how their material is used for training and grounding, seeing which bots respect robots.txt: and which do not: becomes a practical governance tool.
Why it matters for advertisers
For you, this means site owners can identify AI crawlers that disregard robots.txt and quantify the traffic involved. Worth checking the Violations card if you are trying to control AI access to your content, then deciding whether additional blocking measures are warranted.
Perspective from the original PMC newsletter.Sources & contributor credit
- Newsletter coverage · Paid Media Collective newsletter
Original newsletter text, contributor labels and media for this update.
- Source referenced in newsletterMicrosoft Clarity
Linked from the original newsletter. The source publication date has not been independently confirmed.
Original creator unverified
The original creator has not yet been verified. Newsletter curation and publication do not establish original authorship.
Attribution evidence and limitations
Full attribution review pending.
A source link alone does not establish original authorship.
Original image and video creators have not yet been verified.
- Published on this site
This update reflects the dated source reporting. Availability may have changed. Further coverage of this same development will be added to this page.
Original newsletter text and archive evidence
Microsoft Clarity shows which AI bots ignore robots.txt
Microsoft has added a Violations card to the Bot Analytics dashboard in Microsoft Clarity. For the first time, site owners can see which automated crawlers are accessing pages explicitly disallowed in robots.txt, shown as a percentage of total bot traffic with trends over time and filters by operator, bot name, and activity type. It gives publishers visibility into which AI crawlers are ignoring their stated crawl rules.
As AI crawlers proliferate and content owners try to control how their material is used for training and grounding, seeing which bots respect robots.txt: and which do not: becomes a practical governance tool.
For you, this means site owners can identify AI crawlers that disregard robots.txt and quantify the traffic involved. Worth checking the Violations card if you are trying to control AI access to your content, then deciding whether additional blocking measures are warranted.
Source captured . No explicit first-contributor label was provided for this update.
