AT&T: AI-Personalized Creative Outperformed BAU and Random by a Wide Margin | MMA / Marketing + Media Alliance

AT&T: AI-Personalized Creative Outperformed BAU and Random by a Wide Margin

MMAGlobal's picture
Key Stat: 
+272%
Key Stat Supporting Text: 
Lift vs. Random Rotation
Key Stat Footnote: 
AI-Off baseline performance
Key Stat: 
+92%
Key Stat Supporting Text: 
Lift vs. BAU
Key Stat Footnote: 
Google optimized, 65/35 earn-learn
Key Stat: 
+88%
Key Stat Supporting Text: 
Sales Conversion Lift
Key Stat Footnote: 
Matchback validation
Key Stat: 
$15:1
Key Stat Supporting Text: 
ROI on AI Tech Cost
Key Stat Footnote: 
LTV-based, tens of millions generated
AT&T
MMA Member Badge
The Hypothesis

AI decisioning that pairs creative messages to audience/context in real time will materially beat both randomized creative rotation and BAU rules-based optimization on buy-flow, sustain performance (reduced wear-out), and translate into incremental sales.

This application of AI/ML works by identifying cohorts and optimizing pre-approved elements and creative to each. MMA's approach uses two ML techniques combined with unsupervised learning:

  • One-Hot Encoding
  • K-Modes Clustering
Experiment Design

3-cell RCT: AI-On vs Random (AI-Off) vs BAU (Google)

Creative: 112 variants across 4 sizes, 3 audiences

Allocation: 65/35 earn-learn, 4-week flight

Primary KPI: Buy-flow conversions, sales matchback

112
Total Versions
× 1,852,200
Audience variations
✓ DMA ✓ Time ✓ Day of Week ✓ Device OS ✓ Connection Type ✓ Segment ✓ Other Data Signals
= 207,446,400
opportunities to find value in the interactions between message & audience

 

AT&T: Ad variations

The Impact

AI-On delivered +272% vs Random and +92% vs BAU on buy-flow; sales matchback showed +88% lift and a strong ROI (~$15 returned per $1 incremental AI cost). Lift improved week-over-week with no creative wear-out observed, exceeding 500% by week 4 vs Random.

Key Learnings
  • First CAP study to link web optimization directly to verified sales
  • AI dynamically found & scaled the winning message ("Upgrade Early"; >80% of impressions)
  • No wear-out; lift climbed continuously through the flight
  • Scaling is mostly org/process change, not tech. Points to a headless marketing future