AT&T: AI-Personalized Creative Outperformed BAU and Random by a Wide Margin | MMA Global
  2. Resources
  3. Case Studies
  4. AT&T: AI-Personalized Creative Outperformed BAU and Random by a Wide Margin
CAP GenAI personalization
CASE STUDY

AT&T: AI-Personalized Creative Outperformed BAU and Random by a Wide Margin

A four-week, three-cell RCT showed +272% lift vs. random rotation and +92% lift vs. BAU on buy-flow—validated with an 88% sales lift and strong ROI.

Performance Impact

1

The Hypothesis

AI decisioning that pairs creative messages to audience/context in real time will materially beat both randomized creative rotation and BAU rules-based optimization on buy-flow, sustain performance (reduced wear-out), and translate into incremental sales.

This application of AI/ML works by identifying cohorts and optimizing pre-approved elements and creative to each. MMA's approach uses two ML techniques combined with unsupervised learning:

  • One-Hot Encoding
  • K-Modes Clustering
2

Experiment Design

3-cell RCT: AI-On vs Random (AI-Off) vs BAU (Google)

Creative: 112 variants across 4 sizes, 3 audiences

Allocation: 65/35 earn-learn, 4-week flight

Primary KPI: Buy-flow conversions, sales matchback

112
Total Versions		 × 1,852,200
Audience variations
✓ DMA ✓ Time ✓ Day of Week ✓ Device OS ✓ Connection Type ✓ Segment ✓ Other Data Signals		 = 207,446,400
opportunities to find value in the interactions between message & audience

 

AT&T: Ad variations

3

The Impact

AI-On delivered +272% vs Random and +92% vs BAU on buy-flow; sales matchback showed +88% lift and a strong ROI (~$15 returned per $1 incremental AI cost). Lift improved week-over-week with no creative wear-out observed, exceeding 500% by week 4 vs Random.

4

Key Learnings

  • First CAP study to link web optimization directly to verified sales
  • AI dynamically found & scaled the winning message ("Upgrade Early"; >80% of impressions)
  • No wear-out; lift climbed continuously through the flight
  • Scaling is mostly org/process change, not tech. Points to a headless marketing future