person
Sign in
Request Demo

Gaining New Marketing Insights Through A/B lift tests

Sometimes the best experiments aren't planned. Discover how one telecom provider turned an unexpected social media pause into a natural experiment that revealed critical insights about channel dependencies and media mix effectiveness.

Blog post image

While experiments are often used to validate existing strategies, they’re just as powerful when used to challenge assumptions and uncover new opportunities. At Objective Platform, we help marketers design experiments that move them from gut feeling to grounded insight and from reactive optimisation to proactive strategy.

In this example, a telecom provider faced an unexpected disruption in their media mix and turn edit into an opportunity for learning.

Use Case: Telecom Provider Social Media A/B lifttest

The Hypothesis:

Social media drives meaningful incremental value in the media mix.
That was one client's long-standing internal assumption. Like many brands, they invested consistently in social media advertising but couldn’t clearly quantify its impact.

When unforeseen circumstances in early 2022 resulted in a temporary pause of social media campaigns, they saw a rare opportunity to run a natural experiment. Using Objective Platform, the team compared performance during the pause with historical benchmarks and modelled expectations.

The Experiment Setup

Social campaigns were paused across all products for a defined period. Using Objective Platform, the team compared performance during the pause with historical benchmarks and modelled expectations. They focused on three key products and monitored shifts across all media channels – including Display, Programmatic, SEA and Affiliate.

The Results

The findings varied by product, highlighting how media mix effectiveness is context-dependent:

These results challenged the assumption that social media spend could be cut without consequences. Instead, they revealed that social media was more integral to performance than previously thought, especially for Products B and C.

What This Proved

The experiment proved that assumptions, even long-held ones, are worth testing. Rather than blindly cutting or increasing spend, the provider was able to ground future media planning in data. The nuanced results across products also illustrated that there is no one-size-fits-all approach to channel value.

Why Incrementality Tests Matter in MMM

Objective Platform enables teams to do more than model performance – they can test it in the real world. By building learnings from natural experiments into your MMM, you can:

This kind of insight helps marketers align spend with impact and adapt quickly when market conditions change.

Final Takeaway

The takeaway: Sometimes, disruption becomes your best opportunity to learn. When you use experiments to challenge assumptions and not just validate them, you unlock a deeper level of media intelligence that drives both agility and growth.