Creative Testing Under Andromeda: A System for Finding the 5% That Wins (2026)

most creative testing advice is a structure diagram: a testing campaign here, a scaling campaign there, arrows in between. the structure is not the problem. the problem is that testing is a numbers game with a 5% hit rate, most teams read results before there is a result, and the platforms' own "winner" labels are wrong a quarter of the time. this guide is the system for the actual math.

---

what creative testing actually is

creative testing is a search for the rare ad that the delivery system can scale. motion's 2026 creative benchmarks, built on $1.29 billion of meta spend, 578,750 creatives, and 6,015 accounts between september 1, 2025 and january 1, 2026, put the number at "~5%" of creatives qualifying as winners, defined as at least 10x the account's median creative spend and at least $500, and found that "55%" of total spend concentrates on those winners. the concentration rises with scale: 23% of spend on winners in micro accounts, 64% in enterprise accounts, which launch 18.8 creatives a week (motion, creative benchmarks 2026).

here is what most guides get wrong: they treat testing as choosing between ads you already made. under andromeda, creative is what the retrieval engine reads to decide who sees the ad (see the andromeda guide), so testing is really generating enough genuinely different inputs that the system can find one it can scale, and then not killing it before it has had the chance. the structure is secondary. the launch rate, the diversity, the spend floor, and the read date are the system.

---

the fundamental shift: from picking winners to feeding the model

before: six ads per ad set, dynamic creative to mix elements, a testing campaign to isolate each ad's roas, and a decision at day three.

after: meta describes creative diversification as "creating a wide range of ad creatives with different themes, messages, and visuals to cater to diverse audience segments," and says the increase in volume and diversity "also means our system must process more information when determining who to show an ad to" (meta for business, the creative advantage, april 22, 2025). its december 2025 follow-up says to "develop ads that are truly different" and "build a broad portfolio of assets" (meta for business, demystifying creative diversification, december 16, 2025). dynamic creative was withdrawn for sales and app objectives from june 2024 in favor of the flexible ad format (meta business help center, about dynamic creative).

tiktok says the same thing in fewer words: "we suggest between 3-5 different creatives per ad group and 3-5 diversified ad groups per campaign," refresh "when delivery results exhibit a consistently declining trend," and "add new creatives to an existing ad group instead of creating a new ad group to extend its lifetime" (tiktok ads manager, creative best practices).

in plain english: the platforms want a steady supply of different ads inside a stable structure. the job of testing is to supply them and to know, honestly, which ones worked.

---

what this broke in existing accounts

1. testing campaigns that starved the ads they were testing

isolating each new ad in its own ad set with a small budget guarantees it never reaches the spend where a winner can show itself. motion's threshold for a winner is 10x the median creative spend. an ad capped at one-tenth of the scaling ad set's budget cannot get there by design.

2. decisions at day three

andromeda and the ranking models optimize sequences over days, and the learning phase needs about 50 optimization events in 7 days. a creative judged on 48 hours of delivery is judged on the model's first guess about it.

3. variations counted as tests

twenty headline swaps on one video are one creative to andromeda's entity id grouping. teams that "tested 20 ads" and found nothing tested one ad twenty times.

4. winners called at 65% confidence

meta's a/b test tool calls a winner at "a 65 percent or higher confidence percentage" and only treats results above 75 as "very clear and actionable" (meta business help center, about confidence in your tests). berman and van den bulte analyzed 4,964 effects from 2,766 real a/b experiments and found false discovery rates of 28% to 37% at the 10% significance level and 18% to 25% at 5%, mostly because about 70% of tested changes have no true effect (berman and van den bulte, false discovery in a/b testing, management science, 2022). at 65% confidence the false discovery rate is worse than either. a testing program that promotes 65% "winners" is promoting noise roughly half the time.

5. judging on roas at volumes where roas is random

an ad with 6 purchases has a roas, and the roas means nothing. teams that skip the upstream metrics (hook, hold, click) have no way to read a creative until it has spent enough to be judged on conversions, and by then it has often been paused.

6. launch rates that cannot produce winners

at a 5% hit rate, four launches a month yields a winner every five months. accounts with one designer and no ugc pipeline are not testing; they are hoping.

7. leftover dynamic creative setups

ad sets still running dynamic creative on objectives where it was withdrawn, or flexible format used as a dumping ground for every asset, producing combinations nobody chose and nobody can read.

---

the data

• motion, $1.29b meta spend, 578,750 creatives, 6,015 accounts, september 2025 to january 2026: about 5% of creatives are winners (10x account median spend, at least $500), sitting at roughly the 92nd percentile; 55% of spend goes to winners; concentration rises from 23% (micro) to 64% (enterprise); enterprise accounts launch 18.8 creatives per week. (motion, creative benchmarks 2026)
• meta, april 2025: andromeda "led to an 8% improvement in ads quality"; advantage+ sales campaigns show "9% lower cost per action"; advantage+ creative image generation achieves "11% higher click-through-rate and 7.6% higher conversion rate"; one cited advertiser scaled "from 3-4 new creatives a week to almost 50 on average." (meta, the creative advantage)
• meta, december 2025: text generation "3% higher ctr"; advantage+ sales "22% increase in roas." self-reported. (meta, demystifying creative diversification)
• meta a/b tests: winner at 65% confidence, "very clear" above 75, decided on cost per result by simulating outcomes tens of thousands of times. (meta, about confidence in your tests)
• berman and van den bulte, 2022: 2,766 experiments, 4,964 effects; false discovery 28% to 37% at 10% significance, 18% to 25% at 5%; about 70% of tested effects are true nulls; "one in five interventions achieving significance at 5% confidence" should be expected to be ineffective. (management science)
• tiktok: 3 to 5 creatives per ad group, 3 to 5 ad groups per campaign, hook in the first 6 seconds, proposition in the first 3, refresh into the existing ad group when results decline. (tiktok, creative best practices)
• confect, 115.7 billion impressions across 3,014 advertisers in 2025: top-third advertisers lost 31% roas after andromeda; creative diversity was the adaptation that recovered it. (see the andromeda guide)

---

what changed in 2026 specifically

meta's stance is explicit. two meta for business articles in 2025 define creative diversification and tell advertisers to build a broad portfolio of genuinely different ads. this is no longer practitioner folklore; it is the platform's stated operating model for andromeda.

dynamic creative is gone for performance objectives. flexible format is the replacement, and it behaves differently: you supply media and meta chooses which to show per impression. it is a delivery feature, not a test.

ai creative is a volume tool, not a diversity tool. meta's image and text generation lift numbers apply to variations of an existing ad. andromeda's entity grouping still treats a generated background swap as the same creative. generation raises launch rate; it does not by itself raise the number of concepts.

tiktok smart creative. pauses fatigued videos within days and rotates in replacements from a waiting list. useful if you supply the list; it does not make the list.

unchanged: the 5% hit rate, the need for a spend floor before a read, and the false discovery math.

---

what actually works now

1. separate concepts from variations, and count only concepts as tests

a concept changes the angle (problem, proof, offer, demo, comparison), the format (video, static, carousel, ugc, studio), or the messenger (founder, customer, creator, no person). a variation keeps the concept and changes one element: the hook line, the first three seconds, the aspect ratio, the caption. tests are concepts. variations are how you iterate a concept that already won. a testing report that lists variations as tests is lying to itself.

2. set a launch rate from the hit rate

if 5% win, one winner a month needs about 20 concept launches a month. one winner a week needs about 20 a week, which is enterprise territory. decide how many winners the account needs to keep scaling, multiply by 20, and staff the pipeline (in-house, creators, ugc marketplaces, ai variations of winners) to hit it. an account that cannot hit 10 concepts a month should fix the pipeline before touching the structure.

3. test inside a stable structure

under roughly $50,000 a month: add 3 to 5 new concepts at a time into the main scaling ad set, the way tiktok recommends, and let delivery allocate. above that: one testing campaign with campaign budget and a minimum spend per ad set so each concept is guaranteed a floor, plus the scaling ad set that receives promoted winners. either way, one campaign per objective, few ad sets, many ads. do not build a new ad set per creative.

4. spend floor, then seven days, then read

do not read a concept until it has spent at least 2x the target cpa and run at least 7 days. before the floor, the only permitted action is pausing an ad with a policy problem. this rule removes the day-three kill that destroys most testing programs.

5. read up the ladder, not down from roas

• hook: 3-second video views divided by impressions (thumbstop). judge after 1,000 impressions.
• hold: 15-second views divided by 3-second views. judge with the hook.
• click: outbound ctr. judge after the hook and hold.
• conversion: cpa or roas. judge only after the spend floor and 7 days, and against the account's median, not the best ad.

a concept that fails hook and hold is dead before it costs a cpa. a concept with strong hook and hold and weak cpa gets a variation on the landing or offer, not a pause.

6. promote on relative performance and require real confidence

promote a concept to the scaling ad set when its cpa is within 20% of the account median over the read window and it has cleared the spend floor. if you use meta's a/b test tool, require 75% or better, run at least 7 days, and treat a single result as one data point. the false discovery math says a fifth of your 95% winners will not replicate; plan the pipeline so that is survivable.

7. iterate winners hard

the 5% are where the leverage is. every winner gets a variation batch: three new first-three-second hooks, one format flip (static from the video, or video from the static), one messenger swap, and advantage+ creative or flexible format variations on top. this is where ai generation earns its place.

8. refresh into, not instead of

when frequency climbs above 3.5 on prospecting or ctr drops more than 20% week over week, add new concepts to the ad set. do not pause the fatigued winner until the replacements have cleared their own floor. tiktok's guidance is the same: add to the existing ad group.

9. keep a hit rate scorecard

monthly: concepts launched, concepts past the floor, winners, hit rate, spend share on winners. benchmark against 5% and 55%. a hit rate under 3% with adequate launches is a diversity problem. adequate hit rate with too few launches is a pipeline problem. the scorecard tells you which.

---

what to stop doing immediately

• stop counting variations as tests.
• stop reading creatives before 2x target cpa and 7 days.
• stop calling winners at 65% confidence.
• stop building an ad set per creative to "isolate" it.
• stop judging new creatives on roas before hook, hold, and click.
• stop pausing a fatigued winner before its replacements have cleared the floor.
• stop using ai generation as a substitute for new concepts.
• stop running dynamic creative on objectives where it was withdrawn.

---

the orbit agent instructions

{
  "guide": "creative-testing-system-2026",
  "agent_tasks": {
    "audit_concept_vs_variation_mix": {
      "check": "for each active ad set, group creatives by format, angle, and messenger using names, media, and copy, and count distinct concepts versus variations",
      "flag": "fewer than 3 distinct concepts in a scaling ad set, or more than 70 percent of active ads being variations of one concept",
      "recommend": "launch concepts across at least 3 formats and 3 angles before adding more variations"
    },
    "audit_spend_floors_and_read_dates": {
      "check": "for each creative launched in the last 30 days, read days live, spend to date, and the account's target cpa, and list any creative paused before reaching 2x target cpa or 7 days",
      "flag": "any creative paused before both thresholds without a policy reason, or a testing ad set whose minimum spend cannot fund 2x target cpa per creative within 7 days",
      "recommend": "restore early-paused creatives where budget allows and set minimum spend so each concept reaches the floor"
    },
    "read_metrics_ladder": {
      "check": "for each creative past 1,000 impressions, compute thumbstop rate, hold rate, outbound ctr, and after the spend floor, cpa or roas versus the account median",
      "flag": "creatives judged on cpa before the floor, or creatives with hook and hold above the account median but cpa more than 40 percent worse than median",
      "recommend": "kill on hook and hold failures early, hold cpa judgments until the floor, and route strong-hook weak-cpa creatives to landing page or offer variations"
    },
    "audit_ab_test_confidence": {
      "check": "list a/b tests concluded in the last 90 days with their confidence percentage, duration, and result count per cell",
      "flag": "any test acted on below 75 percent confidence or under 7 days, or with fewer than 50 results per cell",
      "recommend": "re-run or extend the test and treat single results as one data point in the scorecard"
    },
    "compute_hit_rate_and_launch_cadence": {
      "check": "monthly, count concepts launched, concepts that cleared the floor, winners at 10x the account median creative spend, spend share on winners, and the resulting hit rate",
      "flag": "fewer than 10 concept launches in a month, hit rate below 3 percent with adequate launches, or spend share on winners below 40 percent",
      "recommend": "if launches are low, expand the creative pipeline; if hit rate is low, increase concept diversity; if winner share is low, promote cleared winners into the scaling ad set"
    },
    "monitor_fatigue_and_refresh": {
      "check": "weekly per scaling ad set, read frequency, week-over-week ctr change per creative, and the number of new concepts added in the last 14 days",
      "flag": "frequency above 3.5 or ctr down more than 20 percent week over week with no new concepts added in 14 days, or a fatigued winner paused before replacements cleared the floor",
      "recommend": "add 3 to 5 new concepts into the ad set, keep the winner live until replacements clear the floor, and schedule the next refresh"
    }
  }
}

---

the bottom line

about one ad in twenty wins and it takes half the money. build the pipeline to launch enough concepts, give each one a spend floor and seven days, read hook and hold before roas, demand real confidence, and iterate the winners hard.

the structure diagram was never the system. the launch rate, the floor, and the read date are.

---

built by orbit - the performance marketing intelligence layer. questions? jose@orbitllm.com