This blog post contains spoilers for “Avengers: Infinity War.”
A big part of my job here at Civis is to test the effectiveness of advertising. I often find myself hypothesizing about advertisements I encounter in my day-to-day life watching TV or scrolling through Instagram. A few weeks ago, I saw an ad online reminding me to pre-order tickets to “Avengers: Endgame.” I texted my favorite movie companion (my fiancé MJ) and told him I would buy tickets. I didn’t even ask if he wanted to see it, I already knew. We’ve both seen every Marvel movie, some of them multiple times, and MJ just finished rewatching every Marvel Cinematic Universe (MCU) movie in order.
This got me wondering: for folks who’ve already seen Marvel movies like me and MJ, does Marvel even need to advertise? In my mind, I’ve already converted. “Thor Ragnarok,” “Black Panther,” “Captain America: Winter Soldier,” and “Captain Marvel” (to name my favorites) were all the branded messaging I needed to get super hype about seeing “Endgame!”
Because I work at Civis, I didn’t need to leave these questions unanswered. So, I generated some testable hypotheses:
I picked three videos from the Marvel YouTube page: Honor TV Spot, Big Game TV Spot, and the most recent full theatrical trailer.
Then I turned to Creative Focus, Civis’s application to quickly test text, video or display advertising with methodological rigor. It took me about five minutes to set up my test with our user-friendly application and get my test out into the field. From there, Creative Focus kicks off a randomized control trial survey (which we data nerds consider the most scientific), where a control group sees no trailer, and, in my case, three treatment groups see one trailer each. The system runs statistical models on the dataset, which shows the average treatment effects for the population (all US adults in this case) and specific subgroups (I was particularly interested in whether or not someone has already seen a Marvel movie).
Spoiler Alert: Much like Tony Stark in “Avengers: Age of Ultron,” I was very wrong. I love when science dispels bad assumptions, especially when they’re my own! Let’s go hypothesis by hypothesis and see just how wrong I was.
All of the trailers tested increased awareness, though the Honor spot was least effective. The Big Game spot increased awareness by five percentage points and has near-zero probability of backlash (i.e. which would make someone less likely to see the movie). The full trailer did not perform best, which is atypical in our experience — usually the longer the video, the more persuasive it is.
This hypothesis is partially correct. Those who have already seen a Marvel movie have very high baseline intent to see “Endgame” — more than 75% of respondents indicated they intend to see the film. However, the trailers tested do have treatment effects, with the full trailer performing best with an average treatment effect of five percentage points. This means if Marvel showed a room of 100 past Marvel film viewers this trailer, five more viewers would show up and see “Endgame” in theaters. Interestingly, the Honor spot is such a bummer it actually had an average negative persuasion effect. Possibly because it reminded Marvel fans that a bunch of their favorite heroes got finger-snapped into dust. (JUSTICE FOR SHURI!!!)
We see that yes, those who have not seen a Marvel movie in the past are at a much lower baseline intent to view than Marvel viewers (27% vs 75%+), but these respondents are still persuaded by the promos! The full trailer and Big Game spot are both positively persuasive, with treatment effects of 6% and 4%, respectively. Again we see Honor with negative average effects.
Turns out Disney and Marvel might know what they’re doing in advertising “Endgame.” I should have guessed, given that the MCU has seven movies that have made over a billion dollars, and “Avengers: Endgame” has already broken records for advanced ticket sales!
If you’re curious about Creative Focus or want to learn more about our research capabilities, feel free to get in touch.
