The Modern Marketing Blog covers the latest in marketing strategy, technology, and innovation.

How & Why to Test During the Holiday Season

Brands are hesitant to test during the holiday season. Code freezes and other organizational priorities take effect, while testing initiatives are left to collect dust until the New Year. It is an organization’s appetite for examination that determines their approach to testing during the holidays. But only those who choose to carry on testing at this critical time stand to gain a unique set of insights.

Are you afraid to risk testing inferior experiences that may lead to lower conversion rates or average order values during the holiday season? Remember, the holiday season is only one piece of the larger customer journey. The testing initiatives you take now provide insights that will help you improve the customer experience year-round.


We understand holiday testing is not right for everyone. But it’s still important to take a look at what you stand to gain & how to mitigate risk during this heavily trafficked time of year.

The Pros

  • Increased traffic means that tests can be run for shorter periods of time, allowing for more testing in less time.
  • Increased traffic also means that you can find quick answers to simple questions, often using your WYSIWYG tool.
  • Running a 1-week A/B test at the start of the holiday season offers insights that lead to increased conversion rates you can leverage for the duration of the season.

How to Mitigate Risk


  • Thanks to increased traffic and conversion rates, you can determine quickly that your variant experience is not working and remove it as soon as it is revealed to be statistically negative.
  • Throttling traffic is a way of sending 10% or 20% of site traffic into a campaign to get an idea if the variant is performing on par or better than the default before opening up the test to all site traffic.
  • There’s also dynamic optimization which can provide additional uplift duringtesting by increasing the traffic to experiences that perform well, and decreasing the weights of those that do not.

The decision to test during the holidays is different for every organization. Before you decide (as with any thoughtful testing strategy) find how what you stand to gain, how to be prepared for adjustments, and what you hope to learn.

Happy testing!

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.