A/B/n Split Test Case Study
SmileyCookie.com
Presented by: Jeremy Weis | Senior Conversion Analyst | Trinity Insight

About Smiley Cookie
SmileyCookie.com started as a half-baked idea more than 20 years ago — a smiley cookie as the vehicle to communicate the ethos of a restaurant — Eat'n Park, the Place for Smiles. Soon their cookie became not just the signifier of a delightful family dining experience, but a symbol of commitment to community and charitable giving, indeed, the mascot of the company. (Who knew a six-foot walking sugar cookie would have such draw. You have to be cracking a smile by now!)

Today, every child that dines at Eat'n Park receives the surprise of a buttery, rich, delicious smiley cookie, totaling more than 1 million free cookie treats each year. Every day, their cookies are distributed to community, charity and fundraising events. Every year, more than 200,000 smiling cookies are donated to local non-profit organizations. That's a lot to smile about.

SmileyCookie.com is a convenient online cookie company that makes it easy for you to order cookies online, share a smile with anyone, anywhere, for any occasion. You can even order customized cookies to match wedding colors, sport teams, corporate logos and more, and have your cookies delivered fresh nearly anywhere to as many addresses as you choose! And with their smart cookie pricing, you don't need to spend a lot of dough! But the one thing that that remains the same no matter how the cookie crumbles? Their signature, friendly smile — the frosting on the cake (or cookie) so speak.

The Challenge
The challenge was to evaluate the opportunity that brand value messaging, placed within an existing header location, presented and determine if speaking to any particular value proposition would resonate with the site's customers. This particular area had been used in the past to feature seasonal and promotional messaging, so the variations chosen were strictly limited to messaging surrounding the core values of the site to avoid any temporary behavior that short-term promotions might generate.

The Variants
Trinity Insight developed five creative banners referencing five different values to the customer that the site offers. Those five creatives contained the following messaging and 100% of the site traffic was split up equally among them:

• Order Today -> Ships Next Business Day
• Want to save $5 OFF your next purchase? SIGN UP NOW ->
• $6.99 Ground Shipping For Your Entire Order
• FREE SHIPPING on any order over $40
• Cookies Made Fresh & Hand Iced For You!

The Solution
An A/B/n Split Test was implemented to display these banners to SmileyCookie.com customers. During the time that this test was running, the overall conversion rate for the site was 8.98% with an average order value was $29.85, yielding a $2.68/per visit value. Over this same period of time, the winning creative, Order Today -> Ships Next Business Day, had a 12.61% conversion rate and a $29.95 average order value resulting in a $3.78/per visit value, a 41% increase over the site average.

Prior to testing, SmileyCookie could only make educated guesses as to which core value resonated with their customers and an opportunity to generate an additional $1.10/per visitor would have never been realized. By continuing to look at ways to make the site more engaging to their customers and testing these opportunities, SmileyCookie is guaranteed to create an experience that's better today than it was yesterday as well as learn great lessons along the way about what their customers truly find value in while visiting SmileyCookie.com.

Disclaimer: No intervention from Smiley Cookie's internal IT resources were needed to execute this test and most others conducted on SmileyCookie.com as a result of working with Trinity Insight's custom conversion optimization team.

About Trinity Insight
Trinity Insight has helped numerous retailers, brands, and enterprises drive more profitable website traffic, improve conversion rates, and lower eCommerce costs.

With over 10 years of eCommerce consulting and internet marketing experience, we have worked with market leaders in the travel, retail, manufacturing, and financial industries, and helped them exceed online growth expectations.

Trinity Insight is a partner and authorized consultant of Visual Website Optimizer (VWO) - a leading testing tool that has been constructed for enterprise level businesses. Before choosing VWO, we evaluated numerous testing platforms across a variety of aspects including reliability, scalability, ease of use, and technical integration.

Based upon our evaluation, we feel that the scripting technology leveraged by VWO, in conjunction with its simplistic and user friendly interface, provide the best software option for testing.

Trinity has been an authorized consultant of VWO since 2010 and with that distinction we receive priority technical support as well as visibility into new developments with the tool that we can incorporate into our client testing efforts.

In conjunction with VWO, we are also an authorized partner of Google Analytics and heavily use the two in tandem within conversion engagements.

Contact Us
Trinity Insight | 32 North Front Street | Philadelphia PA | 19106
This e-mail address is being protected from spambots. You need JavaScript enabled to view it | 866.535.8807