Test and learn

Last updated

Test and learn is a set of practices followed by retailers, banks and other consumer-focused companies to test ideas in a small number of locations or customers to predict impact. The process is often designed to answer three questions about any tested program before rollout:

Contents

  1. What impact will the program have on key performance indicators if executed across the network or customer base?
  2. Will the program have a larger impact on some stores/customers than others?
  3. Which components of the idea are actually working?

Historical context

Test and learn has been systematically applied as far back as 1988 by Capital One. Capital One has been aggressive about testing since the firm was founded, testing everything from product design to marketing to customer selection to collection policies. [1] In a single year, the company performs tens of thousands of tests, allowing it to offer thousands of different types of credit cards to customers, based on the knowledge gained from their tests. Richard Fairbank, CEO of Capital One, called test and learn "a marketing revolution that can be applied to many businesses". [2]

Examples

Test and learn in practice

Test and learn software

There are a variety of software tools available today to support systematic testing within an organization. However, there is currently no one software package that covers all types of tests, and in some cases significant knowledge of statistics is still required for effective analysis. [1]

Many companies, including Capital One and eBay, have developed experiment management software within the company. Alternatives to internal solutions include broad statistical analysis software tools such as SAS or test and learn focused software from Applied Predictive Technologies, Trial Run and MarketDial. There are also a variety of more narrowly tailored products, focused on particular problems or fields. In web analytics, for example, firms such as Omniture, WebTrends, and Google have created specialized software. [1]

Criticism

Companies wanting to conduct Test and Learn must mobilize, track and analyze site and customer data at very granular levels. This can often be a massive and expensive undertaking. Increasingly however, an exponential increase in computing power, and corresponding decrease in its cost, have made testing more accessible. In the Harvard Business Review February 2009 article "How to Design Smart Business Experiments", [1] Thomas Davenport discusses the Test and Learn Process, and describes how several companies have overcome these and other barriers to testing.

See also

Related Research Articles

Tandy Corporation was an American family-owned leather-goods company based in Fort Worth, Texas, United States. Tandy Leather was founded in 1919 as a leather supply store. By the end of the 1950s, under the tutelage of then-CEO Charles Tandy, the company expanded into the hobby market, making leather moccasins and coin purses, making huge sales among Scouts, leading to a fast growth in sales.

<span class="mw-page-title-main">Supermarket</span> Large format of grocery store

A supermarket is a self-service shop offering a wide variety of food, beverages and household products, organized into sections. This kind of store is larger and has a wider selection than earlier grocery stores, but is smaller and more limited in the range of merchandise than a hypermarket or big-box market. In everyday United States usage, however, "grocery store" is often used to mean "supermarket".

<span class="mw-page-title-main">Amazon (company)</span> American multinational technology company

Amazon.com, Inc., doing business as Amazon, is an American multinational technology company, engaged in e-commerce, cloud computing, online advertising, digital streaming, and artificial intelligence. It is considered one of the Big Five American technology companies; the other four are Alphabet, Apple, Meta, and Microsoft.

<span class="mw-page-title-main">Retail</span> Sale of goods and services

Retail is the sale of goods and services to consumers, in contrast to wholesaling, which is sale to business or institutional customers. A retailer purchases goods in large quantities from manufacturers, directly or through a wholesaler, and then sells in smaller quantities to consumers for a profit. Retailers are the final link in the supply chain from producers to consumers.

<span class="mw-page-title-main">Best Buy</span> American multinational consumer electronics retailer

Best Buy Co., Inc. is an American multinational consumer electronics retailer headquartered in Richfield, Minnesota. Originally founded by Richard M. Schulze and James Wheeler in 1966 as an audio specialty store called Sound of Music, it was rebranded under its current name with an emphasis on consumer electronics in 1983.

<span class="mw-page-title-main">Iceland (supermarket)</span> British supermarket chain

Iceland Foods Limited, trading as Iceland, is a British supermarket chain headquartered in Deeside, Wales. It mainly sells frozen foods, including prepared meals and vegetables, alongside non-frozen grocery items such as produce, meat, dairy and dry goods. The company also operates a chain of shops called The Food Warehouse.

<span class="mw-page-title-main">Woolworths Group (Australia)</span> Australian multinational retail company

Woolworths Group Limited is an Australian multinational retail and finance company, primarily known for the operation of its retail chain Woolworths Supermarkets across Australia, Woolworths in New Zealand and its discount department store Big W. Headquartered in Bella Vista, Sydney, it is the largest company in Australia by revenue and number of employees, and the second-largest in New Zealand.

<span class="mw-page-title-main">Online shopping</span> Form of electronic commerce

Online shopping is a form of electronic commerce which allows consumers to directly buy goods or services from a seller over the Internet using a web browser or a mobile app. Consumers find a product of interest by visiting the website of the retailer directly or by searching among alternative vendors using a shopping search engine, which displays the same product's availability and pricing at different e-retailers. As of 2020, customers can shop online using a range of different computers and devices, including desktop computers, laptops, tablet computers and smartphones.

<span class="mw-page-title-main">Sheetz</span> American retail chain

Sheetz, Inc. is an American chain of convenience stores and coffee shops owned by the Sheetz family. The stores sell custom food, beverages and convenience store items, with all locations having offered 24/7 service since the 1980s. Nearly all of them sell gasoline; a few locations are full-scale truck stops, including showers and a laundromat. Sheetz's headquarters is in Altoona, Pennsylvania, with their corporate offices located there as well, with over 700 stores located in Central and Western Pennsylvania, West Virginia, Maryland, Ohio, Virginia, and North Carolina, with plans to expand into Michigan.

<span class="mw-page-title-main">Store-within-a-store</span> Retail concept

A store-within-a-store, also referred to as store-in-store or shop-in-shop, refers to a space within a larger retail store, designated for use by a specific brand to feature its products, clearly branded with signs and other branding elements like color, materials, layout, etc. Such a space may be a section of the main area of the store, or it may have the form of an enclosed store with "walls" an entrance, much like a store in a shopping mall.

<span class="mw-page-title-main">Woolworths Supermarkets</span> Australian multinational supermarket chain owned by Woolworths Group

Woolworths Supermarkets is an Australian chain of supermarkets and grocery stores owned by Woolworths Group. Founded in 1924, Woolworths today is Australia's biggest supermarket chain with a market share of 33% as of 2019.

<span class="mw-page-title-main">Brick and mortar</span> Class of organisations or businesses

Brick and mortar is an organization or business with a physical presence in a building or other structure. The term brick-and-mortar business is often used to refer to a company that possesses or leases retail shops, factory production facilities, or warehouses for its operations. More specifically, in the jargon of e-commerce businesses in the 2000s, brick-and-mortar businesses are companies that have a physical presence and offer face-to-face customer experiences.

<span class="mw-page-title-main">Woolworths South Africa</span> South African retail company

Woolworths Holdings Limited is a South African multinational retail company that owns Woolworths, a South African luxury department store chain, and Australian retailer Country Road Group. Woolworths, however, has no association to Australia's Woolworths supermarket chain.

<span class="mw-page-title-main">Retail marketing</span>

Once the strategic plan is in place, retail managers turn to the more managerial aspects of planning. A retail mix is devised for the purpose of coordinating day-to-day tactical decisions. The retail marketing mix typically consists of six broad decision layers including product decisions, place decisions, promotion, price, personnel and presentation. The retail mix is loosely based on the marketing mix, but has been expanded and modified in line with the unique needs of the retail context. A number of scholars have argued for an expanded marketing, mix with the inclusion of two new Ps, namely, Personnel and Presentation since these contribute to the customer's unique retail experience and are the principal basis for retail differentiation. Yet other scholars argue that the Retail Format should be included. The modified retail marketing mix that is most commonly cited in textbooks is often called the 6 Ps of retailing.

<span class="mw-page-title-main">Visual merchandising</span> Marketing technique emphasizing 3D model displays

Visual merchandising is the practice in the retail industry of optimizing the presentation of products and services to better highlight their features and benefits. The purpose of such visual merchandising is to attract, engage, and motivate the customer towards making a purchase.

Customer analytics is a process by which data from customer behavior is used to help make key business decisions via market segmentation and predictive analytics. This information is used by businesses for direct marketing, site selection, and customer relationship management. Marketing provides services to satisfy customers. With that in mind, the productive system is considered from its beginning at the production level, to the end of the cycle at the consumer. Customer analytics plays an important role in the prediction of customer behavior.

<span class="mw-page-title-main">Hudsons Coffee</span> Australian coffee retailer chain

Hudsons Coffee is an Australian chain of coffee retailers. As of August 2013, it comprises 67 stores across Australia, as well as one store at Changi Airport in Singapore. It is owned by the Emirates Group which includes Emirates Airline, the Costa Coffee franchise in the UAE, and Left Bank lounge bar and restaurants in Dubai, Abu Dhabi, Oman and Southbank in Melbourne.

<span class="mw-page-title-main">99p Stores</span> British discount store chain

99p Stores Ltd. was a family-run business founded in January 2001 by entrepreneur Nadir Lalani, who opened the first store in the chain in Holloway, London, with a further three stores opening later that year. In 2002, Lalani decided to expand the business throughout the UK and had rapidly developed 99p Stores, operating a total of 129 stores as of March 2010 and serving around 1.5 million customers each week, undercutting their main rival Poundland by a penny. As of mid-2009 the company offered more than 3,500 different product lines throughout its stores.

Night 'n Day is a chain of New Zealand grocery stores. The stores operate long hours, and sell a range of ready-to-eat products.

References

  1. 1 2 3 4 5 Davenport, Thomas H. (February 2009). "How to Design Smart Business Experiments". Harvard Business Review .
  2. Fishman, Charles (1999-04-30). "This is a Marketing Revolution". Fast Company . Retrieved September 22, 2009.
  3. "Closing the gap: Test and learn".
  4. "Using A/B testing to measure the efficacy of recommendations generated by Amazon Personalize | AWS Machine Learning Blog". aws.amazon.com. 2020-08-20. Retrieved 2023-07-26.
  5. "PFLB Platform - Exploring Test and Learn: A Scientific Examination of Predictive Business Strategies". platform.pflb.us. Retrieved 2023-07-26.
  6. van Es, Karin (2022-09-26). "Netflix & Big Data: The Strategic Ambivalence of an Entertainment Company". Television & New Media: 152747642211257. doi: 10.1177/15274764221125745 . ISSN   1527-4764.
  7. "High-tech secret behind the future of your local Woolworths". 9News. January 23, 2020. Retrieved July 6, 2020.
  8. Fleenor, D. Gail (June 2009). "These Tests Are Positive". STORES Magazine. Archived from the original on 12 October 2009. Retrieved 18 September 2009.
  9. Angrisani, Carol (December 22, 2008). "Coffee Aisle Revamp Lifts Sales". Supermarket News. Retrieved September 22, 2009.
  10. Wong, Elaine (June 24, 2009). "Case Study: How Search Ads Helped Pier 1 Stay Afloat". Brandweek . Retrieved September 22, 2009.