︎
Suzanne Chong
Founder and Product Designer
San Francisco Bay Area

LinkedIn
Email
        Back

Alexa Multimodal Search



Introduction


In March 2022, Alexa is now more effective in helping customers explore and shop for products on Echo Show devices – whether they have a specific product in mind, looking to find one with a certain feature, or simply exploring. Most of the products come in various sizes, with different features and from many brands. It is not always easy for customers to find products that are right for them or know how to go about finding one that they are looking for. Alexa now helps customers become familiar with the popular product options available and select ones that are closer to their needs.


Customer FAQs


How do I get this experience?
You will receive this experience when you ask Alexa on your Echo Show device to find you a product (e.g. you say ‘Alexa, show me shampoos’). As you explore products you can refine your search using natural language (e.g. you can say ‘Alexa show Pantene’). Alexa will suggest popular product options (e.g. ‘Pantene’, ‘Oily hair’, ‘Frizz Control’) to help you learn about the product space if you are unfamiliar. In addition she enables you to browse by asking her to show you products by top brands, features, prices etc.

How do I ask for products by top brands?
Alexa now helps you browse by offering to organize results by top brands, features etc. When you search for a product, Alexa presents the most popular aspects as suggestions among the results on the screen (e.g. ‘To shop by price, say “Alexa show top prices”’). Alexa will pick the most popular brands available in that product range and will present top recommendations for each, for you to choose from.

Can I ask Alexa to show me products with multiple features e.g. ‘Show me Pantene for oily hair’
Yes, Alexa will be able to recognize that you are requesting shampoos for a specific brand e.g. in this case ‘Pantene’ and for specific hair type such as ‘oily hair’. She will the find the most relevant shampoos with those characteristics.

What if I am unsure where to start my search?
If you are unfamiliar with the product space, Alexa will provide you with suggestions of popular options in her response (e.g. ‘Try saying something like ‘Show Pantene only’). She will also present the top options (such as Sony, Bose, JBL) of a popular aspect (such as brand) on the bottom of the screen to help you with your exploration. You can ask Alexa at any time ‘show more options’ to see a bigger set of popular suggestions. Alexa picks these based on her knowledge of the products as well as taking into account shopping patterns of millions of Amazon customers.


Problem Statements


  • CUSTOMERS CAN’T FILTER PRODUCTS WITHOUT NEEDING TO DO A FULL SEARCH AGAIN. 
    For instance, customers were searching for headphones and they want to search for a specific product category something like: wireless earphones. They need to do a full search again by saying: Alexa, show wireless earphones. This creates inefficient navigation and time consuming product search process.

  • CUSTOMERS CAN’T EXPLORE BASED ON THEIR PREFERENCES.
    Alexa doesn’t let customers to browse based on their top brands, price range, key features, and etc. We need to make it more flexible for customers to find products for their needs.

  • CUSTOMERS NEED ALEXA’S PRODUCT EXPERTISE WHEN THEY ARE UNSURE WHERE TO START EXPLORING.
    Interacting by voice and touch, Alexa could become an expert to provide what customers are looking for and guide them to move along in their shopping mission. The depth of Alexa’s knowledge and refinement guidance will help them to decide more thoughtfully why they need to buy that one product.

  • CUSTOMERS FIND IT DIFFICULT TO COMPARE AND MAKE PURCHASE DECISIONS BETWEEN PRODUCTS.
    Customers may have an idea of what they want, but they may not necessarily be able to express that in the form of product features. They also have to know which attributes matter in terms of assessing the quality of the products. By picking specific attributes, customers would risk excluding some choices that might be right for them.


Scenario 1: Jane, a price conscious buyer.


Jane is a mother of two young children, recently broke her headphones and is looking to find a replacement. She is budget conscious and wants to find a good set within her budget. While having a breakfast in the morning, Jane remembers that she needs to order new headphones.


VUI Framework

Customer: Alexa show me [Product Name].
Alexa: Here are top [Product Name] from Amazon. They come in different [n1 Facet Type] and [n2 Facet Type], and also various [n3 Facet Type] with [Category Examples]. You can say things like: Alexa, show me [n1 Searched Product Category] only.

C: Alexa, show me [n1 Category] [Product Name] / only [n1 Category].
A: Ok, the first one is [ASIN Title]. It’s [Price]. If you like this, try saying “Alexa, show me        number X”. To start a new search, just say "Alexa, start over".

    C: Alexa, show me [n2 Category] [Product Name] / only [n2 Category].
    A: Ok, the first one is [ASIN Title]. It’s [Price]. If you like this, try saying “Alexa, show me     number X”. To start a new search, just say "Alexa, start over".


Illustrative VUI

Jane: Alexa, show me headphones.
Alexa: Here are top headphones from Amazon. They come in different brands and price points, and also various features like wireless or over-ear. You can say things like: Alexa, find a wireless headphone number $100. If you prefere certain brands, I’ve love to know                  too. Go ahead and ell me what you have in mind.

Jane: Alexa, find a noise-cancelling headphone under $100
Alexa: Ok, the first one is Anker Soundcore Life noise-cancelling headphones. It's $49.99 and Amazon’s Choice. If you like this, try saying “Alexa, show me number 1”. To start a new search, just say "Alexa, start over".
      

Illustrative GUI



Scenario 2: Bryan, a customer looking to explore


Bryan is a young professional in his late 20s and wants to replace his old headphones. He does not have any specific brand or type in mind and is looking to explore with Alexa’s help.

Illustrative VUI

Bryan: Alexa, show me traveling headphones
Alexa: Ok, here are some travel headphone recommendations from web. You can ask me other options like wireless, noise-cancelling, and more. Try saying something like “Alexa, show me wireless headphones”.

Bryan: Alexa, show me wireless only/wireless headphones/only wireless.
Alexa: Ok, the first one is Apple AirPods Pro with Charging Case. It's $197.98. If you like this, try saying “Alexa, show me number 1”. To start a new search, just say "Alexa, start over".

Illustrative GUI


     


UI Frameworks


Contextual Pills: We will be extracting popular facets based on Amazon customer’s shopping behavior patterns on the web and will use contextual pills to surface the most relevant refinements and category pills for quick guidance and interactions.

Utterance Cards: The utterance cards will help the customers know what they can say to organize and refine the results. They will appear as the first turn to educate customers what to say to Alexa to refine or reformulate their shopping search.

Facet Tiles: These help customers to reformulate their search queries by exploring new popular things to consider. Customers can easily compare which tile meets their needs the most.They can simply tap each tiles to carry on their shopping journey. These tiles are flexible to suggest as QnA, topics, or categories to inspirerelevant product options.




Other product categories


Here are the most common and searched facets from Amazon.com and this CX should work for more products other than just ‘Headphones’. We should be mindful that there could be unique edge cases per each product categories, but it would be worth to expand more product categories with the same VUI and GUI frameworks to experiment with.

Toy (brand, price, charater, and category)

  • Alexa, show me LEGO only
  • Alexa, show me under $50
  • Alexa, show me Star Wars only
  • Alexa, show me action figures under $50

Food (brand, price, and nutrition facts)

  • Alexa, show me gluten-free only
  • Alexa, show me Kellogg’s only
  • Alexa, show me under $30 only

Medicines (department, price, and care-format)

  • Alexa, show me women’s vitamins only
  • Alexa, show me under $50 only
  • Alexa, show me drops only

Home Decor (category, price, and color)

  • Alexa, show me candles only
  • Alexa, show me under $100 only
  • Alexa, show me blue onese only