CookTrails
CookTrails
CookTrails is a recipe app that can be controlled using gestures in the air.
The goal is to provide users with app functionality while keeping their hands free to interact with other things and environments around them.
SKILLS
Design Research
Sketching
Wireframing
Prototyping
Scene Modeling
Sketching
Style Guide
User Testing
User Flow Diagram
UI/UX Design
TOOLS
Figma
Adobe Dimension
Adobe After Effect
Adobe Premiere
Procreate
LOVO
PROJECT DURATION
Sep 2023 - Nov 2023
The project explores the subtleties of gesture-based interaction, where physical movements serve as a medium for human-computer interaction.
Gestures, often intentional and unintentional, add richness to communication and enhance expressiveness during interactions. The aim is to utilize these natural gestures for digital interfaces, offering smoother interactions with applications beyond standard touch controls.
The key challenge is designing an intuitive gesture interface and language system, enabling users to engage with digital environments as naturally and effortlessly as they would in the real world.
Introduction
When you are learning a new recipe, you often need to refer to the recipe instructions frequently.
However, imagine the typical scenario when you are cooking in the kitchen, your hands may be covered in various food residues, such as sticky dough, greasy sauce, oil, or powders.
When your hands are covered in food residues, touching screens becomes challenging. This make your devices dirty, but it also makes your hands uncomfortable.
Problems
“How might we design a recipe app that allows users to interact without physically touching the device, keeping their cooking experience hygienic and uninterrupted?”
About CookTrails
CookTrails is a gesture-controlled recipe app developed to enhance kitchen convenience.
This app allows hands-free interaction through voice or gesture-sensing technology, letting users browse recipes and set timers without touching the device.
This innovation not only improves hygiene but also makes the cooking process more enjoyable and efficient, enhancing the overall kitchen experience.
CookTrails is presented as a "smoke-and-mirror" prototype, utilizing a video to simulate app interactions through gestures. This method creates an illusion of a more refined and interactive app experience, beyond what is currently available in the early stages of development.
Fisrt Time Open CookTrails
This app's onboarding includes a series of tutorial pages, emphasizing gestures as the primary interaction mode.
Upon first use, CookTrails introduces users to KuKu, a digital assistant. KuKu guides users through step-by-step tutorials to master effective gesture use for app navigation and control.
Importantly, there are nine different gestures that users need to learn from KuKu, which once mastered, allow for the smooth operation of the entire app.
Final Video - Onboarding
Design Process
Researching Gesture
In the initial research phase, I focus on observing both intentional and unintentional gestures in individuals, using video recordings of conversations.
This study shows that gestures are often subtle and overlooked, but they vary widely and align with the content and tone of speech, thus improving communication clarity. Recognizing that the variety and subtlety of gestures can enhance technological interfaces and make them more intuitive.
Based on the discovery that gestures are diverse and often align with verbal content and emotional states, the designed gestures for the app should be universally understandable and applicable to all users.
Exploring Gesture
My goal was to interact with the application using precise and visually understandable gestures.
However, as I progressed, I realized that some gestures that seemed clear to me might not be easily understood by the audience. This realization prompted me to adjust my approach.
How I decided on chosen gestures:
In the process of finding an appropriate gesture for each specific function, I initially created different gestures for various functions based on my own ideas. I designed them in 3 different styles and presented them as animations to my friends. By testing and gathering feedback.
Final Gesture List
How I tested the gestures to find out:
During gesture testing, I asked friends as subjects to see if they could understand and mimic gestures based on animations. This helped identify which gestures were intuitive and easy to replicate. I demonstrated the app's functions and observed whether users could correctly link each gesture to its corresponding function, assessing the clarity and user resonance of my gesture choices.
The final gesture design for CookTrails was refined to be universally recognizable and straightforward, avoiding complex or subtle movements. This process identified clear, effective gestures, making them more user-friendly and understandable.
Design Feedback
Feedback is crucial for any interactive system or product and holds particular significance in gestural interface design.
Feedback reduces user uncertainty in interactive systems and provides real-time responses to user gestures, allowing users to see the system's tracking and understand their actions. This helps in recalibrating gestures for accuracy and boosting confidence.
Echo Feedback:
Echo feedback reflects unprocessed sensor data back to users. It shows that the system acknowledges user interaction, aiding in calibration. An example is a mouse cursor movement mirroring hand motion without activating interface elements.
Semantic Feedback:
This involves processed data providing visual, auditory, or tactile feedback, indicating successful action completion. For example, a button changing color upon being tapped signals recognition of the action by the system.
Initial Feedback
CookTrails Initial Feedback Model:
A popup appears at the bottom of the screen when the user's hand enters the operational range.
Icons move synchronously with the detected gesture (Echo feedback).
An icon indicates correct/successful gesture recognition, with the UI changing accordingly (Semantic feedback).
The popup disappears after the operation.
Final Feedback
CookTrails Final Feedback Model:
Upon hand detection in the operational range, the feedback shows the user's specific location on the page (Echo feedback).
The system monitors the user's gestures, with the UI changing based on the function of the gesture (Semantic feedback).
The final version of CookTrails was refined to use less space and avoid extra design elements, ensuring that the feedback mechanisms didn't cover the page content. This streamlined approach balanced functionality and interface clarity.
User Flow Diagramming
A User Flow diagram is a visual guide showing the steps a user takes in a digital interface, like a website or app. It helps me to plan and streamline the user's journey, highlighting the sequence of interactions and potential issues, and ensuring intuitive navigation.
Task Flow 1
Add Recipe to Favorite
Task Flow 2
Set a Timer
Task Flow 3
Watch Recipe Instructions
Task Flow 4
Looking For a Specific Recipe
Task Flow 5
Record by Take a Photo
Task Flow 6
Preview Recipes
Each user flow is treated as a distinct functional task. Special attention is given to annotating the involved gestures in the flow design, ensuring that suitable user interface components are created for each key interactive element.
Wireframes & Screens
In designing the core user interface of CookTrails, I began with foundational UI research.
This involved studying and integrating UI patterns from a variety of existing apps, and analyzing the strengths and weaknesses of each page design.
From this research, I started with mid-fidelity wireframes, progressively enriching and refining the design details to create a professional and user-friendly interface.
UI Patterns - Research
Mid-Fidelity Wireframes
Mid-Fidelity Wireframes
Mid-fidelity wireframes help me visualize the basic layout and functionality of the user interface. It's a crucial step for refining design concepts and understanding user interactions, allowing for early adjustments.
Final Screens
User Testing
User testing is vital for UX designers to validate design choices and ensure a user-centric product. It helps identify issues for improvement. A testing script is important for structured testing, outlining tasks, and maintaining consistency.
Script：
I am currently developing a gesture-controlled recipe app, and I'd like to show you the prototype. First, take a look at this page. I will then provide you with a scenario. You will be using this application as a user of this software. The scenario is as follows: You have downloaded a contactless recipe app. You need to learn the gestures along with the tutorials in the software, so you should try to perform each gesture after it has been taught in order to operate the software.
Questions:
Do you think the tutorials are clear enough for learning the gestures?
Did the app provide sufficient feedback or cues to ensure you were performing the gestures correctly?
Were the gestures easy to perform and remember? If not, which specific gestures were challenging?
What were your overall thoughts and feelings about the gesture-controlled recipe app?
Can you identify areas that you think could be improved?
Anything else you'd like to add or share?
Notes From Testing
Main takeaways:
Gestures: Adjust the color of a partial gesture when two gestures appear at the same time.
"Select" Gestures: Adjust the motion track animation in the tutorial section.
Arrows in the gesture animation section should change color to make this section stand out more.
Adjust the ”preview” animation to make the preview part smoother.
Add a page for users seeking KuKu for gesture help, including all gesture guides.
Brand Style Guide
The final phase in developing the CookTrails concept involves creating its style guide. This process consolidates the product's aesthetic into a unified visual identity, shaping its overall look and feel.
When creating the style guide for this application, I aimed to ensure harmony and comfort in the design and overall user interface. To achieve this, I developed several different color palettes and tested their usage in various primary frames within the app. This method allowed me to determine which color palette best suited specific interfaces. I opted for Brandon Text to maintain consistency and readability throughout the entire application.
Final Thoughts
This project delved into the design of a gesture-based interface, emphasizing user testing and feedback for refinement. By analyzing gestures, it highlighted cultural, linguistic, and emotional variations crucial for a universally intuitive app. A style guide ensured design consistency, while green screen technology underlined the importance of visual storytelling and lighting. The project showcased the advantages of gesture interaction for enhancing digital engagement, leading to a more immersive user experience. The final design combines visual appeal with user-friendly tutorials, and a detailed video with a backstory, underscoring a deep understanding of design principles and user engagement.