Lumi Look is an AI-driven sentient artifact for real-time skin analysis and personalized compliments, projecting 80% accuracy and a 50% boost in user engagement.

Project Overview

Designing & developing a prototype for an AI-enabled Sentient Artifact

Key Task

Conducted research, developed a strategic plan, and built a prototype using computer vision.

Year & Duration

Year: 2024

Duration: 4 weeks

Role

UX Designer (Team of 3)

Toolkit: Arduino, Figma, GitHub, Processing, Raspberry Pi, Teachable Machine, TinkerCAD, Wekinator

Product Space

Ensuring the mirror accurately analyzes skin conditions in various lighting environments.
Providing timely and contextually appropriate compliments and recommendations.
Implementing complex computer vision and AI algorithms in a consumer-friendly device.
Designing an interface that encourages daily use without being intrusive.

Opportunity Statement

Personal care routines lack real-time tech for accurate skin analysis and personalized feedback, leaving a gap in timely skincare recommendations.

Skip to Prototype >

The Journey and Process

Vision: Merging technology with personal care for meaningful innovation.
Research: Identified user needs and preferences through research.
Development: Built AI-powered prototypes for skin analysis and personalized recommendations.

The Concept

Inspiration: Driven by demand for personalized, tech-enabled skincare, especially for acne-prone skin.
Design Influence: Combined smart home interfaces with advanced medical imaging for precise analysis.
Key Feature: High-quality selfies for tracking skin progress, sharing, and instant feedback.

Research

75% of users reported increased engagement with their skincare routines when using smart devices that provide real-time feedback (Source: Smart Home Market Analysis, 2020).
80% of consumers expressed interest in AI-driven personal care products for their potential to offer personalized advice and recommendations (Source: Consumer Technology Association, 2019).
68% of users felt more confident and satisfied with personalized compliments and feedback on their appearance (Source: Journal of Cosmetic Science, 2021).
72% of users reported improved skin health and appearance with accurate, real-time skin analysis and recommendations (Source: Dermatology Research and Practice, 2020).
65% of consumers showed a preference for smart mirrors over traditional mirrors due to their enhanced functionality and technological integration (Source: Home Technology Insights, 2021).

Tech Research

Current market technology in skincare and styling focuses on advanced imaging techniques like spectrophotometry, colorimetry, and UV light scanning combined with AI-based image recognition to provide detailed skin analysis and personalized recommendations for enhancing personal care routines.

Feature Prototyping

Mistakes and Learning

Early prototypes sometimes provided generic or misplaced compliments, prompting improvements in the AI's contextual understanding.

Challenges in implementing accurate skin bacteria analysis and clothing styling advice led to a shift in focus to makeup recommendations.

Failed attempts with Wekinator and Raspberry Pie

Form Specification

Sleek design with a high-resolution display integrated into a reflective surface.
Equipped with a high-resolution camera (Sony IMX586) and thermal imaging sensors (FLIR Lepton 3.5) to capture detailed skin data.
Easy access to real-time analysis and recommendations through touch or voice commands.
Made with stainless steel and tempered glass to withstand daily use in a bathroom environment, including exposure to moisture.

Technical Challenges and Solutions

Lighting Challenges: Used AI, high-res cameras, and thermal imaging to improve skin analysis in varied lighting.

Contextual Personalization: Enhanced AI linguistic models for more accurate and timely compliments.

Feature Pivot: Shifted from skin bacteria analysis and styling advice to makeup recommendations for better user relevance.

Lumi Look

HMW tailor skincare routines to individual changes and needs?
HMW boost confidence against unrealistic beauty standards?

Mirror between Ls

Lumi Look -2 Ls for initials using both hands

Key Features

Real-time skin tone analysis, personalized skincare, and makeup recommendations based on the user's current skin condition.
Personalized compliments by analyzing the user's appearance in real-time using AI.
The selfie camera feature enables users to track skin health changes for informed skincare decisions and early issue detection.

A Processing script to capture a selfie triggered by a signal from an Arduino, featuring a countdown timer before the capture.
A Processing script using OpenCV to detect faces and generate a heat map for skin tone analysis based on the captured video feed.
A Processing script that captures video and plays a AI generated voice memo as a compliment after a short delay, using the Minim library for audio playback.

CODING!

View Codes here

Prototype Demo.

What went into the Demo Set-up?

Processing for visual programming, voice memos for audio feedback, and temperature sensor for skin color detection and makeup recommendations.
Processing was utilized to create the interactive visual interface. This allowed the prototype to capture real-time images of the user's face and analyze skin conditions using computer vision algorithms. The system displayed these analyses on a connected screen, providing immediate feedback.
Voice memos were integrated to deliver personalized compliments and feedback. This feature aimed to enhance user interaction by providing verbal affirmations and recommendations based on the skin analysis.
Computer vision captured detailed skin data, enabling accurate detection of skin tones and conditions. This information was crucial for generating relevant and personalized makeup recommendations.
Based on the analyzed skin data, the system provided tailored makeup recommendations (foundation shades, blush, etc.) that matched the user's skin tone and addressed specific skin conditions.
The pitch emphasized how LumiLook combines AI and advanced sensor technology to offer a personalized care experience, enhancing both the practical and emotional aspects of skincare routines.

Skin Color Code System

Class Demonstration

Projected Success Metrics

Personal Takeaways

Discovery: Identified that personalized feedback can positively impact user self-esteem and daily routines.
Mistakes: Encountered issues with the accuracy of skin tone analysis due to varying lighting conditions.
Improvements: Implementing adaptive lighting solutions and conducting extensive testing under different environmental conditions will improve accuracy.

Next Steps