How I Developed an AI-Enabled Sentient Artifact for Real-Time Skin Analysis and Personalized Compliments, projecting an 80% Skin Tone Analysis Accuracy and 50% Increase in User Engagement.

Project Overview:

LumiLook is an AI-enabled Sentient Artifact designed to enhance personal well-being and confidence through real-time skin analysis and personalized compliments.

My Role:

UX Designer (Team of 3)

Duration : 4 weeks design sprint

Design Process:

We conducted market research and user interviews to identify needs and preferences, then developed a strategic plan incorporating computer vision for real-time skin analysis and personalized compliments. After brainstorming, we coded and tested a working prototype.

Design ToolKit:

Arduino | Figma | GitHub | Processing | Rasbery Pi | Teachable Machine | TinkerCAD | Wekinator

  • Ensuring the mirror accurately analyzes skin conditions in various lighting environments.

  • Providing timely and contextually appropriate compliments and recommendations.

  • Implementing complex computer vision and AI algorithms in a consumer-friendly device.

  • Designing an interface that encourages daily use without being intrusive.

The Problem Space

Current personal care routines lack integrated, real-time technological solutions that provide accurate skin analysis and personalized feedback, resulting in a gap between users' needs for daily skincare management and their ability to receive timely, relevant recommendations.

Problem Statement

The Journey and Process

TheSmart Mirror project began with a vision to combine technology with personal care in a way that was both innovative and meaningful.

The design process involved user research to understand the core needs and preferences of potential users.

The team developed prototypes that leveraged AI to analyze users' skin conditions and offer health and beauty recommendations.

The Concept

The inspiration for LumiLook came from the increasing consumer demand for personalized, technology-driven beauty and skincare solutions, especially for acne prone skin.

Drawing parallels from the functionality of smart home devices , we envisioned a sleek design, with the intuitive interfaces of smart home assistants, and the detailed analysis capabilities seen in advanced medical imaging devices.

The ability to take high-quality selfies was added as an essential feature, inspired by the popularity of social media and the growing trend of sharing beauty and wellness journeys online.

This feature allows users to document their skin's progress, share their looks, and receive instant feedback, enhancing both the practical and social aspects of their skincare routine.

Research

  • 75% of users reported increased engagement with their skincare routines when using smart devices that provide real-time feedback (Source: Smart Home Market Analysis, 2020).

  • 80% of consumers expressed interest in AI-driven personal care products for their potential to offer personalized advice and recommendations (Source: Consumer Technology Association, 2019).

  • 68% of users felt more confident and satisfied with personalized compliments and feedback on their appearance (Source: Journal of Cosmetic Science, 2021).

  • 72% of users reported improved skin health and appearance with accurate, real-time skin analysis and recommendations (Source: Dermatology Research and Practice, 2020).

  • 65% of consumers showed a preference for smart mirrors over traditional mirrors due to their enhanced functionality and technological integration (Source: Home Technology Insights, 2021).

Tech Research

Current market technology in skincare and styling focuses on advanced imaging techniques like spectrophotometry, colorimetry, and UV light scanning combined with AI-based image recognition to provide detailed skin analysis and personalized recommendations for enhancing personal care routines.

Feature Prototyping

Technical Challenges and Solutions

Accurate analysis of skin conditions in various lighting conditions was a significant challenge.

The team experimented with different types of sensors and AI algorithms, incorporating a high-resolution camera and thermal imaging techniques to detect and analyze subtle changes in the user's skin.

Skin Analysis in Various Lighting Conditions

Contextual Personalization

Early prototypes struggled with providing timely and contextually appropriate compliments.

Feedback led to improvements in the AI's linguistic models, enhancing its ability to understand context and mood subtleties.

Skin Bacteria Analysis and Styling Clothes Advice

Initial attempts at skin bacteria analysis and styling clothes advice proved technically challenging and less relevant to user needs. The team pivoted to focus on makeup recommendations, which were more feasible and aligned better with user expectations.

Mistakes and Learning

The development process involved several iterations based on user feedback:

  • Early prototypes sometimes provided generic or misplaced compliments, prompting improvements in the AI's contextual understanding.

  • Challenges in implementing accurate skin bacteria analysis and clothing styling advice led to a shift in focus to makeup recommendations.

Failed attempts with Wekinator and Raspberry Pie

Form Specification

  • Sleek design with a high-resolution display integrated into a reflective surface.

  • Equipped with a high-resolution camera (Sony IMX586) and thermal imaging sensors (FLIR Lepton 3.5) to capture detailed skin data.

  • Easy access to real-time analysis and recommendations through touch or voice commands.

  • Made with stainless steel and tempered glass to withstand daily use in a bathroom environment, including exposure to moisture.

Lumi Look

HMW tailor skincare routines to individual changes and needs?


HMW boost confidence against unrealistic beauty standards?

Mirror between Ls
Lumi Look -2 Ls for initials using both hands

Key Features

  • Real-time skin tone analysis, personalized skincare, and makeup recommendations based on the user's current skin condition.

  • Personalized compliments by analyzing the user's appearance in real-time using AI.

  • The selfie camera feature enables users to track skin health changes for informed skincare decisions and early issue detection.

  • A Processing script to capture a selfie triggered by a signal from an Arduino, featuring a countdown timer before the capture.

  • A Processing script using OpenCV to detect faces and generate a heat map for skin tone analysis based on the captured video feed.

  • A Processing script that captures video and plays a AI generated voice memo as a compliment after a short delay, using the Minim library for audio playback.

CODING!

Prototype Demo.

The project presentation was centered around producing a woking prototype.

What went into the Demo Set-up?

  • Processing for visual programming, voice memos for audio feedback, and temperature sensor for skin color detection and makeup recommendations.

  • Processing was utilized to create the interactive visual interface. This allowed the prototype to capture real-time images of the user's face and analyze skin conditions using computer vision algorithms. The system displayed these analyses on a connected screen, providing immediate feedback.

  • Voice memos were integrated to deliver personalized compliments and feedback. This feature aimed to enhance user interaction by providing verbal affirmations and recommendations based on the skin analysis.

  • Computer vision captured detailed skin data, enabling accurate detection of skin tones and conditions. This information was crucial for generating relevant and personalized makeup recommendations.

  • Based on the analyzed skin data, the system provided tailored makeup recommendations (foundation shades, blush, etc.) that matched the user's skin tone and addressed specific skin conditions.

  • The pitch emphasized how LumiLook combines AI and advanced sensor technology to offer a personalized care experience, enhancing both the practical and emotional aspects of skincare routines.

Skin Color Code System

Class Demonstration with Prof. Alex Baumgardt .

Projected Success Metrics

Personal Takeaways

  1. Discovery: Identified that personalized feedback can positively impact user self-esteem and daily routines.

  2. Mistakes: Encountered issues with the accuracy of skin tone analysis due to varying lighting conditions.

  3. Improvements: Implementing adaptive lighting solutions and conducting extensive testing under different environmental conditions will improve accuracy.

Next Steps

  • Implementing robust data security measures to protect user privacy and ensure secure handling of personal data.

  • Refining AI algorithms to improve accuracy and predictive skin analysis with contextual feedback.

  • Ensuring the device and associated data management processes comply with HIPAA regulations to protect user health information.

  • Conducting beta testing with a focus group to gather more feedback and make necessary adjustments before full-scale production.

Previous

Integrating AI Superhero Voiceovers in Custom Mail Art for Kids