Project
NeuroPlay
An interactive system based on computer vision and AI that turns rehabilitation exercises into more engaging game experiences.
Brief / Challenges
Context, constraints and the design question.
NeuroPlay explores how rehabilitation exercises can become more engaging without losing their therapeutic purpose. The project responds to a common challenge in neurorehabilitation: exercises are often repetitive, difficult to track objectively and hard to keep motivating over time.
The goal was to design an interactive platform where body movements, facial expressions and eye gestures could be detected through a standard webcam and transformed into simple, measurable minigames.
How can repetitive rehabilitation exercises become interactive, motivating and clinically measurable experiences?
The Process
A concise view of how the project took shape.
The project was developed as a sequence of minigames based on different rehabilitation gestures. Each game connects a physical action to immediate visual feedback, using computer vision to detect hands, facial muscles and eye movement in real time.
The process focused on three main layers: defining therapeutic gestures, translating them into game mechanics, and designing a feedback system able to summarize performance after each session.
Process 01 — Gesture to Game
Turning therapy movements into minigames.
Each exercise was designed around a specific motor or facial action: opening the hand, touching fingers, raising eyebrows, smiling or blinking. These actions were transformed into simple game mechanics to make rehabilitation feel less repetitive and more goal-oriented.
Process 02 — Real-Time Tracking
Using the webcam as an interaction tool.
MediaPipe was used to track hands, face and eyes in real time. The system reads physical movement through the camera and translates it into interactive feedback, allowing the user to train without additional hardware.
Process 03 — Feedback System
From gameplay to rehabilitation data.
At the end of each session, the platform generates a performance summary with scores, progress indicators and a clinical-style report. The aim is to make the experience useful not only for the patient, but also for therapists monitoring improvement over time.
MINIGAMES STRUCTURE
1. Hand Flower
Opening and closing the hand makes the flower bloom.
This exercise uses hand tracking to detect flexion and extension movements. The user opens and closes the hand to animate the petals, turning a repetitive mobility exercise into a simple visual goal.
Training focus: hand mobility and flexion/extension coordination.
2. Digital Musical Scale
Touching each finger builds a musical sequence.
This minigame detects thumb-finger opposition and associates each gesture with a note. The user repeats the sequence by touching the thumb with different fingers, training precision and controlled hand movement.
Training focus: fine motor skills and sequential coordination.
3. Power Brows
Raising the eyebrows lifts virtual weights.
The system tracks eyebrow movement and translates facial muscle activation into a strength-based interaction. The more the user raises the eyebrows, the more the virtual dumbbells respond.
Training focus: frontal muscle control.
4. Smile Shot
Holding a smile triggers the capture.
This exercise detects facial expression and asks the user to maintain a smile until the visual ring is completed. The interaction turns facial muscle activation into a timed challenge.
Training focus: facial muscle activation and expression control.
5. Blink Test
Blinking becomes a reaction-time exercise.
The minigame uses eye tracking to measure how quickly the user closes their eyes after a visual signal. The interface works like a traffic light, turning blink speed into a measurable response.
Training focus: blink reflex and neuromotor response time.
6. Session Report
Completing the session generates a clinical overview.
At the end of the minigames, the platform summarizes the user’s performance through scores, recovery indicators and progress data. The report is designed to make the session readable for both the patient and the therapist, connecting gameplay results to rehabilitation monitoring.
The Outcome
Final direction, delivery and visual output.
The final prototype presents NeuroPlay as a modular neuro-training platform for home-based and clinical rehabilitation. Through a standard webcam, users can complete a sequence of bio-interactive minigames focused on hand mobility, facial muscle control and neuromotor response.
The outcome combines playful interaction with measurable feedback: the patient receives an engaging experience, while the system generates structured data that can support therapists in monitoring progress over time.
