DeepSmile: A Tool for the Future of Facial Rehabilitation
Traditional assessments of facial paralysis — like the House–Brackmann or Sunnybrook grading systems — rely heavily on subjective visual evaluations, which vary between clinicians. DeepSmile, a deep learning-based anomaly detection system, changes the game by offering a repeatable, objective, and quantifiable method to assess facial movement during a smile.
Highlights
• DeepSmile Introduces a novel AI-driven, objective method for evaluating facial movement anomalies.
• Provides an open-source tool that supports real-time, quantitative clinical diagnosis.
o Future directions might include multi-movement facial assessment and a more robust analysis including all the zones in the face.
How does DeepSmile Benefits Patients?
- Detects subtle facial asymmetries for earlier, more precise diagnoses.
- Provides objective scores to personalize and adjust treatment plans.
- Tracks recovery over time to evaluate treatment effectiveness.
- Visual reports improve patient understanding and trust.
- Open-source and scalable for use in clinical and research settings globallly
DeepSmile – How does this work?
1. Motion Capture Session: Reflective markers are placed on the patient’s face using a custom mask for consistency.
2. Patient Performs a Smile: The motion capture system records the 3D trajectory of facial markers.
3. Data Processing: The marker data is exported as a .csv file.
4. Load Into DeepSmile: Clinicians upload the file into the DeepSmile GUI.
5. Instant Report Generation:
o Degree of anomaly per marker (in mm and percentage)
o Global smile anomaly score
o Visual feedback on areas showing deviation from a healthy smile pattern
Motion Capture (MoCap) Platform
• Facial movement data is collected using an optical motion capture system (Vicon) that tracks 105 reflective markers on the face.
• Focus is placed on 52 markers around the lower third of the face, especially the mouth and oral commissures, which are highly involved in smiling.
• MoCap ensures high-precision 3D tracking, and a rigid dental support is used to eliminate head motion from the data.
DeepSmile Open-Source Software
• The trained LSTM model is embedded into DeepSmile, a graphical user interface (GUI) built in Python.
• Clinicians load a patient’s smile motion data (as .csv) into DeepSmile.
• The software outputs:
- A report showing per-marker anomaly scores in both mm and percentage.
- A global anomaly score, helping with diagnosis and monitoring over time.
- Visual comparison of predicted vs. actual movement (e.g., for left/right mouth corners).
- DeepSmile is publicly available in open source : https://github.com/PolezhaevaOlga/Face_Motion_Capture