MoodCam: Emotion Recognition via Webcam 🎭

MoodCam detects human emotions in real time using your webcam and a PyTorch-trained CNN model. This project implements a grayscale emotion recognition system with 7 emotion classes: angry, disgust, fear, happy, sad, surprise, and neutral.

Quick Start (Colab)

1. Clone repo & open notebook

   git clone <your-repo-url>
   cd moodcam

   Open notebook/01_moodcam_colab.ipynb in Google Colab

2. Run setup cell (installs deps, checks data)

3. Train model

   bash scripts/train.sh

4. Evaluate model

   bash scripts/evaluate.sh

   • Check accuracy at artifacts/metrics/accuracy.txt
   • Check detailed metrics at artifacts/metrics/classification_report.csv

5. Generate predictions

   bash scripts/infer_test.sh

   • Output in artifacts/submissions/submission.csv

6. Run webcam demo locally

   bash scripts/webcam.sh

Data Layout

Expected data structure:

data/
├── train/
│   ├── angry/
│   ├── disgust/
│   ├── fear/
│   ├── happy/
│   ├── sad/
│   ├── surprise/
│   └── neutral/
├── test/
│   └── *.jpg
├── train.csv
└── test_template.csv

Single-Command Workflow

Training

bash scripts/train.sh

Evaluation

bash scripts/evaluate.sh

Produces:

• artifacts/metrics/accuracy.txt - Overall accuracy
• artifacts/metrics/classification_report.csv - Per-class metrics

Test Inference

bash scripts/infer_test.sh

Produces:

• artifacts/submissions/submission.csv - Test predictions

Webcam Demo (Local)

bash scripts/webcam.sh

Backend API (for V0 Frontend)

Start the FastAPI backend:

uvicorn backend.fastapi_app:app --host 0.0.0.0 --port 8000

API Endpoints

• POST /predict - Upload image and get emotion prediction

  curl -X POST "http://localhost:8000/predict" \
       -H "Content-Type: multipart/form-data" \
       -F "file=@image.jpg"

  Response:

  {
    "emotion": "happy",
    "confidence": 0.921,
    "all_emotions": {
      "angry": 0.01,
      "disgust": 0.02,
      "fear": 0.01,
      "happy": 0.92,
      "sad": 0.02,
      "surprise": 0.01,
      "neutral": 0.01
    }
  }

• GET /classes - Get list of emotion classes

• GET / - Health check

Repository Structure

moodcam/
├── README.md
├── requirements.txt
├── config.yaml
├── data/
│   ├── train.csv
│   ├── test_template.csv
│   ├── train/                  # class folders: angry/, disgust/, ...
│   └── test/                   # test images (no labels)
├── notebook/
│   └── 01_moodcam_colab.ipynb
├── src/
│   ├── config.py
│   ├── data.py
│   ├── transforms.py
│   ├── model.py
│   ├── train.py
│   ├── evaluate.py
│   ├── infer.py
│   ├── webcam_app.py
│   └── utils.py
├── artifacts/
│   ├── checkpoints/
│   ├── logs/
│   ├── metrics/
│   │   ├── accuracy.txt
│   │   └── classification_report.csv
│   └── submissions/
│       └── submission.csv
├── scripts/
│   ├── train.sh
│   ├── evaluate.sh
│   ├── infer_test.sh
│   └── webcam.sh
├── backend/
│   └── fastapi_app.py
└── frontend/
    └── (empty; will link to backend later)

Technical Details

Model Architecture

• Backbone: ResNet18 with modified first conv layer for 1-channel grayscale input
• Input: 224x224 grayscale images
• Classes: 7 emotions (angry, disgust, fear, happy, sad, surprise, neutral)
• Training: Transfer learning with pretrained weights averaged for grayscale

Key Features

• Grayscale Processing: End-to-end 1-channel processing for efficiency
• Transfer Learning: ResNet18 pretrained weights adapted for grayscale
• Real-time Inference: OpenCV webcam integration with face detection
• API Ready: FastAPI backend for frontend integration
• Colab Optimized: GPU acceleration with CPU fallback

Training Configuration

• Epochs: 25 (configurable)
• Batch Size: 64
• Learning Rate: 1e-3 with AdamW optimizer
• Augmentation: Random horizontal flip, rotation (±10°)
• Early Stopping: 6 epochs patience
• Mixed Precision: Enabled for GPU training

Troubleshooting

GPU Not Found

• The system automatically falls back to CPU if CUDA is not available
• Training will be slower but functional

Missing Haar Cascade

• OpenCV includes the required cascade file
• Path: cv2.data.haarcascades + "haarcascade_frontalface_default.xml"

Unreadable Images

• Corrupted images are skipped with a warning
• Check image formats (supports common formats like JPG, PNG)

Webcam Issues

• Ensure webcam is not being used by another application
• Try different camera indices if multiple cameras are available

Judge Artifacts

The following files are required for evaluation:

1. artifacts/metrics/accuracy.txt

   accuracy: 0.8473
   

2. artifacts/metrics/classification_report.csv

   class,precision,recall,f1-score,support
   angry,0.81,0.79,0.80,1000
   disgust,0.70,0.68,0.69,500
   ...
   macro avg,0.79,0.78,0.78,35000
   weighted avg,0.83,0.84,0.83,35000

3. artifacts/submissions/submission.csv

   id,emotion
   test_001.jpg,happy
   test_002.jpg,neutral
   ...

4. notebook/01_moodcam_colab.ipynb - Complete training notebook

License & Credits

• Dataset: MoodCam Challenge (~35k grayscale images, 7 classes)
• Framework: PyTorch + OpenCV
• Model: ResNet18 with grayscale adaptation
• Author: [Your team name]

Acknowledgments

This project demonstrates modern deep learning techniques for emotion recognition, including transfer learning, grayscale optimization, and real-time inference capabilities.