PPE Compliance Vision Pipeline

Industrial PPE (Personal Protective Equipment) compliance demo for video:

1. Fine-tune RF-DETR on the SH17 safety dataset
2. Run video inference with ByteTrack + temporal compliance logic
3. Query results through an MCP server (no re-inference in the agent loop)

Video → RF-DETR → ByteTrack (track_id) → PPE association → state machine
                                                      ↓
                              annotated MP4 + analytics JSON → MCP tools → Claude/Cursor

Repo: github.com/Pratham-22/ppe-compliance
Weights: Release v0.1 (checkpoint_epoch44.pth)

Demo results

Annotated factory-floor run: ByteTrack IDs + temporal PPE compliance overlays (missing: helmet, safety-vest).

Frame ~2.7s	Frame ~5.7s

Sample clip: outputs/inference/factory_floor_annotated.mp4 · analytics: factory_floor_analytics.json

Low-resolution YouTube warehouse footage often has no real helmets/vests, so this demo mostly shows person tracks flagged as non-compliant (domain shift vs SH17). Prefer clear construction video for positive PPE detections.

Features

Layer	What it does
Detection	RF-DETR Large fine-tuned for person / helmet / safety-vest (17 SH17 classes)
Tracking	ByteTrack assigns stable worker track_ids across frames
Compliance	Helmet/vest associated via geometry; 15-frame window, 60% ratio threshold
MCP	Read-only tools over precomputed JSON (summary, violations, worst offenders, trends, …)

MCP tools

Tool	Purpose
get_site_summary	Site-wide compliance rollup
get_worker / explain_worker	Per-track status + narrative
list_violations / get_worst_offenders	Who failed PPE
get_violations_in_window	Violations between two timestamps
get_compliance_trend	Compliance rate over time buckets
get_flickering_tracks	Short-lived / noisy track IDs
get_worker_summary_ranked	Rank all workers by ratio / duration
get_timeline	Frame-level events
reload_analytics	Re-read JSON after a new inference run

Repo layout

ppe-compliance/
├── scripts/
│   ├── train_smoke.py / train_full.py   # RF-DETR fine-tune
│   ├── video_infer.py                   # end-to-end video pipeline
│   ├── ppe_compliance.py                # association + state machine
│   ├── export_checkpoint.py             # Lightning ckpt → RF-DETR .pth
│   ├── sbatch_*.sh                      # OSC Slurm helpers
│   └── upload_release_weights.sh        # optional: upload .pth to GitHub Releases
├── ppe_mcp/
│   ├── store.py                         # JSON query layer
│   ├── server.py                        # FastMCP tools
│   └── *_config.example.json            # Claude / Cursor wiring
├── outputs/inference/                   # sample annotated video + analytics JSON
├── docs/images/                         # README stills from the annotated demo
├── setup_env.sh
└── requirements.txt

Quick start

1. Environment (GPU recommended)

git clone https://github.com/Pratham-22/ppe-compliance.git
cd ppe-compliance

module load python/3.12   # on OSC Ascend, if available
bash setup_env.sh
source .venv/bin/activate

2. Download model weights

Weights are not in git (GitHub file limit is 100 MB). Download the fine-tuned checkpoint from Release v0.1:

mkdir -p outputs/rfdetr_large
curl -L -o outputs/rfdetr_large/checkpoint_epoch44.pth \
  "https://github.com/Pratham-22/ppe-compliance/releases/download/v0.1/checkpoint_epoch44.pth"

3. Run inference on a video

A sample demo clip and prior analytics are already in the repo under data/videos/ and outputs/inference/.

# GPU job on OSC
sbatch scripts/sbatch_video_infer.sh

# or locally
python scripts/video_infer.py \
  --video data/videos/factory_floor_demo.mp4 \
  --checkpoint outputs/rfdetr_large/checkpoint_epoch44.pth \
  --device cuda

Produces:

• outputs/inference/*_annotated.mp4
• outputs/inference/*_analytics.json

4. Query results with MCP

export PPE_ANALYTICS_JSON=outputs/inference/factory_floor_analytics.json
python -m ppe_mcp.server

Wire into Cursor / Claude Desktop using the examples in ppe_mcp/.

Example agent prompts

• “Give me a site-wide PPE summary.”
• “Who are the 3 worst helmet offenders?”
• “Explain worker #2.”
• “What happened between 5s and 10s?”

Optional: train from scratch

Data

Download SH17 (Kaggle / upstream repo), then convert:

python scripts/convert_sh17_to_rfdetr.py

Train

# smoke test
python scripts/train_smoke.py

# full fine-tune (Slurm on OSC)
sbatch scripts/sbatch_train_rfdetr.sh

Export a usable checkpoint:

python scripts/export_checkpoint.py

Sample outputs

This repo includes a demo run under outputs/inference/ (stills above in docs/images/):

• Annotated video with #track_id labels and compliance colors
• Analytics JSON consumed by the MCP server

Notes

• Worker IDs are ByteTrack track IDs, not employee names.
• MCP is intentionally read-only over JSON — inference stays on GPU.
• Default compliance settings: detection threshold 0.35, window 15 frames, min ratio 0.6.

License / data

SH17 and third-party video remain under their original licenses. This code is provided for portfolio / interview demonstration.