Improving an Industrial Vision System from 84.5% to 92.0% mAP@50

The challenge

The system had to perform reliably on difficult satellite and aerial imagery, where small visual differences, shadows, orientation, background clutter, and inconsistent annotations could all produce costly errors.

Improving the system required more than trying another detector.

• Small or visually difficult targets were missed
• Shadows changed familiar objects
• Some orientations were underrepresented
• Similar backgrounds produced false positives
• Hard examples were underrepresented
• Annotation inconsistencies limited further tuning

1. Establish a measurable baseline

The existing YOLOv8 detector was evaluated using a consistent COCO-style pipeline. That created a baseline of 84.5% mAP@50 and made later model and data changes directly comparable.

2. Analyze failures, not only aggregate accuracy

False negatives and false positives were inspected and grouped by pattern. This changed the question from which model scores higher to what is causing the system to fail and which changes address those failures.

• Shadow-related misses
• Underrepresented orientations
• Very small targets
• Difficult background negatives
• Incorrect or inconsistent annotations

3. Improve the data where the failures pointed

Error analysis informed targeted changes rather than arbitrary augmentation. The goal was to make the training distribution represent the conditions the model encountered in practice.

• Rotation augmentation
• Shadow-related augmentation
• Mosaic and other targeted augmentations
• Additional hard negatives
• Correction of data and annotation issues

4. Benchmark stronger detection architectures

RF-DETR Large achieved 92.0% mAP@50 compared with the 84.5% YOLOv8 baseline—a 7.5 percentage-point absolute improvement.

The model change mattered, but the result was supported by reliable evaluation, failure analysis, and better data.

From model accuracy to production performance

A detector with strong benchmark accuracy can still be impractical if inference is too slow, expensive, or resource-intensive.

Training, evaluation, and export workflows were integrated into an automated pipeline for repeatability and experiment tracking.

The result

A stronger detection architecture, structured error analysis, and targeted data improvements increased detection performance by 7.5 percentage points.

The work also established a systematic production workflow for evaluating, optimizing, exporting, and serving computer-vision models.

Why the engineering process mattered

A model benchmark tells you what performs better. Failure analysis helps explain why the current system fails. Data analysis tells you what needs to change. Production benchmarking tells you whether the improved model can actually be deployed.