Vehicle Detection and Tracking using Coarse-to-Fine Module and Spatial Pyramid Pooling–Fast with Deep Sort

Abstract

Semarang City, a rapidly growing urban area in Indonesia, faces significant traffic challenges stemming from the widespread use of motorcycles, an inefficient public transportation system, and accelerated urban development. These factors contribute to congestion and complicate traffic management efforts. To address this issue and enhance monitoring capabilities, this study develops an automatic vehicle detection system utilizing the YOLOv8 algorithm, applied to CCTV footage obtained from TILIK SEMAR, a local traffic surveillance initiative. The research methodology encompasses several key stages: data collection from real-world traffic scenarios, meticulous annotation of vehicle types, model training using the YOLOv8 framework, and performance evaluation conducted at two distinct locations in Semarang—Banyumanik and Thamrin Pandanaran. The trained model achieved an impressive average accuracy, measured as mean Average Precision (mAP50), exceeding 97%, with a rapid processing time of 4.2 milliseconds per image, making it suitable for real-time applications. Among vehicle categories, the highest detection accuracies were recorded for buses at 99.3% and box trucks at 99.5%, reflecting the model’s robustness for larger vehicles. However, motorcycles presented a challenge, with a lower mAP50-95 score of 64.3%, attributed to variations in shape, size, and lighting conditions. Overall, the system successfully identified 96.77% of 3,036 vehicles across the test dataset, demonstrating strong generalization across diverse traffic conditions. These findings validate YOLOv8 as an effective tool for real-time traffic monitoring in urban settings. Future enhancements will focus on expanding dataset diversity and improving performance under challenging environmental factors, such as adverse weather or low-light scenarios, to further refine the system’s reliability.