MONOCULAR CAMERA-BASED AUTONOMOUS SIDEWALK NAVIGATION USING EFFICIENT MACHINE-LEARNING-DRIVEN IMAGE SEGMENTATION AND IMAGE-SPACE PATH PLANNING

Abstract

This thesis studies sidewalk navigation for a small electric scooter using a deliberately simple sensor setup: one forward-facing commodity RGB camera per run, CPU-only processing, no LiDAR, no stereo pair, and no multi-camera rig. The video data were collected with phone/action-camera hardware, including an iPhone camera and a GoPro-style camera, so the work is framed around low-cost monocular sensing rather than specialized robotics sensors. The pipeline was developed through four design iterations, beginning with BEV skeleton-graph construction and ending with template-approval planning. The evaluation compares ten path-planning methods from geometric, cost-field, topological, and template-based families in both bird's-eye view (BEV) and image-space coordinates. For segmentation, a compact SegFormer-B0 student (3.7M parameters) is trained from OneFormer Swin-L pseudo-labels and then fine-tuned on human-corrected masks, reaching IoU 0.964 at 14.0 ms. Held-out IoU is 0.974, higher than the training-video IoU of 0.963. For midpoint planning, the image-space planner obtains a lateral center error of 14.5 px with a runtime of 3.0 ms per frame on 448 hand-annotated campus sidewalk frames across eight video sequences. This is the best accuracy/runtime result among the ten planners. It also ranks significantly higher than the other methods (Friedman p < 10⁻⁶¹; all nine pairwise comparisons between midpoint and the other planners are significant after Bonferroni correction; 32 of the 45 total pairwise comparisons are significant). Although the midpoint planner has the best accuracy/runtime tradeoff among the tested planners, it is still intent-agnostic: it follows the visible corridor center and cannot perform commanded turns. For this reason, the final system keeps BEV for metric-scale turn compliance. The GPS-conditioned waypoint-turn planner checks the proposed turn against the BEV corridor using a simple containment safety guard. The most costly part of BEV processing for turn validation is not the projection step, but the search for a path. Path extraction through skeleton search followed by Dijkstra takes 540–690 ms per frame. By comparison, the path-acceptance test used in template approval takes only 3–5 ms per frame. By using fixed-arc verification and family-reuse hysteresis in place of the noise-sensitive skeleton graph, the template-approval planner reduces mean heading angle by 40.6% and halves the total number of path-source switches compared with the skeleton baseline. The image-space midpoint path can still contain noticeable heading-angle variation (4.5 deg/frame on average and 18.5 deg at the 95th percentile), which motivates the final temporal path-smoothing stage. On the 1800-frame accepted run, the method achieves 100% template acceptance with mean segmentation IoU of 0.954 while maintaining 9.97 FPS. In summary, the final dual-domain system uses the image-plane midpoint planner for forward motion, BEV for metric-scale turn-compliance checking and corridor extraction, and temporal smoothing for path stability. All results are evaluated offline from daytime sidewalk video in a single university-campus location. Future work includes testing in other locations, times of day, and weather conditions, as well as online closed-loop deployment on the robot.