Reinforcement Learning-based Adaptive Prediction Horizon for Frenet Frame Autonomous Vehicle Path Planning

Abstract

Autonomous vehicles (AV) are gaining attention as technology advances, and the demand is on the rise. With its popularity, it is important to prioritize the safety of the people. The main concern with autonomous driving is the safety and ability to stay on course. This research presents a framework for autonomous vehicle trajectory planning by using three methods. The first was using a fixed prediction horizon. The next method was using an adaptive prediction horizon. The final method and the new approach are using Reinforcement Learning (RL) with Proximal Policy Optimization (PPO) to pick the best prediction horizon value in the Frenet Frame Method (FFM). Our new approach investigates an adaptive trajectory planning framework for autonomous driving that takes the fixed path planning code and is enhanced to adaptively adjust the pre-horizon based on the distance to the obstacle in the path. This approach to the problem allows the prediction horizon to no longer be fixed. It allows the vehicle to detect the barriers and use the horizon value associated with the distance to the vehicle. This modification enables the planner to respond more intelligently to environmental constraints by shortening the horizon in proximity to obstacles and lengthening it in open road segments, thereby improving safety and trajectory smoothness. The next step was to implement RL. For this segment, the PPO algorithm from the Stable Baseline3 RL library was implemented within a custom gym environment. In this environment, the agent consists of discrete candidate prediction horizons, and it learns to select the optimal horizon by minimizing the reward function that minimizes tracking error, steering angle, and computation time. Given the action of different prediction horizons, PPO would select the optimal value for each criterion. The results showed the PPO was able to select the best value based on the reward. Showcasing that our research showed better results than the fixed and adaptive prediction horizons.