From Supervised to Reinforced: A One-Shot Deep Learning Approach to UAV Path Planning

Reinforcement Learning (RL) is increasingly recognized as a valuable fine-tuning tool in machine learning, offering significant benefits over traditional methods. Unlike standard training approaches that often require a detailed understanding of the problem domain, RL thrives on its capacity for trial-and-error learning. This makes it particularly well-suited for refining pre-trained models in complex tasks where conventional techniques fall short. This study introduces a pioneering method for designing a learning-based, single-shot UAV path planner that effectively transitions from 2D to 3D. Our approach combines supervised learning with reinforcement learning under an image-to-image single-shot strategy. Initially, we pre-train the actor, using a UNET architecture, through a behavior cloning method on a dataset derived from the A* algorithm within the Voxelgym2D environment. Concurrently, we train a reward estimator as a critic in the subsequent fine-tuning phase. The fine-tuning process employs the Soft Actor-Critic (SAC) algorithm, tailored with a customized exploration strategy to optimize the model's initial strengths. Through 8 million timesteps, a trained path planner achieves a 99.75% success rate in generating suboptimal paths within Voxelgym2D. Further training in a more generalized environment enhances robustness, and additional experimentation with task-specific objective rewards demonstrates the planner's adaptability. Validated in simulation environment, the planner shows the potential to substitute conventional path planners. Inference tests on an Nvidia Jetson Orin revealed rates surpassing 100Hz demonstrates the model's suitability for real-time applications. The outcome is a robust path planner, adept in training, adaptable to specific tasks, and scalable in 3D environments, showcasing the potential of our novel approach.