基于深度强化学习的无人机智能飞行控制试验

Intelligent flight control test of unmanned aerial vehicle based on deep reinforcement learning

摘要:

深度强化学习提供了一种实现无人机智能飞行控制的新型技术范式，然而对“黑箱”神经网络智能模型的信任问题是限制其实际应用的主要阻碍。为了对基于深度强化学习设计的神经网络智能飞行控制模型进行试验验证，针对某固定翼缩比模型飞行器，基于面向多维连续状态输入、多维连续动作输出的近端策略优化（PPO）深度强化学习算法，发展从飞行状态到舵面/推力控制的纵向通道端到端智能飞行控制模型，并通过仿真手段进行鲁棒性能验证，开展仿真交互设计智能飞行控制模型向真实飞行试验迁移的工程实现，得到便于移植的机载神经网络控制器并进行模型飞行演示验证。试验结果初步验证了神经网络控制器的可行性以及泛化性能。

Abstract:

Deep Reinforcement Learning (DRL) provides a new technological paradigm for the intelligent flight control of unmanned aerial vehicles. However, confidence in the "black box" Artificial Neural Network (ANN) intelligent model is the main obstacle to practical application. To validate the neural network-based intelligent flight control model designed with DRL through flight test, a longitudinal end-to-end intelligent flight control model that maps the flight state to the elevator/thrust commands is developed for a fixed-wing scaled model aircraft, based on the multi-dimensional continuous state input and action output DRL Proximal Policy Optimization (PPO) algorithm. The robustness of ANN control model is validated through the simulation, and its engineering implementation for the sim-to-real transfer is further carried out. A flexible onboard ANN controller is developed and the model flight demonstration is launched. The test results preliminarily verify the applicability and generalization performance of the ANN controller.