Optimal Formation Control for Unmanned Aerial Vehicle Teams with Collision Avoidance Constraint and Unknown Dynamics

Document Type : Dynamics, Vibrations, and Control

Authors

1 Ph.D. Student, Faculty of Electrical Engineering, Iran University of Science and Technology, Tehran, Iran

2 Corresponding author: Associate Professor, Faculty of Electrical Engineering, Iran University of Science and Technology, Tehran, Iran

Abstract

This paper presents distributed training approach for a nonlinear and heterogeneous multi-UAV system to solve a safe and optimal formation control problem. The objective of control is to ensure safety while achieving optimal performance. In this regard, the position and attitude controllers are considered in series. First, the optimal formation control design is defined as the optimal performance in position control and is modeled by the cost function. In this article, with the integration of cost functions and local control barrier functions (CBFs), a novel distributed optimization problems are introduced. Existing the local CBF in the augmented cost function ensures the safety of the position control, and as a result, collisions do not occur along the path of UAVs. The proposed method considers the safe and optimal position controllers by solving unconstrained optimization problems instead of constrained ones. In the next stage, the reference attitudes are driven by virtual position control. The attitude tracking optimal control is considered the optimal performance in the attitude control, and the related cost function models it. Finally, the stability and safety of the proposed controllers are proven. These optimal and safe policies are obtained sequentially using off-policy multi-agent reinforcement learning (MARL) algorithms which do not require knowledge of UAVs' dynamics. The proposed algorithms are validated by simulating the formation control problem of 6 UAVs with collision avoidance constraints.

Highlights

  • Distributed formation control of a nonlinear and heterogeneous multi-UAV system.
  • Integrating local CBF with MARL to guarantee collision-free constraints in a data-driven approach.
  • Proposing two cascade off-policy RL algorithms for controlling position and attitude model-freely to achieve collision-free formation.

Keywords

Smiley face

References

[1] Barbastegan M, Bagheri A, Yazdani E, Chegini S. Optimal control of an aircraft pitch angle using pid and sliding mode control based on PSO algorithm. Journal of Aerospace Mechanics. 2020;15(4):49-66 (In Persian).##
[2] Cao Y, Yu W, Ren W, Chen G. An overview of recent progress in the study of distributed multi-agent coordination. IEEE Transactions on Industrial informatics. 2012;9(1):427-38.##
[3] Oh KK, Park MC, Ahn HS. A survey of multi-agent formation control. Automatica. 2015;53:424-40.##
[4] Xu J, Wang L, Liu Y, Sun J, Pan Y. Finite-time adaptive optimal consensus control for multi-agent systems subject to time-varying output constraints. Applied Mathematics and Computation. 2022;427:127176.##
[5] Zhou J, Zeng D, Lu X. Multi-agent trajectory-tracking flexible formation via generalized flocking and leader-average sliding mode control. IEEE Access. 2020;8:36089-99.##
[6] Hua Y, Dong X, Li Q, Ren Z. Distributed adaptive formation tracking for heterogeneous multiagent systems with multiple nonidentical leaders and without well‐informed follower. International Journal of Robust and Nonlinear Control. 2020;30(6):2131-51.##
[7] Wang L, Xi J, He M, Liu G. Robust time‐varying formation design for multiagent systems with disturbances: extended‐state‐observer method. International Journal of Robust and Nonlinear Control. 2020 May 10;30(7):2796-808.##
[8] Amirani MZ, Bigdeli N, Haeri M. Time varying formation control of unmanned aerial vehicle multi-agent systems with unknown leader input. Journal of Aerospace Mechanics. 2021;17(2):53-69 (In Persian).##
[9] Sayyaadi H, Mostafavi E. Formation control of unmanned helicopters by leader- follower method. Journal of Aerospace Mechanics. 2018;13(4): 59-69 (In Persian).##
[10] E. Zhao, T. Chao, S. Wang, and M. Yang, "Finite-time Formation Control for Multiple Flight Vehicles with Accurate Linearization Model," Aerospace Science and Technology, vol. 71, pp. 90-98, 2017.##
[11] Zhao E, Chao T, Wang S, Yang M. Finite-time formation control for multiple flight vehicles with accurate linearization model. Aerospace Science and Technology. 2017;71:90-8.##
[12] Canese L, Cardarilli GC, Di Nunzio L, Fazzolari R, Giardino D, Re M, Spanò S. Multi-agent reinforcement learning: A review of challenges and applications. Applied Sciences. 2021;11(11):4948.##
[13] Odekunle A, Gao W, Davari M, Jiang ZP. Reinforcement learning and non-zero-sum game output regulation for multi-player linear uncertain systems. Automatica. 2020;112:108672.##
[14] Sutton RS, Barto AG. Reinforcement learning: An introduction. MIT press; 2018.##
[15] Lewis FL, Vrabie D. Reinforcement learning and adaptive dynamic programming for feedback control. IEEE circuits and systems magazine. 2009;9(3):32-50.##
[16] Wen G, Chen CP, Li B. Optimized formation control using simplified reinforcement learning for a class of multiagent systems with unknown dynamics. IEEE Transactions on Industrial Electronics. 2019;67(9):7879-88.##
[17] Qu Q, Sun L, Li Z. Adaptive critic design-based robust cooperative tracking control for nonlinear multi-agent systems with disturbances. IEEE Access. 2021;9:34383-94.##
[18] Bastani O. Safe reinforcement learning with nonlinear dynamics via model predictive shielding. In 2021 American Control Conference (ACC). 2021:3488-3494.##
[19] Yazdani NM, Moghaddam RK, Kiumarsi B, Modares H. A Safety-Certified Policy Iteration Algorithm for Control of Constrained Nonlinear Systems. IEEE Control Systems Letters. 2020;4(3):686-91.##
[20] Marvi Z, Kiumarsi B. Safe reinforcement learning: A control barrier function optimization approach. International Journal of Robust and Nonlinear Control. 2021;31(6):1923-40.##
[21] Qin J, Li M, Shi Y, Ma Q, Zheng WX. Optimal synchronization control of multiagent systems with input saturation via off-policy reinforcement learning. IEEE transactions on neural networks and learning systems. 2018;30(1):85-96.##
[22] B Yan B, Shi P, Lim CC, Shi Z. Optimal robust formation control for heterogeneous multi‐agent systems based on reinforcement learning. International Journal of Robust and Nonlinear Control. 2022;32(5):2683-704.##
[23] Labbadi M, Boudaraia K, Elakkary A, Djemai M, Cherkaoui M. A continuous nonlinear sliding mode control with fractional operators for quadrotor UAV systems in the presence of disturbances. Journal of Aerospace Engineering. 2022;35(1):04021122.##
[24] Raffo GV, Ortega MG, Rubio FR. An integral predictive/nonlinear H∞ control structure for a quadrotor helicopter. Automatica. 2010;46(1):29-39.##
[25] Lee H, Kim HJ. Constraint-based cooperative control of multiple aerial manipulators for handling an unknown payload. IEEE Transactions on Industrial Informatics. 2017;13(6):2780-90.##
[26] Wang JL, Wu HN. Leader-following formation control of multi-agent systems under fixed and switching topologies. International Journal of Control. 2012;85(6):695-705.##
[27] Olfati-Saber R, Murray RM. Consensus problems in networks of agents with switching topology and time-delays. IEEE Transactions on automatic control. 2004;49(9):1520-33.##
Aerospace Mechanics
Volume 19, Issue 1 - Serial Number 71
Serial No. 71, Spring Quarterly
June 2023
Pages 61-79

Files

History

  • Receive Date: 02 September 2022
  • Revise Date: 16 October 2022
  • Accept Date: 24 November 2022
  • Publish Date: 09 April 2023

Share

How to cite

Statistics