Benoit Clement
home
Short bio
CV
Partners
Teaching
Control
miscellaneous
Research
Publications
Projects
Students
Medical
ML-Control

Thomas Chaffre PhD

v.2023

Learning Stochastic Adaptive Control using A Bio-Inspired Experience Replay with an AUV

  • This project is part of the Machine Learning and Control project.

  • Funding is from Region Bretagne, South Austalia and Naval Group.

  • Supervisors : Estelle CHAUVEAU, Gilles LE CHENADEC, Paulo SANTOS, Karl SAMMUT and Benoit CLEMENT

Defense (link)

  • December 13th, 2022 at Flinders University

  • a video link is available is here

  • Dissertation is available here (link) and the slides are here (link)

Board of Examiners

  • M. FILLIAT David, Professeur, ENSTA Paris - reviewer - (rapport)

  • M. BANDYOPADHYAY Tirthankar, Senior Researcher, CSIRO, Brisbane - reviewer - (rapport)

  • M. BUCHE Cédric, Professeur des Universités, IRL CROSSING / ENIB, Adelaide

  • M. CREUZE Vincent, Professeur des Universités, Université de Montpellier

  • Mme FANTONI Isabelle, Directrice de Recherche, LS2N, Nantes

  • M. SAMMUT Karl, Professeur, Flinders University, Adelaide, co-directeur de thèse

  • M. CLEMENT Benoît, Professeur, ENSTA Bretagne, IRL CROSSING, Adelaide, co-directeur de thèse

  • M. LE CHENADEC Gilles, Enseignant Chercheur, ENSTA Bretagne, Lab-STICC, Brest

  • M. SANTOS Paulo, Associate Professor, Flinders University, Adelaide

  • Mme CHAUVEAU Estelle, Docteur, Ingénieur de recherche, Naval Group, Ollioules

Abstract

Sea disturbances vary unrelentingly making it necessary for autonomous underwater vehicles (AUVs) to continuously adapt their control systems to accommodate such changes. The complexity and dynamics of underwater processes are, however, difficult to be described analytically. While adaptive control systems could potentially be used to handle such changes, the field of adaptive control faces three main challenges: the dependency on the availability of a priori knowledge of the underlying processes; the need for well defined governing equations; and the implementation of these equations on physical systems. In the context of the underwater environment, these descriptions are not readily available. The objective of this thesis was to formalize a novel learning- based adaptive control using deep reinforcement learning and adaptive pole placement control to compensate for the known part of the process and to extract information on the unknown part directly from sensors feedback so as to compensate for the unobservable current disturbance. In addition, we proposed a novel experience replay mechanism that considers the characteristic of the biological replay mechanism. The methods were validated in simulation and in real life, demonstrating the benefits of combining both theories against using them separately. Mots clés: Adaptive Control, Deep Reinforcement Learning, Robotics, Underwater Vehicles

Publications

  • Thomas Chaffre, Jonathan Wheare, Andrew Lammas, Paulo Santos, Gilles Le Chenadec, Karl Sammut, Es­telle Chauveau, and Benoit Clement (2022). Sim­to­real transfer of adaptive control parameters for improved robustness to sea current variations, under review.

  • Thomas Chaffre, Paulo E. Santos, Gilles Le Chenadec, Estelle Chauveau, Karl Sammut, and Benoit Clement (2022). Learning Stochastic Adaptive Control using a Bio­Inspired Experience Replay Experience Replay. In TechRxiv.

  • Hector Kohler, Benoit Clement, Thomas Chaffre, and Gilles Le Chenadec (2022). PID Tuning using Cross­Entropy DeepLearning: a Lyapunov Stability Analysis. In Proceedings of the 14th IFAC CAMS.

  • Thomas Chaffre, Julien Moras, Adrien Chan­Hon­Tong, Julien Marzat, Karl Sammut, Gilles Le Chenadec, and Benoit Clement (2022). Learning­based vs Model­free Adaptive Control of a MAV under Wind Gust. In Informatics in Control, Automation and Robotics pp 362–385, LNEE, SPRINGER.

  • Thomas Chaffre, Gilles Le Chenadec, Karl Sammut, Estelle Chauveau, and Benoit Clement (2021). Direct Adaptive Pole­Placement Controller using Deep Reinforcement Learning: Application to AUV Control. In Proceedings of the 13th IFAC Conference on Control Applications in Marine Systems, Robotics and Vehicles (CAMS).

  • Thomas Chaffre, Julien Moras, Adrien Chan Hon Tong, and Julien Marzat (2020). Sim­to­Real Transfer with Incremental Environment Complexity for Reinforcement Learning of Depth­Based Robot Navigation., in Proceedings of the 16th ICINCO.

  • Sola Yoann, Chaffre Thomas, Le Chenadec Gilles, Sammut Karl, and Clement Benoit (2020). Evaluation of a Deep­Reinforcement­Learning­based Controller for the Control of an Autonomous Underwater Vehicle. In Proceedings of Global Oceans 2020.

Seminar presentation

  • Thomas Chaffre. Deep reinforcement learning and transfer of adaptive control parameters for improved robustness to current disturbance. At Centre de Recherche en Automatique de Nancy (CRAN) (2022).

  • Thomas Chaffre. Learning­based adaptive control of AUVs. At Sirehna at Technocampus Ocean (2022).

  • Thomas Chaffre, Paulo E. Santos, Gilles Le Chenadec, Estelle Chauveau, Karl Sammut, and Benoit Clement. Learning Stochastic Adaptive Control using a Bio­Inspired Experience Replay Experience Replay. Poster presentation during the visit of Antoine Petit, President and CEO of CNRS, at IRL CROSSING (2022).

  • Thomas Chaffre. Learning­ based adaptive control of AUVs. At IRL CROSSING (2021).

1000