Model Reference Tracking Control Solutions for a Visual Servo System Based on a Virtual State from Unknown Dynamics

Timotei Lala, Darius-Pavel Chirla and Mircea-Bogdan Radac
Additional contact information
Timotei Lala: Department of Automation and Applied Informatics, Politehnica University of Timisoara, 300223 Timisoara, Romania
Darius-Pavel Chirla: Department of Automation and Applied Informatics, Politehnica University of Timisoara, 300223 Timisoara, Romania
Mircea-Bogdan Radac: Department of Automation and Applied Informatics, Politehnica University of Timisoara, 300223 Timisoara, Romania

Energies, 2021, vol. 15, issue 1, 1-25

Abstract: This paper focuses on validating a model-free Value Iteration Reinforcement Learning (MFVI-RL) control solution on a visual servo tracking system in a comprehensive manner starting from theoretical convergence analysis to detailed hardware and software implementation. Learning is based on a virtual state representation reconstructed from input-output (I/O) system samples under nonlinear observability and unknown dynamics assumptions, while the goal is to ensure linear output reference model (ORM) tracking. Secondary, a competitive model-free Virtual State-Feedback Reference Tuning (VSFRT) is learned from the same I/O data using the same virtual state representation, demonstrating the framework’s learning capability. A model-based two degrees-of-freedom (2DOF) output feedback controller serving as a comparisons baseline is designed and tuned using an identified system model. With similar complexity and linear controller structure, MFVI-RL is shown to be superior, confirming that the model-based design issue of poor identified system model and control performance degradation can be solved in a direct data-driven style. Apart from establishing a formal connection between output feedback control, state feedback control and also between classical control and artificial intelligence methods, the results also point out several practical trade-offs, such as I/O data exploration quality and control performance leverage with data volume, control goal and controller complexity.

Keywords: reinforcement learning and approximate dynamic programming; virtual state feedback reference tuning; model reference control; unknown dynamics; input-output observable system; visual servo; image processing (search for similar items in EconPapers)
JEL-codes: Q Q0 Q4 Q40 Q41 Q42 Q43 Q47 Q48 Q49 (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1996-1073/15/1/267/pdf (application/pdf)
https://www.mdpi.com/1996-1073/15/1/267/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jeners:v:15:y:2021:i:1:p:267-:d:715529

Access Statistics for this article

Energies is currently edited by Ms. Agatha Cao

More articles in Energies from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().