The subject of the project formally belongs to the intersection between Machine Learning, Control and Communication Engineering, and Applied Mathematics. Treated in a unified way, these areas constitute the foundation of Intelligent Networked Cyber-Physical Systems (INCPS) which represent one of the greatest challenges in modern science and technology, with revolutionary impact.

Objectives

The general objective of the project can be formulated as development of advanced methods, algorithms and practical tools for decentralized learning and intelligent control for INCPS, based on applications of consensus and distributed optimization techniques, reinforcement learning, and model predictive and data-driven control, with a focus on specific application use cases: a multi-drone search and dispatch mission, and a smart manufacturing problem. The specific objectives are:

1. Development of data-driven Machine Learning Control (MLC) algorithms for INCPS. Novel algorithms, with advanced properties, and theoretically guaranteed improved performance indicators, will be based on two different approaches to modeling of uncertainties in the multi-agent control problems:

a) Development of novel decentralized and distributed algorithms with superior properties for multi-agent reinforcement learning. New distributed algorithms for iterative multi-agent learning of the optimal control policy in unknown Markov decision processes will be developed. The resulting algorithms will allow a group of autonomous agents to cooperatively evaluate a target control policy and search for the optimal one, independently of their local behaviour.

b) Development of novel algorithms for distributed data-based control with theoretically guaranteed stability, and improved performance indicators. These algorithms are aimed at controlling systems whose models are not easy to derive since most of the key system parameters are not known and the proper identification experiments cannot be conducted due to cost or spatial distribution. Algorithms for direct controller design from data that skip the modelling and identification step for distributed systems will be developed, without the need for the agents to fully share the available information. Also, existing results from distributed Model Predictive Control (MPC) will be used to construct novel powerful control schemes that can adapt to constant changes of the plant dynamics.

2.  Use-Cases Implementation on Real-World Systems. The developed algorithms and methods will be verified and implemented in two use-cases involving real-world testbeds.
a) A multi-drone testbed is an INCPS example of exceptional practical significance. The objective is to test the MLC algorithms developed towards Objective 1a in a multi-agent search and dispatch mission involving learning and control of the swarm of drones in an uncertain environment and carrying uncertain payloads.
b) A de-manufacturing pilot plant (DMP) will be used to demonstrate the applicability and effectiveness of the algorithms developed under Objective 1b. In particular, the algorithms will be used for distributed management and data-driven control of the multi-target, multi-pallet transportation line of the DMP, for which the centralized adaptive control is architecturally hard to implement and computationally demanding, given the complexity of the line and large number of pallets.

Work Packages

The project will be implemented through four work packages, two theoretically oriented, one oriented towards the two practical use cases, and one devoted to management, dissemination and exploitation:

WP1: Decentralized multi-agent reinforcement learning. Novel distributed algorithms with superior properties, applicable to truly decentralized multi-agent settings, for collaborative reinforcement learning in stochastic environments, modelled using the theory of Markov decision processes, will be developed, rigorously theoretically analyzed, and tested using computer simulations.

WP2: Decentralized data-driven machine learning control. Novel decentralized data-based (machine learning) control, and MPC algorithms with superior properties will be developed, rigorously theoretically analyzed, and tested using computer simulations.

WP3: Real-world use-cases. This WP deals with the two use cases, which themselves have promising potential for commercialization, technology transfer to industrial domain, and high prospective for extensions to other related applications in the area of INCPS with larger impact. It is expected to verify the theoretical results from the previous two WPs, and develop proof-of-concept type results.

WP4: Management, Dissemination and Exploitation.