IEEE.org | IEEE Xplore Digital Library | IEEE Standards | IEEE Spectrum | More Sites
Call for Award Nominations
Optimal controllers for linear or nonlinear dynamic systems with known dynamics can be designed by using Riccati and Hamilton-Jacobi-Bellman (HJB) equation respectively. However, optimal control of uncertain linear or nonlinear dynamic systems is a major challenge. Moreover, controllers designed in discrete-time have the important advantage that they can be directly implemented in digital form using modern-day embedded hardware. Unfortunately, discrete-time design using Lyapunov stability analysis is far more complex than the continuous-time counterpart since the first difference in Lyapunov function is quadratic in the states and not linear as in the case of continuous-time. By incorporating learning features with the feedback controller design, optimal adaptive control of such uncertain dynamical systems in discrete-time can be solved.
In this talk, an overview of first and second-generation feedback controllers with a learning component in discrete-time will be discussed. Subsequently, the discrete-time learning-based optimal adaptive control of uncertain nonlinear dynamic systems will be presented in a systematic manner using a forward in time approach based on reinforcement learning (RL)/approximate dynamic programming (ADP). Challenges in developing and implementing the three generations of learning controllers will be addressed using practical examples such as automotive engine emission control, robotics, and others. We will argue that discrete-time controller development is preferred for transitioning the developed theory to practice. Today, the application of learning controllers can be found in areas as diverse as process control, energy or smart grids, civil infrastructure, healthcare, manufacturing, automotive, transportation, entertainment, and consumer appliances. The talk will conclude with a short discussion of open research problems in the area of learning control.
Security and privacy are of growing concern in many control applications. Cyber attacks are frequently reported for a variety of industrial and infrastructure systems. For more than a decade the control community has developed techniques for how to design control systems resilient to cyber-physical attacks. In this talk, we will review some of these results. In particular, as cyber and physical components of networked control systems are tightly interconnected, it is argued that traditional IT security focusing only on the cyber part does not provide appropriate solutions. Modeling the objectives and resources of the adversary together with the plant and control dynamics is shown to be essential. The consequences of common attack scenarios, such as denial-of-service, replay, and bias injection attacks, can be analyzed using the framework presented. It is also shown how to strengthen the control loops by deriving security- and privacy-aware estimation and control schemes. Applications in building automation, power networks, and automotive systems will be used to motivate and illustrate the results. The presentation is based on joint work with several students and colleagues at KTH and elsewhere.
Advances in computing and networking technologies have connected manufacturing systems from the lowest levels of sensors and actuators, across the factory, through the supply chain, and beyond. Large amounts of data have always been available to these systems, with currents and velocities sampled at regular intervals and used to make control decisions, and throughputs tracked hourly or daily. The ability to collect and save this detailed low-level data, send it to a central repository, and store it for days, months, and years, enables better insight into the behavior – and misbehavior – of complex manufacturing systems. The output from high-fidelity models and/or reams of historical data can be compared with streams of data coming off the plant floor to identify anomalies. Early identification of anomalies, before they lead to poor quality products or machine failure, can result in significant productivity improvements. We will discuss multiple approaches for harnessing this data, leveraging both physics-based and data-driven models, and how automation can enable timely responses. Both simulation and experimental results will be presented.
Cyber-physical systems are the basis for the new industrial revolution. Growing energy demand and environmental concerns require a large number of renewable energy resources, efficient energy consumption, and energy storage devices and demand responses. Cyber-physical energy systems (CPES) in a broader sense provide a desirable infrastructure for efficient energy production and consumption with uncertain energy resources. This speech will focus on the structure of CPES, and the problem of security-constrained planning and scheduling of CPES, including new renewable energy sources with high levels of uncertainties. The newly developed analytical conditions are discussed for fast identifying the security bottlenecks in a complex power grid when new renewable energy sources coordinate with storable energy sources such as hydro and pumped storages. A new method is introduced to solve the well-knows N-k contingency security assessment problem with 2-3 orders of reduced computational complexity. Production, storage and transportation, and utilization of hydrogen as the main energy source are also introduced. It is shown that the hydrogen-based CPES will provide an ideal infrastructure for energy supply and consumption with almost no pollution, and it will likely lead to the energy revolution anticipated in the new century.
In the six decades of conventional TUNING-BASED adaptive control, the unattained fundamental goals, in the absence of detrimental artificial excitation, have been (1) exponential regulation, as with robust controllers, and (2) perfect learning of the plant model. More than a quarter-century since I started my career by extending conventional adaptive controllers from linear to nonlinear systems, I reach those decades-old goals with a new non-tuning paradigm: regulation-triggered batch identification. The parameter estimate in the controller is held constant and, only once the regulation error grows “too large,” a parameter estimate update, based on the data since the last update, is “triggered.” Such a simple parameter estimator provably, and remarkably terminates updating after a number of state growth-triggered updates which is no greater than the number of unknown parameters. This yields exponential regulation and perfect identification except for zero-measure initial conditions. I present a design for a more general class of nonlinear systems than ever before, an extension to adaptive PDE control, a flight control example (the “wing rock” instability), and, time permitting, a simple robotics example. This is joint work with Iasson Karafyllis from the Mathematics Department of the National Technical University of Athens.
Recent years have seen a great progress in the area of robotics. Communication signals are also ubiquitous these days. In this talk, I will explore the opportunities and challenges at this intersection, for robotic sensing and communication. In the first part of the talk, I will focus on robotic sensing, and ask the following question "Can everyday communication signals, such as WiFi signals, give new sensing capabilities to unmanned vehicles?" For instance, imagine two unmanned vehicles arriving behind thick concrete walls. Can they image every square inch of the invisible area through the walls with only WiFi signals? I will show that this is indeed possible, and discuss how our methodology for the co-optimization of path planning and communication has enabled the first demonstration of 3D imaging through walls with only drones and WiFi. I will also discuss other new sensing capabilities that have emerged from our approach, such as occupancy estimation and crowd analytics with only WiFi signals. In the second part of the talk, I will focus on communication-aware robotics, a term coined to refer to robotic systems that explicitly take communication issues into account in their decision making. This is an emerging area of research that not only allows a team of unmanned vehicles to attain the desired connectivity during their operation, but can also extend the connectivity of the existing communication systems through the use of mobility. I will then discuss our latest theoretical and experimental results along this line. I will show how each robot can go beyond the over-simplified but commonly-used disk model for connectivity, and realistically model the impact of channel uncertainty for the purpose of path planning. I will then show how the unmanned vehicles can properly co-optimize their communication, sensing and navigation objectives under resource constraints. This co-optimized approach can result in a significant performance improvement and resource saving, as we shall see. I will also discuss the role of human collaboration in these networks.
The goal of reinforcement learning is at the core of the CSS mission: computation of policies that are approximately optimal, subject to information constraints. From the beginning, control foundations have lurked behind the RL curtain: Watkins’ Q-function looks suspiciously like the Hamiltonian in Pontryagin’s minimum principle, and (since Van Roy’s thesis) it has been known that our beloved adjoint operators are the key to understanding what is going on with TD-learning. This talk will briefly survey the goals and foundations of RL, and present new work showing how to dramatically accelerate convergence based on a combination of control techniques. The talk will include a wish-list of open problems in both deterministic and stochastic control settings.
Computer and communication technologies are rapidly developing leading to an increasingly networked and wireless world. This raises new challenging questions in the context of networked control systems, especially when the computation, communication and energy resources for control are limited. To efficiently use the available resources it is desirable to limit the control actions to instances when the system really needs attention. Unfortunately, classical time-triggered control schemes are based on performing sensing and actuation actions periodically in time (irrespective of the state of the system) rather than when the system actually needs attention. This points towards the consideration of event-triggered control as an alternative and (more) resource-aware control paradigm, as it seems natural to trigger control actions by well-designed events involving the system's state, output or any other locally available information: "To act or not to act, that is the question in event-triggered control." The objectives of this talk are to introduce the basics in the field of resource-aware control for distributed and multi-agent systems and to discuss recent advances and open questions. The focus will be on event-triggered control, although we will also touch upon self-triggered control as an alternative paradigm for resource-aware feedback control. We will show that various forms of hybrid systems, combining continuous and discrete dynamics, play instrumental roles in the analysis and the design of event-triggered and self-triggered controllers. The main developments will be illustrated in the context of cooperative driving exploiting wireless communication. The effects of delays, packet losses and (denial-of-service) attacks on the event-triggered cooperative adaptive cruise control (CACC) strategies for vehicle platooning will be discussed and experimental results will be presented.
Feedback is a key element of regulation, as it shapes the sensitivity of a process to its environment. Positive feedback up-regulates, negative feedback down-regulates. Many regulatory processes involve a mixture of both, whether in nature or in engineering. This paper revisits the mixed feedback paradigm, with the aim of investigating control across scales. We propose that mixed feedback regulates excitability and that excitability plays a central role in multi-scale signalling. We analyse this role in a multi-scale network architecture inspired from neurophysiology. The nodal behavior defines a meso-scale that connects actuation at the micro-scale to measurements at the macro-scale. We show that mixed-feedback control at the nodal scale provides regulatory principles at the network scale, with a nodal resolution. In this sense, the mixed feedback paradigm is a control principle across scales.
General anesthesia is a drug-induced, reversible condition comprised of five behavioral states: unconsciousness, amnesia (loss of memory), analgesia (loss of pain sensation), akinesia (immobility), and maintenance of physiological stability and control of the stress response. As a consequence, every time an anesthesiologist administers anesthesia he/she creates a control system with a human in the loop. Our work shows that a primary mechanism through which anesthetics create these altered states of arousal is by initiating and maintaining highly structured oscillations. These oscillations impair communication among brain regions. We show how these dynamics change systematically with different anesthetic classes, anesthetic dose and with age. As a consequence, we have developed a principled, neuroscience-based paradigm for using the EEG to monitor the brain states of patients receiving general anesthesia and for implementing formal control strategies for maintaining anesthetic state. We will illustrate these strategies with results from actual control experiments.
As the characteristic size of a flying robot decreases, the challenges for successful flight revert to basic questions of fabrication, actuation, fluid mechanics, stabilization, and power — whereas such questions have in general been answered for larger aircraft. When developing a robot on the scale of a housefly, all hardware must be developed from scratch as there is nothing “off-the-shelf” which can be used for mechanisms, sensors, or computation that would satisfy the extreme mass and power limitations. With these challenges in mind, this talk will present progress in the essential technologies for insect-scale robots and the latest flight experiments with robotic insects.
The term capacity has natural connotations about fundamental limits and robustness to disruptions. For engineered systems, a rigorous characterization of capacity also provides insight into algorithms with universal performance guarantees and informs optimal strategic resource allocation. We present analysis and optimization of capacity and related performance metrics for societal cyber-physical systems (including traffic, mobility, and power networks) in canonical settings. At the macroscopic scale, we extend static network flow formulations to several flow dynamics and control settings (including cascading failure). The tractability of the resulting nonlinear analysis and optimization is facilitated by the spatial sparsity of dynamics and invariance of key input-output properties, such as monotonicity, across multiple resolutions in the network. At the microscopic scale, we consider spatial queues with state-dependent service rate; for example, such problems arise in networks of dynamically coupled vehicles. While this dependence is complex in general, we provide tight characterization in limiting cases, for instance large queue length, which leads to tight throughput estimates. Case studies are provided to evaluate the effectiveness of the proposed frameworks.
With its thin atmosphere, uncertain wind and terrain, and ever-increasing science requirements, robotic missions to the surface of Mars have presented enormous challenges to the Entry, Descent, and Landing (EDL) and Guidance, Navigation, and Control (GN&C) engineers. Throughout the years, these challenges have been met with a series of landing architectures that spawned different degrees of passive and active control, from the ballistic airbag landers in the Mars Pathfinder and Spirit and Opportunity Rovers, to the guided-entry, SkyCrane-delivered Curiosity. Landing on Europa (the Jovian moon that may have the conditions to harbor life) presents a different set of challenges over a Martian landing. While Europa’s lack of atmosphere relieves the landing engineer from the complexities of heatshields and parachutes and the vagaries of an atmosphere, they now face the enormous challenges of bringing large amounts of fuel and powerful propulsion to do the job the atmosphere does on Mars, while dealing with an extremely uncertain surface topography and radiation environment. These challenges are being addressed with increased automation based on new GN&C sensors and algorithms. In this talk, I will describe the challenges of both Mars and Europa landings, and the intellectual journey trod by engineers in meeting them.
Feedback is as ubiquitous in nature as it is in design. So control theory can help us understand both natural and designed systems. Even better, generalized models abstracted from nature give us a mathematical means to connect control theoretic explanations of nature with opportunities in control design. Control theory is enriched by the language, questions, and perspectives of fields as diverse as animal behavior, cognitive science, and dance. I will present a model for multi-agent dynamics that is informed by these fields. The model derives from principles of symmetry and bifurcation, which exploit instability to recover the remarkable capacity of natural groups to trade off flexibility and stability.
In many application domains, including systems and control theory, the optimization problems that appear are seldom "generic" but instead they often have well-defined structural features. Depending on the situation, such structure may be described algebraically (e.g., by transformations under which the problem is invariant, like linearity or time-invariance), geometrically (by restricting the feasible set to a given manifold/variety), or graphically (e.g., by a graph summarizing the interactions among decision variables). Exploiting this structure is crucial for practical efficiency. In this talk we will provide a gentle introduction to these ideas, surveying the basic notions as well as describing algorithmic techniques to detect and exploit these properties. In particular, we will discuss some recent developments, including dimension/symmetry reduction techniques for SDPs, and chordal networks. As we will illustrate through applications, algorithms that automatically exploit structure can significantly outperform existing techniques.
The 25th anniversary of the commercialization of lithium-ion batteries marks their wide-spread use in handheld consumer electronics and coincides with a period of intense efforts for powering electric vehicles. Managing the potent brew of lithium ions in the large quantities necessary for vehicle propulsion is anything but straightforward. Designing the complex conductive structure, choosing the electrode material for locking the energy in high potential states and synthesizing the interfaces for releasing the chemical energy at fast but controllable rates has been the focus of the electrochemists and material scientists. But from the Rosetta-Philae spacecraft landing three billion miles away from Earth to the daily commute of a hybrid electric automobile, the control engineers behind the battery management system (BMS) have been the unsung heroes. The BMS is the brain of the battery system and is responsible for State of Charge (SOC), State of Health (SOH) and State of Power (SOP) estimation while protecting the cell by limiting its power. The BMS relies on accurate prediction of complex electrochemical, thermal and mechanical phenomena. This raises the question of model and parameter accuracy. Moreover, if the cells are aging, which parameters should we adapt after leveraging limited sensor information from the measured terminal voltage and sparse surface temperatures? With such a frugal sensor set, what is the optimal sensor placement? To this end, control techniques and novel sensors that measure the cell swelling during lithium intercalation and thermal expansion will be presented. We will conclude by highlighting the fundamental difficulties that keep every battery control engineer awake, namely predicting local hot spots, detecting internal shorts, and managing the overwhelming energy released during a thermal runaway.
Recent work on Model Predictive Control has refocused attention on the role of future disturbance uncertainty. One way of dealing with this issue is to use policy rather than sequence optimization. However, this comes at a significant increase in computational burden. In this talk we will outline strategies for dealing with the computational issue, including using quantized scenarios to represent the future disturbances. The related issue of providing performance guarantees in the face of high uncertainty will also be discussed. The ideas will be illustrated by the development of a new treatment strategy for Type 1 diabetes mellitus.
More information provided here.
Electricity production in the US has changed dramatically since 2000, with the percent of electricity produced from gas growing from 16% to 30%, while coal dropped from 52% to 37%. These changes are primarily driven by two technologies used in shale rock formations, directional drilling to create horizontal wells, and hydraulic fracturing to release the gas within the relatively impermeable rock. This presentation will first give a brief operational overview of hydraulic fracturing. Next, challenges that relate to the control of this technology are described. Lastly, two examples are presented, one a theoretical study investigating the potential of model-based feedback control of the hydraulic fracturing process and the other an implementation that highlights the importance of measurements and data uncertainty when designing effective and robust controllers.
The area of polynomial optimization has been actively studied in computer science, operations research, applied mathematics and engineering, where the goal is to find a high-quality solution using an efficient computational method. This area has attracted much attention in the control community since several long-standing control problems could be converted to polynomial optimization problems. The current researches on this area have been mostly focused on various important questions: i) how does the underlying structure of an optimization problem affect its complexity? Ii) how does sparsity help? iii) how to find a near globally optimal solution whenever it is hard to find a global minimum? iv) how to design an efficient numerical algorithm for large-scale non-convex optimization problems? v) how to deal with problems with a mix of continuous and discrete variables? In this talk, we will develop a unified mathematical framework to study the above problems. Our framework rests on recent advances in graph theory and optimization, including the notions of OS-vertex sequence and treewidth, matrix completion, semidefinite programming, and low-rank optimization. We will also apply our results to two areas of power systems and distributed control. In particular, we will discuss how our results could be used to address several hard problems for power systems such as optimal power flow (OPF), security-constrained OPF, state estimation, and unit commitment.