Presentation Library

Abstract

The goal of reinforcement learning is at the core of the CSS mission: computation of policies that are approximately optimal, subject to information constraints. From the beginning, control foundations have lurked behind the RL curtain: Watkins’ Q-function looks suspiciously like the Hamiltonian in Pontryagin’s minimum principle, and (since Van Roy’s thesis) it has been known that our beloved adjoint operators are the key to understanding what is going on with TD-learning. This talk will briefly survey the goals and foundations of RL, and present new work showing how to dramatically accelerate convergence based on a combination of control techniques. The talk will include a wish-list of open problems in both deterministic and stochastic control settings. 

Abstract

Computer and communication technologies are rapidly developing leading to an increasingly networked and wireless world. This raises new challenging questions in the context of networked control systems, especially when the computation, communication and energy resources for control are limited. To efficiently use the available resources it is desirable to limit the control actions to instances when the system really needs attention. Unfortunately, classical time-triggered control schemes are based on performing sensing and actuation actions periodically in time (irrespective of the state of the system) rather than when the system actually needs attention. This points towards the consideration of event-triggered control as an alternative and (more) resource-aware control paradigm, as it seems natural to trigger control actions by well-designed events involving the system's state, output or any other locally available information: "To act or not to act, that is the question in event-triggered control." The objectives of this talk are to introduce the basics in the field of resource-aware control for distributed and multi-agent systems and to discuss recent advances and open questions. The focus will be on event-triggered control, although we will also touch upon self-triggered control as an alternative paradigm for resource-aware feedback control. We will show that various forms of hybrid systems, combining continuous and discrete dynamics, play instrumental roles in the analysis and the design of event-triggered and self-triggered controllers. The main developments will be illustrated in the context of cooperative driving exploiting wireless communication. The effects of delays, packet losses and (denial-of-service) attacks on the event-triggered cooperative adaptive cruise control (CACC) strategies for vehicle platooning will be discussed and experimental results will be presented. 

Abstract

Feedback is a key element of regulation, as it shapes the sensitivity of a process to its environment. Positive feedback up-regulates, negative feedback down-regulates. Many regulatory processes involve a mixture of both, whether in nature or in engineering. This paper revisits the mixed feedback paradigm, with the aim of investigating control across scales. We propose that mixed feedback regulates excitability and that excitability plays a central role in multi-scale signalling. We analyse this role in a multi-scale network architecture inspired from neurophysiology. The nodal behavior defines a meso-scale that connects actuation at the micro-scale to measurements at the macro-scale. We show that mixed-feedback control at the nodal scale provides regulatory principles at the network scale, with a nodal resolution. In this sense, the mixed feedback paradigm is a control principle across scales. 

Abstract

As the characteristic size of a flying robot decreases, the challenges for successful flight revert to basic questions of fabrication, actuation, fluid mechanics, stabilization, and power — whereas such questions have in general been answered for larger aircraft. When developing a robot on the scale of a housefly, all hardware must be developed from scratch as there is nothing “off-the-shelf” which can be used for mechanisms, sensors, or computation that would satisfy the extreme mass and power limitations. With these challenges in mind, this talk will present progress in the essential technologies for insect-scale robots and the latest flight experiments with robotic insects.

Abstract

The term capacity has natural connotations about fundamental limits and robustness to disruptions. For engineered systems, a rigorous characterization of capacity also provides insight into algorithms with universal performance guarantees and informs optimal strategic resource allocation. We present analysis and optimization of capacity and related performance metrics for societal cyber-physical systems (including traffic, mobility, and power networks) in canonical settings. At the macroscopic scale, we extend static network flow formulations to several flow dynamics and control settings (including cascading failure). The tractability of the resulting nonlinear analysis and optimization is facilitated by the spatial sparsity of dynamics and invariance of key input-output properties, such as monotonicity, across multiple resolutions in the network. At the microscopic scale, we consider spatial queues with state-dependent service rate; for example, such problems arise in networks of dynamically coupled vehicles. While this dependence is complex in general, we provide tight characterization in limiting cases, for instance large queue length, which leads to tight throughput estimates. Case studies are provided to evaluate the effectiveness of the proposed frameworks.

Abstract

With its thin atmosphere, uncertain wind and terrain, and ever-increasing science requirements, robotic missions to the surface of Mars have presented enormous challenges to the Entry, Descent, and Landing (EDL) and Guidance, Navigation, and Control (GN&C) engineers. Throughout the years, these challenges have been met with a series of landing architectures that spawned different degrees of passive and active control, from the ballistic airbag landers in the Mars Pathfinder and Spirit and Opportunity Rovers, to the guided-entry, SkyCrane-delivered Curiosity. Landing on Europa (the Jovian moon that may have the conditions to harbor life) presents a different set of challenges over a Martian landing. While Europa’s lack of atmosphere relieves the landing engineer from the complexities of heatshields and parachutes and the vagaries of an atmosphere, they now face the enormous challenges of bringing large amounts of fuel and powerful propulsion to do the job the atmosphere does on Mars, while dealing with an extremely uncertain surface topography and radiation environment. These challenges are being addressed with increased automation based on new GN&C sensors and algorithms. In this talk, I will describe the challenges of both Mars and Europa landings, and the intellectual journey trod by engineers in meeting them.

Abstract

General anesthesia is a drug-induced, reversible condition comprised of five behavioral states: unconsciousness, amnesia (loss of memory), analgesia (loss of pain sensation), akinesia (immobility), and maintenance of physiological stability and control of the stress response. As a consequence, every time an anesthesiologist administers anesthesia he/she creates a control system with a human in the loop. Our work shows that a primary mechanism through which anesthetics create these altered states of arousal is by initiating and maintaining highly structured oscillations. These oscillations impair communication among brain regions. We show how these dynamics change systematically with different anesthetic classes, anesthetic dose and with age. As a consequence, we have developed a principled, neuroscience-based paradigm for using the EEG to monitor the brain states of patients receiving general anesthesia and for implementing formal control strategies for maintaining anesthetic state. We will illustrate these strategies with results from actual control experiments.

Abstract

In many application domains, including systems and control theory, the optimization problems that appear are seldom "generic" but instead they often have well-defined structural features. Depending on the situation, such structure may be described algebraically (e.g., by transformations under which the problem is invariant, like linearity or time-invariance), geometrically (by restricting the feasible set to a given manifold/variety), or graphically (e.g., by a graph summarizing the interactions among decision variables). Exploiting this structure is crucial for practical efficiency. In this talk we will provide a gentle introduction to these ideas, surveying the basic notions as well as describing algorithmic techniques to detect and exploit these properties. In particular, we will discuss some recent developments, including dimension/symmetry reduction techniques for SDPs, and chordal networks. As we will illustrate through applications, algorithms that automatically exploit structure can significantly outperform existing techniques. 

Abstract

The 25th anniversary of the commercialization of lithium-ion batteries marks their wide-spread use in handheld consumer electronics and coincides with a period of intense efforts for powering electric vehicles. Managing the potent brew of lithium ions in the large quantities necessary for vehicle propulsion is anything but straightforward. Designing the complex conductive structure, choosing the electrode material for locking the energy in high potential states and synthesizing the interfaces for releasing the chemical energy at fast but controllable rates has been the focus of the electrochemists and material scientists. But from the Rosetta-Philae spacecraft landing three billion miles away from Earth to the daily commute of a hybrid electric automobile, the control engineers behind the battery management system (BMS) have been the unsung heroes. The BMS is the brain of the battery system and is responsible for State of Charge (SOC), State of Health (SOH) and State of Power (SOP) estimation while protecting the cell by limiting its power. The BMS relies on accurate prediction of complex electrochemical, thermal and mechanical phenomena. This raises the question of model and parameter accuracy. Moreover, if the cells are aging, which parameters should we adapt after leveraging limited sensor information from the measured terminal voltage and sparse surface temperatures? With such a frugal sensor set, what is the optimal sensor placement? To this end, control techniques and novel sensors that measure the cell swelling during lithium intercalation and thermal expansion will be presented. We will conclude by highlighting the fundamental difficulties that keep every battery control engineer awake, namely predicting local hot spots, detecting internal shorts, and managing the overwhelming energy released during a thermal runaway. 

Abstract

Recent work on Model Predictive Control has refocused attention on the role of future disturbance uncertainty. One way of dealing with this issue is to use policy rather than sequence optimization. However, this comes at a significant increase in computational burden. In this talk we will outline strategies for dealing with the computational issue, including using quantized scenarios to represent the future disturbances. The related issue of providing performance guarantees in the face of high uncertainty will also be discussed. The ideas will be illustrated by the development of a new treatment strategy for Type 1 diabetes mellitus. 

More information provided here.

Abstract

The area of polynomial optimization has been actively studied in computer science, operations research, applied mathematics and engineering, where the goal is to find a high-quality solution using an efficient computational method. This area has attracted much attention in the control community since several long-standing control problems could be converted to polynomial optimization problems. The current researches on this area have been mostly focused on various important questions: i) how does the underlying structure of an optimization problem affect its complexity? Ii) how does sparsity help? iii) how to find a near globally optimal solution whenever it is hard to find a global minimum? iv) how to design an efficient numerical algorithm for large-scale non-convex optimization problems? v) how to deal with problems with a mix of continuous and discrete variables? In this talk, we will develop a unified mathematical framework to study the above problems. Our framework rests on recent advances in graph theory and optimization, including the notions of OS-vertex sequence and treewidth, matrix completion, semidefinite programming, and low-rank optimization. We will also apply our results to two areas of power systems and distributed control. In particular, we will discuss how our results could be used to address several hard problems for power systems such as optimal power flow (OPF), security-constrained OPF, state estimation, and unit commitment.

Abstract

The number of unmanned aerial vehicles or drones has grown exponentially over the last three decades. Yet we are only now seeing autonomous flying robots that can operate in three-dimensional indoor environments and in outdoor environments without GPS. I will discuss the need for smaller, safer, smarter, and faster flying robots and the challenges in control, planning, and coordinating swarms of robots with applications to search and rescue, first response and precision farming. Publications and videos are available at kumarrobotics.org.

Abstract

The area of polynomial optimization has been actively studied in computer science, operations research, applied mathematics and engineering, where the goal is to find a high-quality solution using an efficient computational method. This area has attracted much attention in the control community since several long-standing control problems could be converted to polynomial optimization problems. The current researches on this area have been mostly focused on various important questions: i) how does the underlying structure of an optimization problem affect its complexity? Ii) how does sparsity help? iii) how to find a near globally optimal solution whenever it is hard to find a global minimum? iv) how to design an efficient numerical algorithm for large-scale non-convex optimization problems? v) how to deal with problems with a mix of continuous and discrete variables? In this talk, we will develop a unified mathematical framework to study the above problems. Our framework rests on recent advances in graph theory and optimization, including the notions of OS-vertex sequence and treewidth, matrix completion, semidefinite programming, and low-rank optimization. We will also apply our results to two areas of power systems and distributed control. In particular, we will discuss how our results could be used to address several hard problems for power systems such as optimal power flow (OPF), security-constrained OPF, state estimation, and unit commitment.

Abstract

Network systems have received a lot of attention in the past decade. They are used to analyze and design communication network, smart grid technology, social media, social dynamics, formation and consensus problems, etc. Several analysis and control methods have been developed for network systems. However, often, their large scale nature makes it difficult to analyze and to design a controller. We develop methods to reduce the order of the network while preserving the network structure, as well as some structure of the (linear) node dynamics. In particular, second order network dynamics structure is preserved. We use node clustering methods, as well as a state space singular value decomposition based method. For the first we provide error bounds. We illustrate the results with help of some relevant high order examples.

Abstract

Model predictive control has become a pervasive advanced control technology in which optimal control of a multivariable system with input and state constraints is combined with a moving horizon to produce a feedback controller. In applications, model predictive control is often used to solve constrained tracking problems. The tracking problem arises in some settings as the basic goal of the control system, and the constraint handling capabilities of MPC are what make it attractive. In other applications, however, there may be a higher-level goal, such as economic optimization of a process, and this goal is first translated into a steady-state tracking problem. Since MPC enables the designer to choose the objective function that is optimized online, it offers the potential to treat the higher-level control goal directly within the MPC controller bypassing this translation into a steady-state setpoint and tracking problem. In this talk we explore the possibilities enabled by MPC to address these types of high-level goals. We also outline some of the open research challenges presented by this approach; these include modeling, optimization, and controller design challenges. The talk concludes with a brief presentation of a recently deployed economic optimization technology developed by Johnson Controls to control the campus energy system at Stanford University.

Abstract

Distributed and large-scale optimization problems have gained a significant attention in the context of cyber-physical, peer-to-peer, and ad-hoc networked systems. The large-scale property is reflected in the number of decision variables, the number of constraints, or both, while the distributed nature of the problems is inherent due to partial (local) knowledge of the problem data (e.g., a portion of the cost function or a subset of the constraints is known to different entities in the system). The talk will focus on some recent developments on optimization models and algorithmic approaches for solving such problems with applications in domains ranging from control to machine learning. 

Abstract

Smart Cities are an example of Cyber-Physical Systems whose goals include improvements in transportation, energy distribution, emergency response, and infrastructure maintenance, to name a few. One of the key elements of a Smart City is the ability to monitor and dynamically allocate its resources. The availability of large amounts of data, ubiquitous wireless connectivity, and the critical need for scalability open the door for new control and optimization methods which are both data-driven and event-driven. The talk will present such an optimization framework and its properties. It will then describe several applications that arise in Smart Cities, some of which have been tested in the City of Boston: a “Smart Parking” system which dynamically assigns and reserves an optimal parking space for a user (driver); the “Street Bump” system which uses standard smartphone capabilities to collect roadway obstacle data and identify and classify them for efficient maintenance and repair; adaptive traffic light control; optimal control of connected autonomous vehicles. Lastly, to address the “social’’ dimension, the talk will describe how a large traffic data set from the Massachusetts road network was analyzed to estimate the Price of Anarchy in comparing “selfish” user-centric behavior to “social” system-centric optimal traffic routing solutions.

Abstract

This talk examines the transient modeling of power flow for transient thermal systems. The focus is on dynamic phenomena starting with a basic thermodynamic cycle and building up to more complex systems. The overall goal of the modeling process is to develop systems-level models that are sufficiently flexible to be used on a variety of different applications. These models balance complexity with accuracy to obtain models that are sufficient for dynamic optimization and design as well as control algorithms 
In addition to the modeling approach we present control strategies aimed at managing the flow of thermal power. We present a particular hierarchical approach to power flow that accommodates multiple power modes. The hierarchy allows for systems operating on different time scales to be coordinated. It also allows for different control tools to be used at different levels of the hierarchy based on the needs of the physical systems under control. Stability results exploit the system structure to provide guarantees. Recent results will be presented representing both interconnected complex systems with specific examples from industrial partners. 

Abstract

At the quantum level, feedback loops have to take into account measurement back-action. The goal of this talk is to explain, in a tutorial way and on the first experimental realization of a quantum-state feedback, how such purely quantum effect can be exploited in models and stabilization control schemes. This closed-loop experiment was conducted in 2011 by the group of Serge Haroche (Physics Nobel Prize 2012). The control goal was to stabilize a small number of micro-wave photons trapped between two super-conducting mirrors and subject to quantum non-demolition measurement via probe off-resonant Rydberg atoms. The implemented control scheme was decomposed into two parts. The first part estimates in real-time the quantum state of the trapped photons via a discrete-time Belavkin quantum filter. The second part is a nonlinear quantum-state feedback based on control Lyapunov functions. It stabilizes via suitable coherent displacements the number of photon(s) towards its set-point, namely an integer less than 5 in the experiment. This control scheme relies on a hidden control Markov model whose structure combines three quantum rules: unitary deterministic Schrödinger evolution; stochastic collapse of the wave packet induced by the measurement; tensor product for the composite systems. These basic quantum rules characterize the structure of all Markovian models describing open-quantum systems. These rules explain also the existence to two kinds of feedback schemes currently developed for quantum error correction: measurement-based feedback where an open quantum system is stabilized by a classical controller; coherent or autonomous feedback (reservoir engineering) where an open quantum system is passively stabilized through its coupling with another highly dissipative quantum system, namely the quantum controller.