CPS: Small: Data-Driven Reinforcement Learning Control of Large CPS Networks using Multi-Stage Hierarchical Decompositions
This 3-year NSF CPS proposal seeks to solve the problem of real-time control and decision-making for large-scale complex networks with an approach utilizing hierarchical machine learning. Hierarchical control is obtained through projection matrices that create non-overlapping sets for low-rank properties of controllability grammian. Model-free reinforcement learning is used to design local and global controllers with sparsity-promoting structures to reduce communication complexity, and recurrent neural networks are trained to rapidly predict sparse projections. NC State's FREEDM Systems Center and its ExoGENI cloud computing network will be used to validate the results on simplified Japanese models with high solar penetration, a Duke Energy power grid, and IEEE standard models.
Sponsor
National Science Foundation (NSF)
The grant—running from January 1, 2020 to December 31, 2023—is for a total of $353,237.
Principle Investigators
Wenyuan Tang
Aranya Chakrabortty
More Details
In the current state-of-the-art machine learning based real-time control and decision-making in large-scale complex networks such as electric power systems is largely bottlenecked by the curse of dimensionality. Even the simplest linear quadratic regulator design demands cubic numerical complexity. The problem becomes even more complex when the network model is unknown, due to which an additional learning time needs to be accommodated. In this 3-year NSF CPS proposal, we take a new stance for solving this problem, and propose a hierarchical or nested machine learning-based scheme for real-time control of extreme-dimensional networks. Our approach will be to design appropriate projection matrices by which a network can be divided into disparate sets of non-overlapping groups depending on the low-rank properties of their controllability grammian, and multiple sets of composite controllers can be learned independently for each group using model-free reinforcement learning. Accordingly, the control goals of the network will also be decomposed into local (microscopic) and global (macroscopic) reward functions. Local controllers will be designed via privacy preserving group learning, and the global controllers via model reduction and averaging. Sparsity-promoting structures will be imposed on top of the local controllers to reduce their communication complexity. Deep learning algorithms based on historical events will be used to train recurrent neural networks so that they can rapidly predicting these sparse projections after any disturbance event in the network. Throughout this entire exercise, wide-area control of power systems using streaming Synchrophasor data from Phasor Measurements Units (PMUs) will be treated as a driving example. Results will be validated using standard IEEE models, a simplified model of the Japanese power grid with high-scale solar penetration, and an Opal-RT model of the Duke Energy power grid integrated with the ExoGENI cloud computing network at the FREEDM Systems Center.