Elise Miller-Hooks

Deep reinforcement learning for multi-asset infrastructure management incorporating traffic operations adaptations and control

Project Abstract

proj 69
Andriotis, C.P. and Papakonstantinou, K.G. (2020). Deep reinforcement learning driven inspection and maintenance planning under incomplete information and constraints, arXiv preprint arXiv:2007.01380.

Preservation of structural integrity through inspections and maintenance (I&M) constitutes a recurrent task and its execution involves a complex set of decisions optimally placed in space and time. At regional scales, optimal management of entire networks renders such decisions particularly challenging due to the large number of involved assets that have intrinsic dependencies and correlations. In a transportation network these dependencies are, among others, manifested through the global effects that I&M policies have on the traffic this network supports, since the involved assets, with their individual contributions to capacity, affect system performance jointly and in a nonlinear manner. As such, determining the potential impacts of combinations of I&M decisions, each associated with an individual asset, becomes a complicated process. For this purpose, for example, user equilibrium concepts are often applied in predicting the impact of such actions on traffic flows in congested environments. Embedding this conceptualization results in models that are unavoidably bound by this nonlinearity and interdependencies among components. These interdependencies can be characterized as a measure of system correlation among the network components. In addition to this, the different assets are also parametrically correlated, due to material, structural type, local geological conditions, and traffic levels due to events and time of day. From a modeling perspective, this is reflected in statistical correlation of constitutive properties and responses of different assets.

Both system and statistical correlations are key for optimal decision-making, implicitly introducing correlation to the optimal I&M actions. This project will develop methodologies to quantify the effects of both system and statistical correlations in I&M decision-making processes by integrating advanced statistical learning and inference methods with machine learning and artificial intelligence algorithms in a synergistic computational platform. To this end, a framework that integrates probabilistic methods, such as hidden Markov and Bayesian inference models, with machine learning techniques, such as Partially Observable Markov Decision Processes (POMDPs) and multi-agent Deep Reinforcement Learning (DRL), will be developed. Through this framework, solutions can scale in high-dimensional spaces and long life-cycle horizons, whereas related system-level correlations, e.g. due to traffic patterns, can be holistically incorporated in the optimization process.

Award Period:
Source of Funding:
Center for Integrated Asset Management for Multi-modal Transportation Infrastructure Systems (CIAMTIS): Region 3 Univ Transportation Ctr
Co-PI: Elise Miller-Hooks, GMU
PI – Kostas Papakonstantinou, Penn State and
Co-PI Shelley Stoffels, Penn State
Total Award Amount:
$192,711 ($66,666 to George Mason University)

Project 69


Elise Miller-Hooks, Ph.D.
Bill & Eleanor Hazel Chair in Infrastructure Engineering

Phone: 703.993.1685
Email: miller@gmu.edu

Office: 4614 Nguyen Engineering Building

Sid and Reva Dewberry Department of Civil, Environmental and Infrastructure Engineering
George Mason University
4400 University Drive, MS 6C1
Fairfax, VA 22030


More Projects


Volgenau School of Engineering
George Mason University