Multi-asset closed-loop reservoir management using deep reinforcement learning
From MaRDI portal
Abstract: Closed-loop reservoir management (CLRM), in which history matching and production optimization are performed multiple times over the life of an asset, can provide significant improvement in the specified objective. These procedures are computationally expensive due to the large number of flow simulations required for data assimilation and optimization. Existing CLRM procedures are applied asset by asset, without utilizing information that could be useful over a range assets. Here, we develop a CLRM framework for multiple assets with varying numbers of wells. We use deep reinforcement learning to train a single global control policy that is applicable for all assets considered. The new framework is an extension of a recently introduced control policy methodology for individual assets. Embedding layers are incorporated into the representation to handle the different numbers of decision variables that arise for the different assets. Because the global control policy learns a unified representation of useful features from multiple assets, it is less expensive to construct than asset-by-asset training (we observe about 3x speedup in our examples). The production optimization problem includes a relative-change constraint on the well settings, which renders the results suitable for practical use. We apply the multi-asset CLRM framework to 2D and 3D water-flooding examples. In both cases, four assets with different well counts, well configurations, and geostatistical descriptions are considered. Numerical experiments demonstrate that the global control policy provides objective function values, for both the 2D and 3D cases, that are nearly identical to those from control policies trained individually for each asset. This promising finding suggests that multi-asset CLRM may indeed represent a viable practical strategy.
Recommendations
- Deep reinforcement learning for optimal well control in subsurface systems with uncertain geology
- Convolutional -- recurrent neural network proxy for robust optimization and closed-loop reservoir management
- Comprehensive framework for gradient-based optimization in closed-loop reservoir management
- Robust optimal well control using an adaptive multigrid reinforcement learning framework
- Theoretical research on reservoir closed-loop production management
Cites work
- Application of a particle swarm optimization algorithm for determining optimum well location and type
- Comprehensive framework for gradient-based optimization in closed-loop reservoir management
- Convolutional -- recurrent neural network proxy for robust optimization and closed-loop reservoir management
- Deep reinforcement learning for optimal well control in subsurface systems with uncertain geology
- Derivative-free trust region optimization for robust well control under geological uncertainty
- Efficient real-time reservoir management using adjoint-based optimal control and model updating
- The benefit of multitask representation learning
Cited in
(1)
This page was built for publication: Multi-asset closed-loop reservoir management using deep reinforcement learning
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6125020)
