Learning in continuous action space for developing high dimensional potential energy models

Sukriti Manna, Troy D. Loeffler, Rohit Batra, Suvo Banik, Henry Chan, Bilvin Varughese, Kiran Sasikumar, Michael Sternberg, Tom Peterka, Mathew J. Cherukara, Stephen K. Gray, Bobby G. Sumpter and Subramanian K. R. S. Sankaranarayanan ()
Additional contact information
Sukriti Manna: Argonne National Laboratory
Troy D. Loeffler: Argonne National Laboratory
Rohit Batra: Argonne National Laboratory
Suvo Banik: Argonne National Laboratory
Henry Chan: Argonne National Laboratory
Bilvin Varughese: Argonne National Laboratory
Kiran Sasikumar: Argonne National Laboratory
Michael Sternberg: Argonne National Laboratory
Tom Peterka: Argonne National Laboratory
Mathew J. Cherukara: Argonne National Laboratory
Stephen K. Gray: Argonne National Laboratory
Bobby G. Sumpter: Oak Ridge National Laboratory
Subramanian K. R. S. Sankaranarayanan: Argonne National Laboratory

Nature Communications, 2022, vol. 13, issue 1, 1-10

Abstract: Abstract Reinforcement learning (RL) approaches that combine a tree search with deep learning have found remarkable success in searching exorbitantly large, albeit discrete action spaces, as in chess, Shogi and Go. Many real-world materials discovery and design applications, however, involve multi-dimensional search problems and learning domains that have continuous action spaces. Exploring high-dimensional potential energy models of materials is an example. Traditionally, these searches are time consuming (often several years for a single bulk system) and driven by human intuition and/or expertise and more recently by global/local optimization searches that have issues with convergence and/or do not scale well with the search dimensionality. Here, in a departure from discrete action and other gradient-based approaches, we introduce a RL strategy based on decision trees that incorporates modified rewards for improved exploration, efficient sampling during playouts and a “window scaling scheme" for enhanced exploitation, to enable efficient and scalable search for continuous action space problems. Using high-dimensional artificial landscapes and control RL problems, we successfully benchmark our approach against popular global optimization schemes and state of the art policy gradient methods, respectively. We demonstrate its efficacy to parameterize potential models (physics based and high-dimensional neural networks) for 54 different elemental systems across the periodic table as well as alloys. We analyze error trends across different elements in the latent space and trace their origin to elemental structural diversity and the smoothness of the element energy surface. Broadly, our RL strategy will be applicable to many other physical science problems involving search over continuous action spaces.

Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://www.nature.com/articles/s41467-021-27849-6 Abstract (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nat:natcom:v:13:y:2022:i:1:d:10.1038_s41467-021-27849-6

Ordering information: This journal article can be ordered from
https://www.nature.com/ncomms/

DOI: 10.1038/s41467-021-27849-6

Access Statistics for this article

Nature Communications is currently edited by Nathalie Le Bot, Enda Bergin and Fiona Gillespie

More articles in Nature Communications from Nature
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().