Autonomous navigation of stratospheric balloons using reinforcement learning

Marc G. Bellemare (), Salvatore Candido (), Pablo Samuel Castro, Jun Gong, Marlos C. Machado, Subhodeep Moitra, Sameera S. Ponda and Ziyu Wang
Additional contact information
Marc G. Bellemare: Brain Team, Google Research
Salvatore Candido: Loon
Pablo Samuel Castro: Brain Team, Google Research
Jun Gong: Loon
Marlos C. Machado: Brain Team, Google Research
Subhodeep Moitra: Brain Team, Google Research
Sameera S. Ponda: Loon
Ziyu Wang: Brain Team, Google Research

Nature, 2020, vol. 588, issue 7836, 77-82

Abstract: Abstract Efficiently navigating a superpressure balloon in the stratosphere1 requires the integration of a multitude of cues, such as wind speed and solar elevation, and the process is complicated by forecast errors and sparse wind measurements. Coupled with the need to make decisions in real time, these factors rule out the use of conventional control techniques2,3. Here we describe the use of reinforcement learning4,5 to create a high-performing flight controller. Our algorithm uses data augmentation6,7 and a self-correcting design to overcome the key technical challenge of reinforcement learning from imperfect data, which has proved to be a major obstacle to its application to physical systems8. We deployed our controller to station Loon superpressure balloons at multiple locations across the globe, including a 39-day controlled experiment over the Pacific Ocean. Analyses show that the controller outperforms Loon’s previous algorithm and is robust to the natural diversity in stratospheric winds. These results demonstrate that reinforcement learning is an effective solution to real-world autonomous control problems in which neither conventional methods nor human intervention suffice, offering clues about what may be needed to create artificially intelligent agents that continuously interact with real, dynamic environments.

Date: 2020
References: Add references at CitEc
Citations: View citations in EconPapers (7)

Downloads: (external link)
https://www.nature.com/articles/s41586-020-2939-8 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nat:nature:v:588:y:2020:i:7836:d:10.1038_s41586-020-2939-8

Ordering information: This journal article can be ordered from
https://www.nature.com/

DOI: 10.1038/s41586-020-2939-8

Access Statistics for this article

Nature is currently edited by Magdalena Skipper

More articles in Nature from Nature
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().