Reinforcement Learning to Send Reminders at Right Moments in Smartphone Exercise Application: A Feasibility Study

Shihan Wang, Karlijn Sporrel, Herke van Hoof, Monique Simons, Rémi D. D. de Boer, Dick Ettema, Nicky Nibbeling, Marije Deutekom and Ben Kröse
Additional contact information
Shihan Wang: Informatics Institute, University of Amsterdam, 1090 GH Amsterdam, The Netherlands
Karlijn Sporrel: Department of Human Geography and Spatial Planning, Utrecht University, 3584 CS Utrecht, The Netherlands
Herke van Hoof: Informatics Institute, University of Amsterdam, 1090 GH Amsterdam, The Netherlands
Monique Simons: Consumption & Healthy Lifestyles Group, Wageningen University & Research, 6700 HB Wageningen, The Netherlands
Rémi D. D. de Boer: Digital Life Centre, Amsterdam University of Applied Science, 1091 GC Amsterdam, The Netherlands
Dick Ettema: Department of Human Geography and Spatial Planning, Utrecht University, 3584 CS Utrecht, The Netherlands
Nicky Nibbeling: Centre of Expertise Urban Vitality, Amsterdam University of Applied Science, 1097 DZ Amsterdam, The Netherlands
Marije Deutekom: Faculty of Health, Sports and Welfare, Inholland University of Applied Sciences, 2015 CE Haarlem, The Netherlands
Ben Kröse: Informatics Institute, University of Amsterdam, 1090 GH Amsterdam, The Netherlands

IJERPH, 2021, vol. 18, issue 11, 1-15

Abstract: Just-in-time adaptive intervention (JITAI) has gained attention recently and previous studies have indicated that it is an effective strategy in the field of mobile healthcare intervention. Identifying the right moment for the intervention is a crucial component. In this paper the reinforcement learning (RL) technique has been used in a smartphone exercise application to promote physical activity. This RL model determines the ‘right’ time to deliver a restricted number of notifications adaptively, with respect to users’ temporary context information (i.e., time and calendar). A four-week trial study was conducted to examine the feasibility of our model with real target users. JITAI reminders were sent by the RL model in the fourth week of the intervention, while the participants could only access the app’s other functionalities during the first 3 weeks. Eleven target users registered for this study, and the data from 7 participants using the application for 4 weeks and receiving the intervening reminders were analyzed. Not only were the reaction behaviors of users after receiving the reminders analyzed from the application data, but the user experience with the reminders was also explored in a questionnaire and exit interviews. The results show that 83.3% reminders sent at adaptive moments were able to elicit user reaction within 50 min, and 66.7% of physical activities in the intervention week were performed within 5 h of the delivery of a reminder. Our findings indicated the usability of the RL model, while the timing of the moments to deliver reminders can be further improved based on lessons learned.

Keywords: reinforcement learning; just-in-time adaptive intervention; reminder; physical activity; mobile application (search for similar items in EconPapers)
JEL-codes: I I1 I3 Q Q5 (search for similar items in EconPapers)
Date: 2021
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1660-4601/18/11/6059/pdf (application/pdf)
https://www.mdpi.com/1660-4601/18/11/6059/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jijerp:v:18:y:2021:i:11:p:6059-:d:568915

Access Statistics for this article

IJERPH is currently edited by Ms. Jenna Liu

More articles in IJERPH from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().