Autonomous Optimization of Targeted Stimulation of Neuronal Networks

Sreedhar S Kumar, Jan Wülfing, Samora Okujeni, Joschka Boedecker, Martin Riedmiller and Ulrich Egert

PLOS Computational Biology, 2016, vol. 12, issue 8, 1-22

Abstract: Driven by clinical needs and progress in neurotechnology, targeted interaction with neuronal networks is of increasing importance. Yet, the dynamics of interaction between intrinsic ongoing activity in neuronal networks and their response to stimulation is unknown. Nonetheless, electrical stimulation of the brain is increasingly explored as a therapeutic strategy and as a means to artificially inject information into neural circuits. Strategies using regular or event-triggered fixed stimuli discount the influence of ongoing neuronal activity on the stimulation outcome and are therefore not optimal to induce specific responses reliably. Yet, without suitable mechanistic models, it is hardly possible to optimize such interactions, in particular when desired response features are network-dependent and are initially unknown. In this proof-of-principle study, we present an experimental paradigm using reinforcement-learning (RL) to optimize stimulus settings autonomously and evaluate the learned control strategy using phenomenological models. We asked how to (1) capture the interaction of ongoing network activity, electrical stimulation and evoked responses in a quantifiable ‘state’ to formulate a well-posed control problem, (2) find the optimal state for stimulation, and (3) evaluate the quality of the solution found. Electrical stimulation of generic neuronal networks grown from rat cortical tissue in vitro evoked bursts of action potentials (responses). We show that the dynamic interplay of their magnitudes and the probability to be intercepted by spontaneous events defines a trade-off scenario with a network-specific unique optimal latency maximizing stimulus efficacy. An RL controller was set to find this optimum autonomously. Across networks, stimulation efficacy increased in 90% of the sessions after learning and learned latencies strongly agreed with those predicted from open-loop experiments. Our results show that autonomous techniques can exploit quantitative relationships underlying activity-response interaction in biological neuronal networks to choose optimal actions. Simple phenomenological models can be useful to validate the quality of the resulting controllers.Author Summary: Electrical stimulation of the brain is increasingly used to alleviate the symptoms of a range of neurological disorders and as a means to artificially inject information into neural circuits in neuroprosthetic applications. Machine learning has been proposed to find optimal stimulation settings autonomously. However, this approach is impeded by the complexity of the interaction between the stimulus and the activity of the network, which makes it difficult to test how good the result actually is. We used phenomenological models of the interaction between stimulus and spontaneous activity in a neuronal network to design a testable machine learning challenge and evaluate the quality of the solution found by the algorithm. In this task, the learning algorithm had to find a solution that balances competing interdependencies of ongoing neuronal activity with opposing effects on the efficacy of stimulation. We show that machine learning can successfully solve this task and that the solutions found are close to the optimal settings to maximize the efficacy of stimulation. Since the paradigm involves several typical problems found in other settings, such concepts could help to formalize machine learning problems in more complex biological networks and to test the quality of their performance.

Date: 2016
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005054 (text/html)
https://journals.plos.org/ploscompbiol/article/fil ... 05054&type=printable (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:plo:pcbi00:1005054

DOI: 10.1371/journal.pcbi.1005054

Access Statistics for this article

More articles in PLOS Computational Biology from Public Library of Science
Bibliographic data for series maintained by ploscompbiol ().