Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks

Hendrik Strobelt, Sebastian Gehrmann, Bernd Huber, Hanspeter Pfister and Alexander M. Rush

Working Paper from Harvard University OpenScholar

Abstract: Recurrent neural networks, and in particular long short-term memory networks (LSTMs), are a remarkably effective tool for sequence modeling that learn a dense black-box hidden representation of their sequential input. Researchers interested in better understanding these models have studied the changes in hidden state representations over time and noticed some interpretable patterns but also significant noise. In this work, we present LSTMVis a visual analysis tool for recurrent neural networks with a focus on understanding these hidden state dynamics. The tool allows a user to select a hypothesis input range to focus on local state changes, to match these states changes to similar patterns in a large data set, and to align these results with domain specific structural annotations. We further show several use cases of the tool for analyzing specific hidden state properties on datasets containing nesting, phrase structure, and chord progressions, and demonstrate how the tool can be used to isolate patterns for further statistical analysis.

Date: 2016-01
New Economics Papers: this item is included in nep-cmp and nep-ure
References: Add references at CitEc
Citations:

Downloads: (external link)
http://scholar.harvard.edu/gehrmann/node/413341

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:qsh:wpaper:413341

Access Statistics for this paper

More papers in Working Paper from Harvard University OpenScholar Contact information at EDIRC.
Bibliographic data for series maintained by Richard Brandon ( this e-mail address is bad, please contact ).