Information loss in a non-linear neuronal model

Eitan Z. Gross

Physica A: Statistical Mechanics and its Applications, 2019, vol. 525, issue C, 443-449

Abstract: The Fisher information forms a classical analytical tool for the problem of inferring an unknown deterministic parameter of a given neuronal input by observing and measuring the response of the neuron. Alas, given the complex non-linear response of a neuron, the form of the conditional output probability function can render direct calculation of the Fisher information rather difficult. Using the Cauchy–Schwarz inequality, an alternative information measure has been recently proposed (Stein and Nossek, 2017) which serves as a conservative approximation to the Fisher information. The alternative information measure can be calculated using the first four output moments, which in many cases can be easily evaluated by measuring neuronal output statistics. In an application of this alternative information measure, we demonstrate here how to conservatively establish the intrinsic inference capability of a non-linear neuron model with a hyperbolic tangent activation function, when the analytic form of the parametric output statistic is not available in closed form. We found that for neuronal input variables corrupted by either Gaussian or Poisson noise, the information loss of the neuron increases with an increase in the slope (i.e. non-linearity) of the activation function, with a steeper increase for a Gaussian as compared to a Poisson input. The current approach can be used to study the effect of noise on both the input signal and neuronal output in neurons with other activation functions.

Keywords: Cauchy–Schwarz inequality; Fisher information; Non-linear neuron (search for similar items in EconPapers)
Date: 2019
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0378437119303061
Full text for ScienceDirect subscribers only. Journal offers the option of making the article available online on Science direct for a fee of $3,000

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:phsmap:v:525:y:2019:i:c:p:443-449

DOI: 10.1016/j.physa.2019.03.071

Access Statistics for this article

Physica A: Statistical Mechanics and its Applications is currently edited by K. A. Dawson, J. O. Indekeu, H.E. Stanley and C. Tsallis

More articles in Physica A: Statistical Mechanics and its Applications from Elsevier
Bibliographic data for series maintained by Catherine Liu ().