Revealing and reducing bias when modelling choice behaviour on imbalanced panel datasets

Mirosława Łukawska, Laurent Cazor, Mads Paulsen, Thomas Kjær Rasmussen and Otto Anker Nielsen

Journal of choice modelling, 2024, vol. 50, issue C

Abstract: The emergence of modern tools and technologies gives a unique opportunity to collect large amounts of data for understanding behaviour. However, the generated datasets are often imbalanced, as individuals might contribute to the datasets at different frequencies and periods. Models based on these datasets are challenging to estimate, and the results are not straightforward to interpret without considering the sample structure. This study investigates the issue of handling imbalanced panel datasets for modelling individual behaviour. It first conducts a simulation experiment to study to which degree mixed logit models with and without panel reproduce the population preferences when using imbalanced data. It then investigates how the application of bias reduction strategies, such as subsampling and likelihood weighting, influences model results and finds that combining these techniques helps to find an optimal trade-off between bias and variance of the estimates. Considering the conclusions from the simulation study, a large-scale case study estimates bicycle route choice models with different correction strategies. These strategies are compared in terms of efficiency, weighted fit measures, and computational burden to provide recommendations that fit the modelling purpose. We find that the weighted panel mixed multinomial logit model, estimated on the entire dataset, performs best in terms of minimising the bias-efficiency trade-off in the estimates. Finally, we propose a strategy that ensures equal contribution of each individual to the estimation results, regardless of their representation in the sample, while reducing the computational burden related to estimating models on large datasets.

Keywords: Imbalanced panel; Panel mixed multinomial logit model; Subsampling; Weighting; Bias-efficiency trade-off (search for similar items in EconPapers)
Date: 2024
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S1755534524000046
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:eejocm:v:50:y:2024:i:c:s1755534524000046

DOI: 10.1016/j.jocm.2024.100471

Access Statistics for this article

Journal of choice modelling is currently edited by S. Hess and J.M. Rose

More articles in Journal of choice modelling from Elsevier
Bibliographic data for series maintained by Catherine Liu ().