Flat rent price prediction in Berlin with web scraping

Camilo Meyberg (), Ulrich Rendtel () and Holger Leerhoff ()
Additional contact information
Camilo Meyberg: Freie Universität Berlin
Ulrich Rendtel: Freie Universität Berlin
Holger Leerhoff: Amt für Statistik Berlin-Brandenburg

AStA Wirtschafts- und Sozialstatistisches Archiv, 2024, vol. 18, issue 2, No 5, 245-278

Abstract: Abstract Internet data pose a challenge to the traditional system of official statistics, which relies on more conventional sources such as surveys and registers, not readily adaptable to rapid changes. Expanding this system to include internet data is currently at an experimental stage, exploring these sources’ potentials and benefits. This paper describes a project conducted within the ESSnet Trusted Smart Statistics – Web Intelligence Network framework. It investigates the use of online apartment listings to analyze the rental market. We used web scraping to extract information from two online real estate portals for flats in the city of Berlin. Using this data, we developed a model to predict rental prices per square meter based on the accommodation’s features and location within the city. We detected offers which appear in both portals by means of statistical matching and removed duplicate offers. Missing values were treated by multiple imputation. The prediction model is a semi-parametric approach where the postal districts are used to describe the location effect. Comparisons with microcensus results and the local rent index reveal significant differences between the market of online flat offers and the stock of existing flat contracts. Interested readers will find the commented programming code in the internet supplement.

Keywords: Web scraping; Flat offers; Official statistics; Rent indices; Statistical matching; Multiple imputation; Semi-parametric regression (search for similar items in EconPapers)
JEL-codes: L86 R21 R31 (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s11943-024-00340-6 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:astaws:v:18:y:2024:i:2:d:10.1007_s11943-024-00340-6

Ordering information: This journal article can be ordered from
http://www.springer. ... ce/journal/11943/PS2

DOI: 10.1007/s11943-024-00340-6

Access Statistics for this article

AStA Wirtschafts- und Sozialstatistisches Archiv is currently edited by Ralf Münnich

More articles in AStA Wirtschafts- und Sozialstatistisches Archiv from Springer, Deutsche Statistische Gesellschaft - German Statistical Society
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().