Scraping Airlines Bots: Insights Obtained Studying Honeypot Data

Elisa Chiapponi, Marc Dacier, Onur Catakoglu, Olivier Thonnard and Massimiliano Todisco
Additional contact information
Elisa Chiapponi: EURECOM, France
Marc Dacier: EURECOM, France
Onur Catakoglu: Amadeus IT Group, France
Olivier Thonnard: Amadeus IT Group, France
Massimiliano Todisco: EURECOM, France

International Journal of Cyber Forensics and Advanced Threat Investigations, 2021, vol. 2, issue 1, 3-28

Abstract: Airline websites are the victims of unauthorised online travel agencies and aggregators that use armies of bots to scrape prices and flight information. These so-called Advanced Persistent Bots (APBs) are highly sophisticated. On top of the valuable information taken away, these huge quantities of requests consume a very substantial amount of resources on the airlines' websites. In this work, we propose a deceptive approach to counter scraping bots. We present a platform capable of mimicking airlines' sites changing prices at will. We provide results on the case studies we performed with it. We have lured bots for almost 2 months, fed them with indistinguishable inaccurate information. Studying the collected requests, we have found behavioural patterns that could be used as complementary bot detection. Moreover, based on the gathered empirical pieces of evidence, we propose a method to investigate the claim commonly made that proxy services used by web scraping bots have millions of residential IPs at their disposal. Our mathematical models indicate that the amount of IPs is likely 2 to 3 orders of magnitude smaller than the one claimed. This finding suggests that an IP reputation-based blocking strategy could be effective, contrary to what operators of these websites think today.

Keywords: Scraping bots; Honeypot; Deception; Mathematical Modelling (search for similar items in EconPapers)
Date: 2021
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://conceptechint.net/index.php/CFATI/article/view/23/8 (application/pdf)
https://conceptechint.net/index.php/CFATI/article/view/23 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:aox:rcfati:v:2:y:2021:i:1:p:3-28

DOI: 10.46386/ijcfati.v2i1.23

Access Statistics for this article

International Journal of Cyber Forensics and Advanced Threat Investigations is currently edited by Professor John William Walker

More articles in International Journal of Cyber Forensics and Advanced Threat Investigations from Concept Tech Publishing
Bibliographic data for series maintained by Mira Leonko ().