Why Is More Efficient to Combine BeautifulSoup and Selenium in Scraping For Data Under Energy Crisis

Simona Vasilica Oprea () and Adela Bâra ()
Additional contact information
Simona Vasilica Oprea: The Bucharest University of Economic Studies, Romania
Adela Bâra: The Bucharest University of Economic Studies, Romania

Ovidius University Annals, Economic Sciences Series, 2022, vol. XXII, issue 2, 146-152

Abstract: The electricity prices are often sensitive data that pose challenges in terms of collection. The volatility and soaring prices make the day-ahead electricity markets appealing for scientists. Several events took place from 2020, such as COVID-19 and military conflict in Ukraine leading to higher inflation, interest rates and energy crisis. Since 2020, the electricity prices on day-ahead market (DAM) increased even up to ten times. To perform the electricity price prediction, extensive feature engineering and historical data are required. Data sources from Romania are assessed to analyze the opportunity to extract relevant data for the electricity price and traded quantities. Thus, in this, paper, we investigate opportunity to extract data from web pages. We will suggest solutions to extract historical data from web sites that do not provide APIs or csv files. Several Python libraries such as BeautifulSoup and Selenium will be showcased, and approaches will be compared.

Keywords: electricity price; data scrapin; BeautifulSoup; Selenium; prediction (search for similar items in EconPapers)
JEL-codes: C82 Q41 Q47 (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://stec.univ-ovidius.ro/html/anale/RO/2022-is ... 201%20and%202/19.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ovi:oviste:v:xxii:y:2022:i:2:p:146-152

Access Statistics for this article

Ovidius University Annals, Economic Sciences Series is currently edited by Spatariu Cerasela

More articles in Ovidius University Annals, Economic Sciences Series from Ovidius University of Constantza, Faculty of Economic Sciences Contact information at EDIRC.
Bibliographic data for series maintained by Gheorghiu Gabriela ().