Engineering Resource-Efficient Data Management for Smart Cities with Apache Kafka

Theofanis P. Raptis (), Claudio Cicconetti, Manolis Falelakis, Grigorios Kalogiannis, Tassos Kanellos and Tomás Pariente Lobo
Additional contact information
Theofanis P. Raptis: Institute of Informatics and Telematics, National Research Council, 56124 Pisa, Italy
Claudio Cicconetti: Institute of Informatics and Telematics, National Research Council, 56124 Pisa, Italy
Manolis Falelakis: Netcompany-Intrasoft, 190 02 Athens, Greece
Grigorios Kalogiannis: Sphynx Technologies Solution AG, 6300 Zug, Switzerland
Tassos Kanellos: ITML, 115 25 Athens, Greece
Tomás Pariente Lobo: Atos Spain, 28037 Madrid, Spain

Future Internet, 2023, vol. 15, issue 2, 1-22

Abstract: In terms of the calibre and variety of services offered to end users, smart city management is undergoing a dramatic transformation. The parties involved in delivering pervasive applications can now solve key issues in the big data value chain, including data gathering, analysis, and processing, storage, curation, and real-world data visualisation. This trend is being driven by Industry 4.0, which calls for the servitisation of data and products across all industries, including the field of smart cities, where people, sensors, and technology work closely together. In order to implement reactive services such as situational awareness, video surveillance, and geo-localisation while constantly preserving the safety and privacy of affected persons, the data generated by omnipresent devices needs to be processed fast. This paper proposes a modular architecture to (i) leverage cutting-edge technologies for data acquisition, management, and distribution (such as Apache Kafka and Apache NiFi); (ii) develop a multi-layer engineering solution for revealing valuable and hidden societal knowledge in the context of smart cities processing multi-modal, real-time, and heterogeneous data flows; and (iii) address the key challenges in tasks involving complex data flows and offer general guidelines to solve them. In order to create an effective system for the monitoring and servitisation of smart city assets with a scalable platform that proves its usefulness in numerous smart city use cases with various needs, we deduced some guidelines from an experimental setting performed in collaboration with leading industrial technical departments. Ultimately, when deployed in production, the proposed data platform will contribute toward the goal of revealing valuable and hidden societal knowledge in the context of smart cities.

Keywords: smart cities; Apache Kafka; Apache NiFi; data management; Industry 4.0 (search for similar items in EconPapers)
JEL-codes: O3 (search for similar items in EconPapers)
Date: 2023
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1999-5903/15/2/43/pdf (application/pdf)
https://www.mdpi.com/1999-5903/15/2/43/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jftint:v:15:y:2023:i:2:p:43-:d:1044519

Access Statistics for this article

Future Internet is currently edited by Ms. Grace You

More articles in Future Internet from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().