Event-Driven Data Pipelines: A Cloud-Based Approach to Real-Time Data Processing

Venkata Nagendra Kumar Kundavaram

International Journal of Scientific Research in Computer Science, Engineering and Information Technology, 2024, vol. 10, issue 6, 364-369

Abstract: This article comprehensively analyzes event-driven data pipelines in cloud computing environments, examining their architecture, implementation considerations, and real-world applications. The article explores the fundamental components of event-driven systems, from event sources and message brokers to processing layers, while evaluating their performance characteristics and reliability mechanisms. Through detailed analysis of system architectures, we investigate the integration of various cloud services and their role in enabling scalable, real-time data processing. The article demonstrates how modern event-driven architectures achieve sub-millisecond processing times and handle millions of events per second while maintaining system resilience. Our findings reveal significant improvements in operational efficiency across various industries, including financial services, marketing automation, and IoT monitoring solutions. The article also addresses critical implementation aspects, including security frameworks, integration protocols, and best practices for system optimization. These insights provide valuable guidance for organizations leveraging event-driven architectures in their digital transformation initiatives. The article concludes that event-driven data pipelines represent a crucial advancement in data processing technology, offering unprecedented capabilities in handling real-time data streams while maintaining scalability, reliability, and cost-effectiveness.

Keywords: Real-Time Analytics; Event-Driven Data Processing; Message Broker Systems; Cloud-Native Architecture; Distributed Event Processing (search for similar items in EconPapers)
Date: 2024
Note: Article URL: https://ijsrcseit.com/home/article/view/CSEIT24106183
References: Add references at CitEc
Citations:

Downloads: (external link)
https://ijsrcseit.com/home/article/view/CSEIT24106183 Article URL (text/html)
https://ijsrcseit.com/home/article/download/CSEIT24106183/CSEIT24106183 Full text (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:jbh:ijsrcs:v10:y2024:i6:id:424

DOI: 10.32628/CSEIT24106183

Access Statistics for this article

More articles in International Journal of Scientific Research in Computer Science, Engineering and Information Technology from International Journal of Scientific Research in Computer Science, Engineering and Information Technology
Bibliographic data for series maintained by Pankaj Sharma (USA) ().