25. januar 2023.
Predavanje u okviru predmeta na doktorskim studijama: Savremena istraživanja u oblasti informacionih sistema i tehnologija
Mesto i vreme: FON, D-300, četvrtak, 26. 1. 2023. u 18.00 časova
Predavač: Artem Trofimov, doktorand na St. Petersburg State University, Mentor: Prof. Boris Novikov
Naslov: Distributed Stream Processing: Substream Management and Fault Tolerance
Stream processing aims to handle potentially unbounded input data. In order to ensure efficient and reliable processing of a potentially infinite stream, stream processing engines should solve the following two problems:
- The first task is to split an infinite stream into finite substreams: usually, aggregation over an infinite stream is not possible, so the stream processing system must determine when a stream with certain properties has ended in order to return the result of aggregation over a substream.
- The second challenge is fault tolerance. In a distributed system individual compute nodes may fail. The system must recover from failures in a reasonable time (without a complete repetition of calculations) and provide a consistent result.
In this talk we discuss various approaches to solve these two problems in modern distributed stream processing systems.
Artem Trofimov has both academic and industrial experience in building distributed systems for data science problems. He was a research engineer at JetBrains Research and then joined Yandex Cloud and worked as a team lead of the Yandex DataSphere. Currently, he is a lead of the machine learning infrastructure team at Nebius.
Prezentaciju održanog predavanja možete preuzeti na linku.