Support Ukrain
Adaltas logoAdaltasAdaltas logoAdaltas

Streaming

Streaming is the continuous transmission of data over a network. Data streams are a continuous flow of data records, the end of which can usually not be foreseen in advance. The data records are processed continuously as soon as a new data record is received. The amount of data records per unit of time (data rate) can vary and possibly become so large that the limited resources are insufficient for further processing and the recipient has to react accordingly (e.g. discarding data records). In contrast to other data sources, data streams can only be processed continuously record by record - in particular, in contrast to data structures with random access (such as arrays), only sequential access to the individual data records is usually possible.

Data streams are often used for interprocess communication (communication between processes on a computer) and for the transmission of data over networks, especially IoT and for streaming media. They can be used in many ways within the framework of the Pipes and Filters programming paradigm. Pipe is a common functionnality of Unix shells. Examples of data streams are weather data, system metrics, factory devices information, as well as audio and video streams (streaming media).

Related articles

Databricks logs collection with Azure Monitor at a Workspace Scale

Databricks logs collection with Azure Monitor at a Workspace Scale

Categories: Cloud Computing, Data Engineering, Adaltas Summit 2021 | Tags: Metrics, Monitoring, Spark, Azure, Databricks, Log4j

Databricks is an optimized data analytics platform based on Apache Spark. Monitoring Databricks plateform is crucial to ensure data quality, job performance, and security issues by limiting access to…

Claire PLAYE

By Claire PLAYE

May 10, 2022

Canada - Morocco - France

International locations

10 rue de la Kasbah
2393 Rabbat
Canada

We are a team of Open Source enthusiasts doing consulting in Big Data, Cloud, DevOps, Data Engineering, Data Science…

We provide our customers with accurate insights on how to leverage technologies to convert their use cases to projects in production, how to reduce their costs and increase the time to market.

If you enjoy reading our publications and have an interest in what we do, contact us and we will be thrilled to cooperate with you.