3.5.1.2. Stream Processing

Handling streaming data sources and mining knowledge from them requires special types of processing tasks because of their live behaviour. Stream operations process new instances as they are available at every step. Along with a number of external and internal stream resources, MLPro’s stream module provides processing functionalities like sliding window, rearranger, etc. specialized for streaming data.

In MLPro, streaming data is processed with a task and workflow architecture. A StreamTask is single operation performed on new stream instances and a StreamWorkflow is a list of tasks arranged sequentially with defined dependencies. StreamTask and StreamWorkflow are specialized classes inherited from MLPro’s multiprocessing module. As shown in the above figure the scenario fetches new \(I_{tn}\) instances from the stream object and each task then processes a list of new instances \(I_{tn}\) and deleted/obsolete instances \(I_{td}\) as shown in the figure. The processed instances are stored in the shared object for further accessibility.

Learn more

• 3.5.1.2.1. Stream Task
• 3.5.1.2.2. Stream Workflow
• 3.5.1.2.3. Stream Scenario
• 3.5.1.2.4. Stream Plotting

Cross References

• Howto BF-STREAMS-102: Tasks Workflows And Stream Scenarios

• BF-MT - Multitasking

• API Reference: Streams