06. Spark Streaming

This chapter moves beyond the core Spark engine to explore the powerful libraries built on top of it: Spark SQL for structured data, Spark Streaming for real-time processing, and MLlib for machine learning.

Learning Objectives

• Spark SQL: Query structured data using SQL and DataFrames
• Spark Streaming: Process real-time data streams with fault tolerance
• MLlib: Build and deploy scalable machine learning models
• GraphX: (Overview) Analyze graph-structured data

The Unified Engine

The true power of Spark lies in its unified stack. You can load data using Spark SQL, train a model using MLlib, and apply that model to a real-time stream using Spark Streaming—all within the same application.

• 06 Spark Streaming - Real-time Data Processing
• 06-03 Spark SQL and DataFrames Required
• 06-04 Scalable Machine Learning with MLlib Optional