Course: Faust

Faust is a stream processing library that allows you to write code to process data available as streams. In this course, you'll explore the basics of stream processing and how it fundamentally differs from batch processing. You'll start by examining the components of a stream processing system architecture, specifically the role of stream transport. You'll then investigate the Faust stream processing library, which uses native Python code for transformations on streaming data. Moving on, you'll explore what's meant by producers, consumers, and topics in Apache Kafka. You'll install Faust on your local machine as well as the Apache Kafka messaging service. You'll then use these to write a stream processing application. When you're finished with this course, you'll be able to clearly articulate the characteristics of stream processing and work with Apache Kafka and Faust to perform simple operations on input streams.

Stream processing in Faust uses native Python code, meaning you don't have to learn a new domain-specific language to perform data transformations. All you need to know is how existing concepts, such as models, work within a Faust context. Faust models allow you to specify fields and their data types and use this well-defined data structure to access streaming data. In this course, you'll learn how to represent the individual elements of a stream using Faust models. You'll work with agents, which are at the heart of every Faust stream processing application. You'll perform operations using agents and invoke agents synchronously and asynchronously from within your application. You'll then work with channels in Faust. At the end of this course, you'll have the confidence to use models, agents, and channels in the right way to build a successful application.

Faust streams support a wide range of operations. In this course, you'll learn how to perform several of these. You'll also work with Faust tables - which store state in the form of key-value pairs and allow for the recovery of failed processing, making Faust fault-tolerant. You'll start off by using the group by operation to designate a key used to repartition an input stream and create a new topic in Kafka. You'll then use the items() operation to access the key and message value and take() operation to buffer multiple elements in a stream. Next, you'll work with tables to conduct stateful stream processing, illustrating how table data is stored in an embedded RocksDB database. When you've finished this course, you'll be able to apply a wide range of operations on input streams and perform stateful stream processing using tables.

When working with data, windows are a handy tool to accumulate data subsets from input streams and perform aggregation operations on this specific data. In this course, you'll learn how to perform stream processing through windowing operations in Faust. You'll start by examining the different windowing operations possible on input streams, including tumbling, sliding, count, session, and global windows. Next, you'll distinguish the three notions of time associated with streaming events: event, ingestion, and processing time. You'll then use Faust window features to perform windowing operations on input streams and emit aggregation results for every window. Finally, you'll use the REST API server, which all Faust applications have, to make streaming code metrics and table data accessible to the user. Once you're done with this course, you'll be able to use windowing operations via Faust and expose metrics using web views.