Serverless Data Processing with Dataflow: Develop Pipelines (Coursera)

Offered by Google Cloud,
Serverless Data Processing with Dataflow: Develop Pipelines (Coursera)

In this second installment of the Dataflow course series, we are going to be diving deeper on developing pipelines using the Beam SDK. We start with a review of Apache Beam concepts. Next, we discuss processing streaming data using windows, watermarks and triggers. We then cover options for sources and sinks in your pipelines, schemas to express your structured data, and how to do stateful transformations using State and Timer APIs.

Class Deals by MOOC List - Click here and see Coursera's Active Discounts, Deals, and Promo Codes.

We move onto reviewing best practices that help maximize your pipeline performance. Towards the end of the course, we introduce SQL and Dataframes to represent your business logic in Beam and how to iteratively develop pipelines using Beam notebooks.

Syllabus

WEEK 1
Introduction
This module covers the course outline
Beam Concepts Review
Review main concepts of Apache Beam, and how to apply them to write your own data processing pipelines.
Windows, Watermarks Triggers
In this module, you will learn about how to process data in streaming with Dataflow. For that, there are three main concepts that you need to learn: how to group data in windows, the importance of watermark to know when the window is ready to produce results, and how you can control when and how many times the window will emit output.
Sources & Sinks
In this module, you will learn about what makes sources and sinks in Google Cloud Dataflow. The module will go over some examples of Text IO, FileIO, BigQueryIO, PubSub IO, KafKa IO, BigTable IO, Avro IO, and Splittable DoFn. The module will also point out some useful features associated with each IO.

WEEK 2
Schemas
This module will introduce schemas, which give developers a way to express structured data in their Beam pipelines.
State and Timers
This module covers State and Timers, two powerful features that you can use in your DoFn to implement stateful transformations.
Best Practices
This module will discuss best practices and review common patterns that maximize performance for your Dataflow pipelines.

WEEK 3
Dataflow SQL & DataFrames
This modules introduces two new APIs to represent your business logic in Beam: SQL and Dataframes.
Beam Notebooks
This module will cover Beam notebooks, an interface for Python developers to onboard onto the Beam SDK and develop their pipelines iteratively in a Jupyter notebook environment.
Summary
This module provides a recap of the course

Go to Class
MOOC List is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

Related Courses

Oracle SQL Performance Tuning (Coursera) Coursera
EDUCBA

Oracle SQL Performance Tuning (Coursera)

This course is designed to provide you with skills needed to improve query performance in Oracle databases. In real-world scenarios, slow report generation and query execution often result from the sheer volume of data. To address this issue, this course covers topics such as Query Compilation and Execution, Indexing, Materialized Views, Common Table Expressions etc. Emphasis is on practical learning with step-by-step demonstrations.

Nov 30th 2026
4 Weeks
Programming and Application Foundations (Coursera) Coursera
Illinois Tech

Programming and Application Foundations (Coursera)

This course introduces the students to the creation and deployment of modern, standards-compliant web pages written in HTML, CSS, and JavaScript in the context of the client-server architecture of the web. The course also covers the fundamentals of programming in a modern programming language, basic data modeling, database design, implementation, and administration of single-user and shared multi-user database applications using a contemporary relational database management system.

Dec 7th 2026
5-12 Weeks
Introduction to Relational Databases (Coursera) Coursera
Illinois Tech

Introduction to Relational Databases (Coursera)

Database management systems are a crucial part of most large-scale industry and open-source systems. This course will introduce you to important concepts of database systems and design. We will learn what relational databases are, what they are used for, the theory underlying their design, and how to query and modify a database using the declarative SQL language.

Dec 7th 2026
4 Weeks
Using Databases with Python (Coursera) Coursera
University of Michigan

Using Databases with Python (Coursera)

This course will introduce students to the basics of the Structured Query Language (SQL) as well as basic database design for storing data as part of a multi-step data gathering, analysis, and processing effort. The course will use SQLite3 as its database. We will also build web crawlers and multi-step data gathering and visualization processes. We will use the D3.js library to do basic data visualization.

Nov 30th 2026
5-12 Weeks
BigQuery Fundamentals for Oracle Professionals (Coursera) Coursera
Google Cloud

BigQuery Fundamentals for Oracle Professionals (Coursera)

This course covers BigQuery fundamentals for professionals who are familiar with SQL-based cloud data warehouses in Oracle and want to begin working in BigQuery. Through interactive lecture content and hands-on labs, you learn how to provision resources, create and share data assets, ingest data, and optimize query performance in BigQuery. Drawing upon your knowledge of Oracle, you also learn about similarities and differences between Oracle and BigQuery to help you get started with data warehouses in BigQuery.

Dec 7th 2026
2 Weeks
Exploiting and Securing Vulnerabilities in Java Applications (Coursera) Coursera
University of California, Davis

Exploiting and Securing Vulnerabilities in Java Applications (Coursera)

In this course, we will wear many hats. With our Attacker Hats on, we will exploit Injection issues that allow us to steal data, exploit Cross Site Scripting issues to compromise a users browser, break authentication to gain access to data and functionality reserved for the ‘Admins’, and even exploit vulnerable components to run our code on a remote server and access some secrets.

Nov 23rd 2026
4 Weeks
Applications in Engineering Mechanics (Coursera) Coursera
Georgia Institute of Technology

Applications in Engineering Mechanics (Coursera)

This course applies principles learned in my course “Introduction to Engineering Mechanics” to analyze real world engineering structures. You will need to have mastered the engineering fundamentals from that class in order to be successful in this course offering. This course addresses the modeling and analysis of static equilibrium problems with an emphasis on real world engineering systems and problem solving.

Nov 30th 2026
5-12 Weeks
Relational Database Support for Data Warehouses (Coursera) Coursera
University of Colorado System

Relational Database Support for Data Warehouses (Coursera)

Relational Database Support for Data Warehouses is the third course in the Data Warehousing for Business Intelligence specialization. In this course, you'll use analytical elements of SQL for answering business intelligence questions. You'll learn features of relational database management systems for managing summary data commonly used in business intelligence reporting. Because of the importance and difficulty of managing implementations of data warehouses, we'll also delve into storage architectures, scalable parallel processing, data governance, and big data impacts. In the assignments in this course, you can use either Oracle or PostgreSQL.

Nov 23rd 2026
5-12 Weeks
Managing Big Data in Clusters and Cloud Storage (Coursera) Coursera
Cloudera

Managing Big Data in Clusters and Cloud Storage (Coursera)

In this course, you'll learn how to manage big datasets, how to load them into clusters and cloud storage, and how to apply structure to the data so that you can run queries on it using distributed SQL engines like Apache Hive and Apache Impala. You’ll learn how to choose the right data types, storage systems, and file formats based on which tools you’ll use and what performance you need.

Nov 23rd 2026
5-12 Weeks
VR and 360 Video Production (Coursera) Coursera
Google AR & VR

VR and 360 Video Production (Coursera)

Welcome to Daydream Impact Virtual Reality (VR) and 360 video production course! Our mission is to give you the skills needed to use VR to advocate for a cause and communicate your mission. This course will introduce you to Virtual Reality (VR) and 360 video production, guiding you through a step-by-step process to create VR content. To begin, we recommend taking a few minutes to explore the course site and review the material.

Nov 23rd 2026
4 Weeks
MySQL for Data Analytics and Business Intelligence (Coursera) Coursera
Edureka

MySQL for Data Analytics and Business Intelligence (Coursera)

Embark on a thorough educational expedition through the "Fundamentals of Database Management with MySQL" module. This foundational program aims to impart a robust comprehension of MySQL, a prominent Relational Database Management System (RDBMS). Participants will delve into crucial concepts, ranging from the fundamentals of SQL to more advanced topics like data retrieval, relationships, business intelligence, and data visualization.

Nov 30th 2026
1 Week