EdX

Data Engineering with Databricks (edX)

Offered by AI (Pragmatic AI Labs),
Data Engineering with Databricks (edX)

Become an expert in modern data engineering on Databricks' unified lakehouse platform. Master ETL pipelines, data transformations with Apache Spark, and Delta Lake for reliable data management.

Class Deals by MOOC List - Click here and see EdX's Active Discounts, Deals, and Promo Codes.

Master Data Engineering on Databricks Lakehouse Platform

  • Learn Databricks architecture, cluster management & notebook analysis
  • Build reliable ETL pipelines with Delta Lake for data transformation
  • Implement advanced data processing techniques with Apache Spark

Course Highlights:

  • Create & scale Databricks clusters for workloads
  • Load data from diverse sources into notebooks
  • Explore, visualize & profile datasets with notebooks
  • Version control & share notebooks via Git integration
  • Read & ingest data in various file formats
  • Transform data with SQL & DataFrame operations
  • Handle complex data types like arrays, structs, timestamps
  • Deduplicate, join & flatten nested data structures
  • Identify & fix data quality issues with UDFs
  • Load cleansed data into Delta Lake for reliability
  • Build production-ready pipelines with Delta Live Tables
  • Schedule & monitor workloads using Databricks Jobs
  • Secure data access with Unity Catalog

Gain comprehensive skills in data engineering on Databricks through hands-on labs, real-world projects and best practices for the modern data lakehouse.
This course is part of the Large Language Model Operations (LLMOps) Professional Certificate.

What you'll learn

  • Use Databricks for data engineering and ML workloads
  • Create and design ML pipelines
  • Use Llamafile and other local LLMs like Mixtral

Syllabus

Module 1: Databricks Lakehouse Platform Fundamentals

  • Introduction to the Databricks Lakehouse Platform and its architecture
  • Creating, managing, and configuring clusters
  • Setting up and using Databricks with IntelliJ, RStudio, and the Databricks CLI
  • Introduction to notebooks, including execution, sharing, and multi-language support
  • Efficient data transformation with Spark SQL and the Catalog Explorer
  • Creating tables from files and querying external data sources
  • Reliable data pipelines with Delta Lake, ACID transactions, and Z-Ordering optimization

Module 2: Data Transformation and Pipelines
Automated pipelines with Delta Live Tables
Delta Live Tables components
Continuous vs triggered pipelines
Configuring Auto Loader
Querying pipeline events
End-to-end example of Delta Live
Vacuum and garbage collection
Orchestrating workloads with Databricks Jobs
Multi-task workflows and task dependencies
Viewing job history
Using dashboards
Handling failures and configuring retries
Unified data access with Unity Catalog
Catalogs vs metastores
Unity Catalog quickstart in Python
Applying object security
Best practices for catalogs, connections, and business units

Go to Class
MOOC List is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

Related Courses

Data Engineering Career Guide and Interview Preparation (Coursera) Coursera
IBM

Data Engineering Career Guide and Interview Preparation (Coursera)

Personal and Professional Development

This course is designed to prepare you to enter the job market as a data engineer. It provides guidance about the regular functions and tasks of data engineers and their place in the data ecosystem, as well as the opportunities of the profession and some options for career development. It explains practical techniques for creating essential job-seeking materials such as a resume and a portfolio, as well as auxiliary tools like a cover letter and an elevator pitch.

Sep 14th 2026
4 Weeks
Interview Interviewing Job Interview
Details
AI Skills for Engineers: Data Engineering and Data Pipelines (edX) EdX
Delft University of Technology,DelftX

AI Skills for Engineers: Data Engineering and Data Pipelines (edX)

Statistics & Data Analysis

Good data is central to effective AI applications. This course teaches the basics of data for AI, covering what data is needed, how to extract data from existing databases and basic data skills including setup of a Python notebook environment, basic data exploration and simple data visualizations.

Self Paced
Self-Paced
Artificial Intelligence Data Management AI
Details
Python for Data Engineering Project (edX) EdX
IBM

Python for Data Engineering Project (edX)

Statistics & Data Analysis

An opportunity to apply your foundational Python skills via a project, using various techniques to collect and work with data. Journey into the realm of becoming a Data Engineer and apply your basic Python knowledge of working with data. You will exercise various techniques in Python to extract data in multiple file formats from different sources, transform it into specific datatypes, and then prepare it for loading it into a database.

Self Paced
Self-Paced
Python APIs Jupyter Notebooks
Details
Machine Learning With Big Data (Coursera) Coursera
University of California, San Diego

Machine Learning With Big Data (Coursera)

Statistics & Data Analysis Data Science

Want to make sense of the volumes of data you have collected? Need to incorporate data-driven decisions into your process? This course provides an overview of machine learning techniques to explore, analyze, and leverage data. You will be introduced to tools and algorithms you can use to create machine learning models that learn from data, and to scale those models up to big data problems.

Aug 31st 2026
5-12 Weeks
Algorithms Machine Learning Big Data
Details
Big Data for Agri-Food: Principles and Tools (edX) EdX
Wageningen University,WageningenX

Big Data for Agri-Food: Principles and Tools (edX)

Statistics & Data Analysis Business

As the big data era unfolds, developments in sensor and information technologies are evolving quickly. As a result, science and businesses are yielding enormous amounts of data. Yet, to reap the actionable business solutions data can unveil, we must learn to ask the right questions. Join Wageningen Wageningen University & Research as the data team bridges the gap between the complexity of computer science and its practical application. Decipher your unsampled big data set.

Self Paced
Self-Paced
Big Data Clustering Hadoop
Details