Virtualization, Docker, and Kubernetes for Data Engineering (edX)

Dive into the world of virtualization, containerization, and orchestration for data engineering:

• Understand virtualization fundamentals and work with virtual machines
• Explore Docker containers and build scalable microservices
• Orchestrate containers using Kubernetes and cloud platforms
• Utilize cloud development environments like GitHub Codespaces
• Learn production best practices, including monitoring, testing, and CI/CD

Gain practical experience with industry-standard tools and techniques. Develop the skills to build, deploy, and manage containerized data solutions at scale. Whether you're a student or data professional, level up your data engineering capabilities.
This course is part of the Data Engineering Foundations Professional Certificate.

What you'll learn

• Virtualization concepts and virtual machines
• Docker containers and microservices
• Kubernetes architecture and deployments
• Cloud development with GitHub Codespaces
• Container registries for Kubernetes
• Cloud-based Kubernetes solutions
• Production monitoring, testing, and CI/CD

Syllabus

\- Module 1: Virtualization Theory and Concepts (6 hours to complete)
\- 8 videos (Total 26 minutes)
\- Virtualization (2 minutes)
\- Scaling Applications (1 minute)
\- Hardware Utilization (0 minutes)
\- Introduction to Virtual Machines (2 minutes)
\- Virtual Box Demo (10 minutes)
\- Container Concepts (2 minutes)
\- Introduction to Docker (5 minutes)
\- Docker Architecture (1 minute)
\- 8 readings (Total 80 minutes)
\- Welcome to Kubernetes for Data Engineering with Python! (10 minutes)
\- Meet your Instructors: Noah Gift and Kennedy Behrman (10 minutes)
\- Tools and Platforms (10 minutes)
\- What is Virtualization? (10 minutes)
\- What is a Virtual Machine? (10 minutes)
\- Introduction to Containers (10 minutes)
\- Docker: The Container Platform (10 minutes)
\- Spin up a local Docker container (10 minutes)
\- 8 quizzes (Total 240 minutes)
\- Virtualization (30 minutes)
\- Virtualization (30 minutes)
\- Scaling Applications (30 minutes)
\- Introduction to Virtual Machines (30 minutes)
\- Virtual Box (30 minutes)
\- Containers (30 minutes)
\- Introduction to Docker (30 minutes)
\- Docker Architecture (30 minutes)
\- 2 discussion prompts (Total 20 minutes)
\- Meet and Greet (optional) (10 minutes)
\- Let Us Know if Something's Not Working (10 minutes)

\- Module 2: Using Docker (5 hours to complete)
\- 9 videos (Total 42 minutes)
\- Docker Client (5 minutes)
\- Creating a Volume (5 minutes)
\- Running a Database in a Container (6 minutes)
\- Building an Image (4 minutes)
\- Dockerfiles (2 minutes)
\- Dockerfile Examples (4 minutes)
\- Orchestration with Docker Compose (3 minutes)
\- Introduction to Airflow (5 minutes)
\- Airflow Demonstration using Compose (4 minutes)
\- 6 readings (Total 60 minutes)
\- Use the Docker Command Line (10 minutes)
\- Creating a Docker Image (Step-by-Step) (10 minutes)
\- Getting Started with Docker Compose (10 minutes)
\- Getting Started with Apache Airflow (10 minutes)
\- Docker vs. Kubernetes: A Primer (10 minutes)
\- Use Docker to Spin Up Airflow (10 minutes)
\- 8 quizzes (Total 240 minutes)
\- Docker (30 minutes)
\- Docker Client (30 minutes)
\- Volumes (30 minutes)
\- Running a Database in a Container (30 minutes)
\- Building an Image (30 minutes)
\- Dockerfiles (30 minutes)
\- Compose (30 minutes)
\- Airflow (30 minutes)

\- Module 3: Kubernetes: Container Orchestration in Action (6 hours to complete)
\- 14 videos (Total 52 minutes)
\- Kubernetes Key Concepts (1 minute)
\- Kubernetes Clusters (1 minute)
\- Kubernetes Nodes (1 minute)
\- Kubernetes Service Deployments (1 minute)
\- Cloud Developer Workspace Advantage (4 minutes)
\- Key Concepts in the GitHub Ecosystem (3 minutes)
\- Using GitHub Templates (2 minutes)
\- Using GitHub Codespaces (6 minutes)
\- Using OpenAI Codewhisper (1 minute)
\- Fine-Tuning a Model with Hugging Face (3 minutes)
\- Using GitHub Copilot (8 minutes)
\- GitHub Actions (3 minutes)
\- Running Minikube in GitHub Codespaces (6 minutes)
\- Deploying a Service with Minikube (7 minutes)
\- 7 readings (Total 70 minutes)
\- What is Kubernetes? (10 minutes)
\- Virtualization, Containerization, and Elasticity (10 minutes)
\- Fine-Tune a Pretrained Model (10 minutes)
\- Getting Started with GitHub Copilot (10 minutes)
\- Hello Minikube (10 minutes)
\- Minikube + Kubernetes: A Recap (10 minutes)
\- Deploying FastAPI to AWS with ECR and App Runner (10 minutes)
\- 8 quizzes (Total 240 minutes)
\- Kubernetes, GitHub, and Minikube (30 minutes)
\- Kubernetes Key Concepts (30 minutes)
\- Kubernetes Clusters (30 minutes)
\- Kubernetes Nodes (30 minutes)
\- Kubernetes Service Deployments (30 minutes)
\- Key Concepts in the GitHub Ecosystem (30 minutes)
\- Running Minikube in GitHub Codespaces (30 minutes)
\- Deploying a Service with Minikube (30 minutes)

\- Module 4: Building Kubernetes Solutions (9 hours to complete)
\- 13 videos (Total 66 minutes)
\- Build a Tiny Bash Container using GitHub Codespaces (8 minutes)
\- Build FastAPI Microservice in Cloud9 in Python (5 minutes)
\- Deploy a FastAPI PyTorch Containerized Application to AWS App Runner (7 minutes)
\- Options for Container Orchestration (2 minutes)
\- GCP Cloud Run (4 minutes)
\- Build Microservice in Cloud9 in C# (6 minutes)
\- AWS Copilot - Command Line Interface for Containerized Applications (9 minutes)
\- Load-Testing with Locust (3 minutes)
\- Monitoring Systems (1 minute)
\- SRE Mindset for MLOps (5 minutes)
\- Operationalize Microservices (1 minute)
\- CI for Microservices (6 minutes)
\- What is Continuous Delivery? (2 minutes)
\- 7 readings (Total 70 minutes)
\- Using Container Registries with Kubernetes: Azure Container Registry and Amazon Elastic Container Registry (ECR) (10 minutes)
\- Kubernetes and Google Cloud (10 minutes)
\- Deploying Containerized Applications and Kubernetes in the Cloud with AWS (10 minutes)
\- Getting Started with Site Reliability Engineering (SRE) (10 minutes)
\- Continuous Delivery of FastAPI App to AWS App Runner (10 minutes)
\- Final Project Explained (10 minutes)
\- Next Steps (10 minutes)
\- 14 quizzes (Total 420 minutes)
\- Kubernetes Data Engineering Solutions (30 minutes)
\- Build a Tiny Bash Container using GitHub Codespaces (30 minutes)
\- Build FastAPI Microservice in Cloud9 in Python (30 minutes)
\- Deploying a FastAPI PyTorch Containerized Application to AWS App Runner (30 minutes)
\- Options for Container Orchestration (30 minutes)
\- GCP Cloud Run (30 minutes)
\- Build Microservice in Cloud9 in C# (30 minutes)
\- AWS Copilot (30 minutes)
\- Load-Testing with Locust (30 minutes)
\- Monitoring Systems (30 minutes)
\- SRE Mindset for MLOps (30 minutes)
\- Operationalize Microservices (30 minutes)
\- CI for Microservices (30 minutes)
\- Continuous Delivery (30 minutes)