EdX

Scripting with Python and SQL for Data Engineering (edX)

Offered by AI (Pragmatic AI Labs),
Scripting with Python and SQL for Data Engineering (edX)

Master data manipulation using Python and SQL for data engineering. Extract, transform, and load data efficiently to drive insights and decision-making. In this course, part of the Python, Bash and SQL Essentials for Data Engineering Specialization, you will gain the skills to work effectively with data using Python and SQL.

Class Deals by MOOC List - Click here and see EdX's Active Discounts, Deals, and Promo Codes.

You will learn to:

  • Leverage Python's powerful data structures to load, manipulate, and analyze data
  • Create robust Python scripts to automate data processing tasks
  • Utilize SQLite to store and retrieve data from within your Python scripts
  • Extract data from websites using web scraping techniques and persist it in databases
  • Work with MySQL databases using modern tools like VSCode to execute queries and manage data

Whether you are a data engineer, analyst, or aspiring data professional, this course will equip you with the essential skills to efficiently handle data and drive meaningful insights in today's data-driven world.
This course is part of the Data Engineering Foundations Professional Certificate.

What you'll learn

  • Manipulate data using Python's built-in data structures
  • Create Python scripts to automate data tasks
  • Use SQLite to store and query data in Python
  • Extract data from websites via web scraping
  • Execute MySQL queries and operations in VSCode
  • Import and export data in MySQL databases

Syllabus

Here is the course structure formatted with bullets for each module:

Module 1: Working with Data in Python (5 hours)
\- Videos (Total 62 minutes):
\- Welcome to Scripting with Python and SQL for Data Engineering (Preview module)
\- Meet your Course Instructor: Alfredo Deza (0 minutes)
\- Overview of Key Concepts (4 minutes)
\- Introduction to Working with Data in Python (0 minutes)
\- Introduction to Python Data Structures (0 minutes)
\- Using Lists to Save and Retrieve Data in Python (5 minutes)
\- Using Dictionaries to Save and Retrieve Data in Python (7 minutes)
\- Overview of Less Common Data Structures in Python (4 minutes)
\- Recap of Data Structures in Python (1 minute)
\- Introduction to Choosing Data Structures in Python (1 minute)
\- Iterating Over Lists and Dictionaries in Python (7 minutes)
\- Iterating Over Other Data Structures in Python (2 minutes)
\- Storing Data Between Data Structures in Python (6 minutes)
\- Recap of Mapping Data Structures in Python (1 minute)
\- Introduction to Data Sources and Formats in Python (0 minutes)
\- Loading Data from Files and File Paths in Python (6 minutes)
\- Working with JSON in Python (6 minutes)
\- Saving Data from Python to Disk (4 minutes)
\- Recap of Persisting and Loading Data in Python (0 minutes)
\- Readings (Total 80 minutes):
\- Connect with your instructor (10 minutes)
\- Meet your Supporting Instructors: Kennedy Behrman and Noah Gift (10 minutes)
\- Course Structure and Discussion Etiquette (10 minutes)
\- Key Terms (10 minutes)
\- Python Lists and Dictionaries (10 minutes)
\- Key Terms (10 minutes)
\- Key Terms (10 minutes)
\- Build a notebook that reformats data into JSON (10 minutes)
\- Quizzes (Total 60 minutes):
\- Using Data Structures in Python (30 minutes)
\- Working with Data in Python (30 minutes)
\- Discussion Prompt (Total 10 minutes):
\- Meet and Greet (optional) (10 minutes)
\- Ungraded Labs (Total 120 minutes):
\- Looping Over Lists and Dictionaries in Python (60 minutes)
\- Loading and Saving JSON Files (60 minutes)

Module 2: Python Scripting and SQL (5 hours)
\- Videos (Total 56 minutes):
\- Introduction to Python Scripting and SQL (Preview module)
\- Introduction to Scripting in Python (0 minutes)
\- Creating a Script as a Module in Python (5 minutes)
\- Traversing the File System with a Script in Python (7 minutes)
\- Recap of Python Scripting Basics (0 minutes)
\- Introduction to Embedded Databases (0 minutes)
\- What is SQLite? (2 minutes)
\- Creating and Connecting to a SQLite Database in Python (5 minutes)
\- Saving and Querying from a SQLite Database in Python (10 minutes)
\- Recap of SQLite and Python (1 minute)
\- Introduction to Querying with SQL in Python (1 minute)
\- Basic SQL Commands in Python (6 minutes)
\- Extracting Distinct Data using SQL in Python (7 minutes)
\- Searching with SQL in Python (5 minutes)
\- Recap of Querying with SQL in Python (1 minute)
\- Readings (Total 70 minutes):
\- Key Terms (10 minutes)
\- Minimal Python, Chapter 2: Learn to Store Data (10 minutes)
\- Key Terms (10 minutes)
\- Example GitHub Repository (10 minutes)
\- Key Terms (10 minutes)
\- Example GitHub Repository (10 minutes)
\- Create a Reporting Script for File Sizes (10 minutes)
\- Quiz (Total 30 minutes):
\- Python Scripting and SQL (30 minutes)
\- Ungraded Labs (Total 180 minutes):
\- Build a Script to Find Large Files (60 minutes)
\- Create a SQLite Database and Store Data (60 minutes)
\- Querying a SQLite Database (60 minutes)

Module 3: Web Scraping using Python (4 hours)
\- Videos (Total 64 minutes):
\- Introduction to Web Scraping using Python (Preview module)
\- Introduction to Extracting Data from Unstructured HTML (1 minute)
\- Challenges with Web Data (3 minutes)
\- Parsing HTML with HTMLParser in Python (5 minutes)
\- Recap of Web Scraping Techniques in Python (0 minutes)
\- Introduction to Scrapy and XPath in Python (1 minute)
\- Creating a Web Scraping Project with Scrapy in Python (6 minutes)
\- Parsing Data with XPath and Scrapy Shell (11 minutes)
\- Using Scrapy Spider for Web Scraping (5 minutes)
\- Recap of Scrapy and XPath in Python (0 minutes)
\- Overview of Challenges with Web Scraping (2 minutes)
\- Scraping Locally (8 minutes)
\- Persisting Data in CSV and JSON Formats (9 minutes)
\- Persisting Data to a SQLite Database (6 minutes)
\- Recap of Persistence and Efficacy with Web Scraping (1 minute)
\- Readings (Total 70 minutes):
\- Key Terms (10 minutes)
\- Parsing Techniques with HTMLParser (10 minutes)
\- Key Terms (10 minutes)
\- Codespaces Lab: Parse HTML with Scrapy and XPath (10 minutes)
\- Key Terms (10 minutes)
\- Efficient Scraping Techniques (10 minutes)
\- Build a Web Scraping Tool (10 minutes)
\- Quiz (Total 30 minutes):
\- Web Scraping using Python (30 minutes)
\- Ungraded Labs (Total 120 minutes):
\- Parse HTML with HTMLParser (60 minutes)
\- Parse HTML and Persist it to a SQLite Database (60 minutes)

Module 4: Working with MySQL (7 hours)
\- Videos (Total 69 minutes):
\- Introduction to Working with MySQL (Preview module)
\- Introduction to VSCode with MySQL (1 minute)
\- Connecting to a MySQL Server (8 minutes)
\- Recap of VSCode with MySQL (1 minute)
\- Challenges of Running MySQL Queries (1 minute)
\- Using VSCode to Execute MySQL Queries (6 minutes)
\- Recap of Running MySQL Queries (1 minute)
\- Challenges with Importing to Databases (1 minute)
\- Importing CSV Data into MySQL (6 minutes)
\- Exporting Data from MySQL (6 minutes)
\- Recap of Importing and Exporting Data in MySQL (1 minute)
\- MySQL Hacking Overview (2 minutes)
\- MySQL from Terminal (3 minutes)
\- Archive and Drop Database (5 minutes)
\- Import external database Sakila (7 minutes)
\- Modify database Sakila (4 minutes)
\- Bash pipelines with MySQL (5 minutes)
\- MySQL to Python Standard Library Web Server (4 minutes)
\- Readings (Total 70 minutes):
\- Key terms (10 minutes)
\- Key terms (10 minutes)
\- Working with VSCode and MySQL (10 minutes)
\- Key Terms (10 minutes)
\- Lesson Reflection (10 minutes)
\- Loading and Exporting Data using MySQL (10 minutes)
\- Next Steps (10 minutes)
\- Quiz (Total 30 minutes):
\- Working with MySQL (30 minutes)
\- Ungraded Labs (Total 300 minutes):
\- Connect to a Running MySQL Server (60 minutes)
\- Running Queries Repository (60 minutes)
\- Export and Import Data in MySQL (60 minutes)
\- Linux Hacking with MySQL (60 minutes)
\- MySQL Sandbox (60 minutes)

Go to Class
MOOC List is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

Related Courses

Full Stack Application Development Project (edX) EdX
IBM

Full Stack Application Development Project (edX)

Computer Science

For this project you will apply your knowledge of front end and back end development gained from previous courses and build and deploy a full stack real-world web application on the cloud. Your project will showcase your understanding of the user experience and user interface design process, and your skills and technologies with Django, Python, Node.JS and Containers.

Self Paced
Self-Paced
Python Javascript UX
Details
Visualizing Data with Python (edX) EdX
IBM

Visualizing Data with Python (edX)

Statistics & Data Analysis CS: Programming

Data visualization is the graphical representation of data in order to interactively and efficiently convey insights to clients, customers, and stakeholders in general. "A picture is worth a thousand words." We are all familiar with this expression. It especially applies when trying to explain the insights obtained from the analysis of increasingly large datasets. Data visualization plays an essential role in the representation of both small and large-scale data. One of the key skills of a data scientist is the ability to tell a compelling story, visualizing data and findings in an approachable and stimulating way.

Self Paced
Self-Paced
Python Maps Charts
Details
Introduction to SQL (edX) EdX
Davidson College,DavidsonX

Introduction to SQL (edX)

Statistics & Data Analysis

In this introduction to SQL Online Course you will gain strong understanding and utilize SQL to aggregate, manipulate, analyze, and visualize data in your field. SQL (Structured Query Language) is the most commonly used language to communicate with databases and extract data for application development, reporting and analytics. It is ubiquitous for industries across the world and a vital skill for aspiring data professions to have. This course will introduce you to the SQL language and give you the skills you need to query and analyze data from complex relational databases.

Self Paced
Self-Paced
SQL Relational Databases Structured Query Language
Details
Relational Database Basics (edX) EdX
IBM

Relational Database Basics (edX)

Statistics & Data Analysis

This course teaches you the fundamental concepts of relational databases and Relational Database Management Systems (RDBMS). This course is an introduction to the world of relational databases. You will explore the fundamental concepts of relational databases and Relational Database Management Systems (RDBMS), learn about relational database design, and understand how to transform source data into tables with clearly defined relationships.

Self Paced
Self-Paced
SQL MySQL PostgreSQL
Details
Intermediate PostgreSQL (edX) EdX
University of Michigan,MichiganX

Intermediate PostgreSQL (edX)

Computer Science

This second course in PostgreSQL for Everybody explores a wide range of SQL techniques to aggregate and transact data in PostgreSQL.This course also explores handling, reading, and parsing data with PostgreSQL and working with text in databases with regular expressions. This course covers a wide range of SQL techniques, beyond basic CRUD (Create, Read, Update, and Delete) operations in PostgreSQL. You will learn the specifics of aggregation, transactions, reading and parsing CSV files and inserting data into a database.

Self Paced
Self-Paced
SQL CRUD PostgreSQL
Details
Python for Data Science (edX) EdX
University of California, San Diego,UC San DiegoX

Python for Data Science (edX)

Statistics & Data Analysis CS: Programming

Learn to use powerful, open-source, Python tools, including Pandas, Git and Matplotlib, to manipulate, analyze, and visualize complex datasets. In the information age, data is all around us. Within this data are answers to compelling questions across many societal domains (politics, business, science, etc.). But if you had access to a large dataset, would you be able to find the answers you seek?

Self Paced
Self-Paced
Programming Python Machine Learning
Details