EdX

Introduction to Designing Data Lakes on AWS (edX)

Offered by AWS,
Introduction to Designing Data Lakes on AWS (edX)

In this class, we will help you understand how to create and operate a data lake in a secure and scalable way, without previous knowledge of data science! Designing a data lake is challenging because of the scale and growth of data. This course is for professionals (Architects, System Administrators and DevOps) who need to design and build an architecture for secure and scalable Data Lake components. Students will learn about the use cases for a Data Lake and, contrast that with a traditional infrastructure of servers and storage.

Class Deals by MOOC List - Click here and see EdX's Active Discounts, Deals, and Promo Codes.

Developers need to understand best practices to avoid common mistakes that could be hard to rectify. In this course we will cover the foundations of what a Data Lake is, how to ingest and organize data into the Data Lake, and dive into the data processing that can be done to optimize performance and costs when consuming the data at scale.
This course is part of the Cloud Solutions Architecture Professional Certificate.

What you'll learn

  • Where to start with a Data Lake?
  • How to build a secure and scalable Data Lake?
  • What are the common components of a Data Lake?
  • Why do you need a Data Lake and what it's value?

Syllabus

Week 1: Hello World, I mean, Hello Data Lakes!
Video: Meet the Instructors
Video: Introduction to Week 1
Video: Why Data Lakes?
Video: Characteristics of a Data Lake
Video: Data Lake Components
Reading: Data Lake Characteristics and Components
Video: Comparison of a Data Lake to a Data Warehouse
Reading: Data Lakes and Data Warehouses
Video: Discussing sample Data Lake Architectures
Quiz/Assessment: Week 1 quiz

Week 2: AWS data related services
Video: Introduction to Week 2
Video: AWS Data Lake related services
Video: Amazon S3
Video: AWS Glue Data Catalog
Reading: S3 and Glue Data Catalog
Video: AWS Services used for data movement
Reading: Kinesis, API Gateway, etc
Video: AWS Services for Data processing
Video: AWS Services for Analytics
Video: AWS Services used for Predictive Analytics and Machine Learning
Reading: EMR, Glue Jobs, Lambda, Kinesis Analytics, Redshift
Video: Introduction to AWS LakeFormation
Reading: LakeFormation
Lab: Get familiar with AWS Services and create your first simple data lake

Week 3: Ingesting the rivers
Video: Introduction to Week 3
Video: Use the right tool for the job
Video: Understanding Data Structure and when to process data
Video: Data Streaming ingestion with Amazon Kinesis Services
Video: Diving Deep on Amazon Kinesis
Demo: Batch Data Ingestion with AWS Transfer Family
Reading: Batch Data Ingestion with AWS Services
Video: Data Cataloging
Demo: Using Glue Crawlers
Reading: The importance of data cataloging
Video: Reviewing the ingestion part of some Data Lake architectures
Lab: Ingesting Web Logs

Week 4: Processing and Analyzing data that sits in the Data Lake
Video: Introduction to Week 4
Video: Data prep and AWS Glue jobs
Video: File optimizations
Demo: Using S3, Glue and Athena to get insights about NYC Taxi data
Reading: Glue Jobs, Data Prep, Athena? Columnar Data Formats and Amazon Athena Optimizations
Video: Introduction to Data Lake security
Reading: Security and compliance
Video: The power of data visualization
Video: Introduction to Amazon QuickSight
Demo: Amazon Quicksight
Reading: Data visualization, Amazon QuickSight
Video: Registry of Open Data on AWS
Lab: Create an end-to-end Data Lake with AWS Services
Video: Course wrap-up!

Go to Class
MOOC List is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

Related Courses

Data Analytics in Health - From Basics to Business (edX) EdX
KU Leuven University

Data Analytics in Health - From Basics to Business (edX)

Health & Society Medicine & Pharmacology

Improve diagnostics, care and curing by effectively applying data analytics in healthcare and spot entrepreneurial opportunities. Many people talk about the promise of “big data” to health care. But how can the application of data analytics to big data actually improve health and health care? We will show that novel data analytics based solutions can result in better diagnosis, better care and better curing. This provides fertile ground for entrepreneurship and the development of new businesses.

No session available
4 Weeks
Health Healthcare Entrepreneurship
Details
DevOps on AWS: Code, Build, and Test (edX) EdX
AWS

DevOps on AWS: Code, Build, and Test (edX)

CS: Information & Technology Computer Science

Learn the basic concepts, technology, use cases, and benefits of implementing DevOps on AWS. DevOps is the combination of cultural philosophies, practices, and tools that increase an organization’s ability to deliver applications and services at high velocity: evolving and improving products at a faster pace than organizations using traditional software development and infrastructure management processes. This speed enables organizations to better serve their customers and compete more effectively in the market.

Self Paced
Self-Paced
DevOps AWS DevOps AWS
Details
Data Analytics Basics for Everyone (edX) EdX
IBM

Data Analytics Basics for Everyone (edX)

Statistics & Data Analysis

Learn the fundamentals of Data Analytics and gain an understanding of the data ecosystem, the process and lifecycle of data analytics, career opportunities, and the different learning paths you can take to be a Data Analyst. In this course, you will learn about the various components of a modern data ecosystem and the role Data Analysts, Data Scientists, and Data Engineers play in this ecosystem.

Self Paced
Self-Paced
Big Data Data Science Data Scientist
Details
The Power of Data (edX) EdX
Rolls Royce

The Power of Data (edX)

Statistics & Data Analysis

'The power of data’ is an interactive introduction to data and why it matters, with a focus on data analytics. In under an hour, you'll discover what data is and how it improves our world, through a series of animations, case studies and tips. You’ll learn that to get the most from your data, you need to go on a journey, gaining insight along the way. This course highlights key factors relating to collecting, sharing, analysing and generating value from data.

No sessions available
2 Weeks
Data Data Analytics
Details
Effecting Digital Transformation with Data Analytics Projects (edX) EdX
University of Maryland, College Park,University System of Maryland - USM,USMx,UMD

Effecting Digital Transformation with Data Analytics Projects (edX)

Statistics & Data Analysis Business

Learn how to transform an organization digitally through simulations and case studies by applying the Disciplined Agile Delivery framework to work through three simulations. As a business leader, you may be asking yourself, what’s next? R? Python? Graph Databases? The latest data analytics technology that was launched last week? How do you craft an effective digital strategy from these technologies? How do you build the data transformation roadmap to give your organization a competitive advantage? How will you digitally transform your organization to give the best customer experience? In this course, we will explore three case studies to learn the practical tools and methods used to build a digital organization.

Self Paced
Self-Paced
Digital Transformation Data Analytics Transforming Your Company’s Data Analytics: Championing the Digital Enterprise Professional Certificate
Details
Big Data sin misterios (edX) EdX
Inter-American Development Bank - IDB,IDBx

Big Data sin misterios (edX)

Statistics & Data Analysis

Transforma los datos en valor. Aprende el potencial de Big Data para aumentar la eficiencia y eficacia de tu organización en este MOOC desarrollado con Telefónica. 51.5% de los estudiantes que participaron a la encuesta dicen que el curso les servirá para mejorar la capacidad para formular, implementar, y/o evaluar políticas públicas.

Self Paced
Self-Paced
Big Data Data Visualization Data Analytics
Details
Cloud Cost Management: Optimization Strategies (edX) EdX
AWS

Cloud Cost Management: Optimization Strategies (edX)

CS: Information & Technology Computer Science

Cloud cost management is the organizational planning that allows an enterprise to understand and manage the costs and needs associated with its cloud technology. In particular, this means finding cost-effective ways to maximize cloud usage and efficiency. AWS has a set of services, tools and resources to help the customers with cost management and optimization. This course will highlight the solutions and best practices to get started with optimizing AWS cost and capacity.

Self Paced
Self-Paced
Cloud Cost Management AWS
Details
Building Modern .NET Applications on AWS (edX) EdX
AWS

Building Modern .NET Applications on AWS (edX)

Computer Science

Learn how to build agile and scalable cloud native .NET applications using AWS with our new digital course, Building Modern .NET Applications on AWS. Learn how to build modern .NET applications on the AWS Cloud using AWS Lambda, Amazon API Gateway, and Amazon Cognito with our new digital course, Building Modern .NET Applications on AWS. This course is designed for intermediate-level developers who are interested in learning how to build agile, scalable, and fully-available cloud native applications on the AWS Cloud.

Self Paced
Self-Paced
.NET Framework Amazon Cognito AWS
Details
Introduction to Data Analytics for Managers (edX) EdX
University of Michigan,MichiganX

Introduction to Data Analytics for Managers (edX)

Management & Leadership Business

Explore data science and analyze business data with Microsoft Azure through the use of SQL, machine learning, and other statistical techniques focused on business applications. Through a combination of lectures, business case studies, and hands-on learning this course provides an introduction to data analytics techniques and their application in business.

Self Paced
Self-Paced
SQL Machine Learning Manager
Details
Analytics for the Classroom Teacher (edX) EdX
Curtin University,CurtinX

Analytics for the Classroom Teacher (edX)

Education Statistics & Data Analysis

This course is ideal for school teachers who want to improve their teaching through valuable data-driven insights. Do you want to be more reflective in your teaching practice and wonder if there are technologies that can help? Are you curious about how data-driven, evidence-based teaching practices can improve your students’ learning? This is the course for you!

No sessions available
5-12 Weeks
Teaching Teacher Learning Analytics
Details
Data Analytics and Visualization Capstone Project (edX) EdX
IBM

Data Analytics and Visualization Capstone Project (edX)

Statistics & Data Analysis

Accelerate the knowledge you gain from previous courses in the IBM Data Analyst Professional Certificate program. Assume the role of an Associate Data Analyst and use various skills and techniques on real-world datasets to accomplish a task. This course provides students with the opportunity to assume the role of an Associate Data Analyst who has recently joined an organization. In this role, you will use Data Analytics skills and techniques on real-world datasets to complete a business task.

Self Paced
Self-Paced
Data Data Wrangling Data Visualization
Details