Rootkits: What they are, and how to find them (OST)

Offered by OpenSecurityTraining,
Rootkits: What they are, and how to find them (OST)

This class discusses stealth malware techniques used on Windows. Rootkits are a type of stealth malware that try to hide their presence, and this class shows the data structures they manipulate to achieve this.

Rootkits are a class of malware which are dedicated to hiding the attacker’s presence on a compromised system. This class will focus on understanding how rootkits work, and what tools can be used to help find them.

This will be a very hands-on class where we talk about specific techniques which rootkits use, and then do labs where we show how a proof of concept rootkit is able to hide things from a defender. Example techniques include
•Trojaned binaries
•Inline hooks
•Import Address Table (IAT) hooking
•System Call Table/System Service Descriptor Table (SSDT) hooking
•Interrupt Descriptor Table (IDT) hooking
•Direct Kernel Object Manipulation (DKOM)
•Kernel Object Hooking (KOH)
•IO Request Packet (IRP) filtering
•Hiding files/processes/open ports
•Compromising the Master Boot Record (MBR) to install a “bootkit”

The class will help the student learn which tools to use to look for rootkits on Windows systems, how to evaluate the breadth of a tool’s detection capabilities, and how to interpret tool results.

This class is structured so that students are given a homework to detect rootkits before they have taken the class. This homework is given in the context of the following scenario:

“You, being the only ‘security person’ in the area, have been called in to
examine a running Windows server because "it's acting funny." They don't
care that you like Mac/Linux/BSD/Plan9 better, you need to look at it! You
are solemnly informed that this is system is mission critical and can only
be rebooted if absolutely necessary. You must investigate whether any sort
of compromise has taken place on the system, with minimal impact to the
mission. What do you do? What DO you DO?”

The homework is then for the student to use any means at their disposal to write up answers to the following questions: “What malicious changes were made to the system?”, “What tools did you use to detect the changes?”, “How can you remove the changes?”. The students’ answers are then anonymized and shared with the rest of the class afterwards, so that they can see how others approached the problem, and learn from their techniques. The anonymization of the homework before distribution is important so that students know that even though they don’t know, and aren’t expected to know, anything about the area yet, their entry will not be judged by other students.

Recommended Readings:
Amazon Item Amazon Item

Go to Class
MOOC List is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

Related Courses

Working with Integrated Development Environments (Coursera) Coursera
University of Leeds

Working with Integrated Development Environments (Coursera)

Integrated Development Environments (IDEs) are vital in increasing developer productivity and learning to work with them will help you create impressive software projects whether you're a beginner or have some programming experience. This course provides the essential skills and knowledge to excel in software development using Integrated Development Environments (IDEs). You'll cover IDE fundamentals, including environment setup, interface navigation, and valuable features like debugging.

Nov 16th 2026
2 Weeks
Introductory Intel x86: Architecture, Assembly, Applications, & Alliteration (OST) OpenSecurityTraining
OpenSecurityTraining

Introductory Intel x86: Architecture, Assembly, Applications, & Alliteration (OST)

Intel processors have been a major force in personal computing for more than 30 years. An understanding of low level computing mechanisms used in Intel chips as taught in this course serves as a foundation upon which to better understand other hardware, as well as many technical specialties such as reverse engineering, compiler design, operating system design, code optimization, and vulnerability exploitation.

Self Paced
Self-Paced
Security Management and Governance (Coursera) Coursera
Royal Holloway, University of London

Security Management and Governance (Coursera)

In this course you will look at some of the key components of practical cyber security management and its governance. This includes what happens when things go wrong, understanding how organisations can respond to incidents, through to the essential role of people in achieving better cyber security outcomes. Together, you will examine how incident management, cyber resilience, and developing an effective appreciation of people, not simply as users but as active participants, can enable better cyber security outcomes.

Nov 16th 2026
4 Weeks
Fundamental Skills in Bioinformatics (Coursera) Coursera
King Abdullah University of Science and Technology (KAUST)

Fundamental Skills in Bioinformatics (Coursera)

The course provides a broad and mainly practical overview of fundamental skills for bioinformatics (and, in general, data analysis). The aim is to support the simultaneous development of quantitative and programming skills for biological and biomedical students with little or no background in programming or quantitative analysis.

Nov 16th 2026
4 Weeks
Java程序设计 (Coursera) Coursera
Peking University

Java程序设计 (Coursera)

《Java程序设计》课程是使用Java语言进行应用程序设计的课程,针对各专业的大学本科生开设。课程的主要目标有三: 一、掌握Java语言的语法,能够较为深入理解Java语言机制,掌握Java语言面向对象的特点。 二、掌握JavaSE中基本的API,掌握在集合、线程、输入输出、图形用户界面、网络等方面的应用。三、能够编写有一定规模的应用程序,养成良好的编程习惯,会使用重构、设计模式、单元测试、日志、质量管理工具提高代码的质量。 对于学过“计算机基础、计算概论或C语言的学生”尤为适用。

Nov 16th 2026
5-12 Weeks
Programming with Cloud IoT Platforms (Coursera) Coursera
Pohang University of Science and Technology - POSTECH

Programming with Cloud IoT Platforms (Coursera)

Internet of Things (IoT) is an emerging area of information and communications technology (ICT) involving many disciplines of computer science and engineering including sensors/actuators, communications networking, server platforms, data analytics and smart applications. IoT is considered to be an essential part of the 4th Industrial Revolution along with AI and Big Data.

Nov 16th 2026
5-12 Weeks
¡A Programar! Una introducción a la programación (Coursera) Coursera
University of Edinburgh,Universidad ORT Uruguay

¡A Programar! Una introducción a la programación (Coursera)

¿Alguna vez pensaste en crear tus propios juegos de computadora, pero no tenías idea cómo hacerlo o por dónde comenzar? Este curso te enseñará a programar utilizando Scratch, un lenguaje de programación visual muy fácil de usar, y más importante aún, aprenderás los principios fundamentales de la computación para que comiences a pensar como ingeniero/a de software.

Nov 16th 2026
5-12 Weeks
Development of Real-Time Systems (Coursera) Coursera
EIT Digital

Development of Real-Time Systems (Coursera)

This course is intended for the Master's student and computer engineer who likes practical programming and problem solving! After completing this course, you will have the knowledge to plan and set-up a real-time system both on paper and in practice. The course centers around the problem of achieving timing correctness in embedded systems, which means to guarantee that the system reacts within the real-time requirements.

Nov 16th 2026
5-12 Weeks
Algorithms, Part I (Coursera) Coursera
Princeton University

Algorithms, Part I (Coursera)

This course covers the essential information that every serious programmer needs to know about algorithms and data structures, with emphasis on applications and scientific performance analysis of Java implementations. Part I covers elementary data structures, sorting, and searching algorithms. Part II focuses on graph- and string-processing algorithms.

Nov 16th 2026
5-12 Weeks
Cryptography I (Coursera) Coursera
Stanford University

Cryptography I (Coursera)

Cryptography is an indispensable tool for protecting information in computer systems. In this course you will learn the inner workings of cryptographic systems and how to correctly use them in real-world applications. The course begins with a detailed discussion of how two parties who have a shared secret key can communicate securely when a powerful adversary eavesdrops and tampers with traffic.

Nov 16th 2026
5-12 Weeks