Rootkits: What they are, and how to find them (OST)

Offered by OpenSecurityTraining,
Rootkits: What they are, and how to find them (OST)

This class discusses stealth malware techniques used on Windows. Rootkits are a type of stealth malware that try to hide their presence, and this class shows the data structures they manipulate to achieve this.

Rootkits are a class of malware which are dedicated to hiding the attacker’s presence on a compromised system. This class will focus on understanding how rootkits work, and what tools can be used to help find them.

This will be a very hands-on class where we talk about specific techniques which rootkits use, and then do labs where we show how a proof of concept rootkit is able to hide things from a defender. Example techniques include
•Trojaned binaries
•Inline hooks
•Import Address Table (IAT) hooking
•System Call Table/System Service Descriptor Table (SSDT) hooking
•Interrupt Descriptor Table (IDT) hooking
•Direct Kernel Object Manipulation (DKOM)
•Kernel Object Hooking (KOH)
•IO Request Packet (IRP) filtering
•Hiding files/processes/open ports
•Compromising the Master Boot Record (MBR) to install a “bootkit”

The class will help the student learn which tools to use to look for rootkits on Windows systems, how to evaluate the breadth of a tool’s detection capabilities, and how to interpret tool results.

This class is structured so that students are given a homework to detect rootkits before they have taken the class. This homework is given in the context of the following scenario:

“You, being the only ‘security person’ in the area, have been called in to
examine a running Windows server because "it's acting funny." They don't
care that you like Mac/Linux/BSD/Plan9 better, you need to look at it! You
are solemnly informed that this is system is mission critical and can only
be rebooted if absolutely necessary. You must investigate whether any sort
of compromise has taken place on the system, with minimal impact to the
mission. What do you do? What DO you DO?”

The homework is then for the student to use any means at their disposal to write up answers to the following questions: “What malicious changes were made to the system?”, “What tools did you use to detect the changes?”, “How can you remove the changes?”. The students’ answers are then anonymized and shared with the rest of the class afterwards, so that they can see how others approached the problem, and learn from their techniques. The anonymization of the homework before distribution is important so that students know that even though they don’t know, and aren’t expected to know, anything about the area yet, their entry will not be judged by other students.

Recommended Readings:
Amazon Item Amazon Item

Go to Class
MOOC List is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

Related Courses

Developing Android Apps with App Inventor (Coursera) Coursera
The Hong Kong University of Science and Technology - HKUST

Developing Android Apps with App Inventor (Coursera)

The course will give students hands-on experience in developing interesting Android applications. No previous experience in programming is needed, and the course is suitable for students with any level of computing experience. MIT App Inventor will be used in the course. It is a blocks-based programming tool that allows everyone, even novices, to start programming and build fully functional apps for Android devices. Students are encouraged to use their own Android devices for hands-on testing and exploitation.

Nov 16th 2026
5-12 Weeks
¡A Programar! Una introducción a la programación (Coursera) Coursera
University of Edinburgh,Universidad ORT Uruguay

¡A Programar! Una introducción a la programación (Coursera)

¿Alguna vez pensaste en crear tus propios juegos de computadora, pero no tenías idea cómo hacerlo o por dónde comenzar? Este curso te enseñará a programar utilizando Scratch, un lenguaje de programación visual muy fácil de usar, y más importante aún, aprenderás los principios fundamentales de la computación para que comiences a pensar como ingeniero/a de software.

Nov 16th 2026
5-12 Weeks
Cybersecurity for Everyone (Coursera) Coursera
University of Maryland, College Park

Cybersecurity for Everyone (Coursera)

Cybersecurity affects everyone, including in the delivery of basic products and services. If you or your organization want to better understand how to address your cybersecurity, this is the course for you and your colleagues to take -- from seasoned professionals to your non-technical colleagues. Your instructor, Dr. Charles Harry, has served on the front lines with the NSA (National Security Agency) and as an expert advising corporate and institutional leaders on managing cybersecurity risk.

Nov 16th 2026
5-12 Weeks
C++程序设计 (Coursera) Coursera
Peking University

C++程序设计 (Coursera)

讲授C++语言中和面向对象有关的概念和语法,如类和对象、运算符重载、继承和多态、类模板和算法模板、标准模板库等。目标是使得学生能够用C++语言,以面向对象的方法编写可维护性、可扩充性好的,较大规模的程序,初步领会面向对象程序设计方法的优势所在。

Nov 16th 2026
5-12 Weeks
Security Management and Governance (Coursera) Coursera
Royal Holloway, University of London

Security Management and Governance (Coursera)

In this course you will look at some of the key components of practical cyber security management and its governance. This includes what happens when things go wrong, understanding how organisations can respond to incidents, through to the essential role of people in achieving better cyber security outcomes. Together, you will examine how incident management, cyber resilience, and developing an effective appreciation of people, not simply as users but as active participants, can enable better cyber security outcomes.

Nov 16th 2026
4 Weeks
Developing FPGA-accelerated cloud applications with SDAccel: Theory (Coursera) Coursera
Politecnico di Milano

Developing FPGA-accelerated cloud applications with SDAccel: Theory (Coursera)

This course is for anyone passionate in learning how to develop FPGA-accelerated applications with SDAccel! We are entering in an era in which technology progress induces paradigm shifts in computing! As a tradeoff between the two extreme characteristics of GPP and ASIC, we can find a new concept, a new idea of computing... the reconfigurable computing, which has combined the advantages of both the previous worlds.

Nov 16th 2026
4 Weeks
Securing Digital Democracy (Coursera) Coursera
University of Michigan

Securing Digital Democracy (Coursera)

In this course, you'll learn what every citizen should know about the security risks--and future potential — of electronic voting and Internet voting. We'll take a look at the past, present, and future of election technologies and explore the various spaces intersected by voting, including computer security, human factors, public policy, and more.

Nov 16th 2026
5-12 Weeks