“When you're building your own OS, debugging can't be an afterthought.”
As part of the SNU Systems infrastructure stack, we designed a lightweight, socket-based debugger for SNU's custom kernel (VMKernel).
This post walks through how we built that debugger and why it matters.
Why a Custom Debugger?
Most production debuggers are too heavy for OS-level work. GDB stubs assume a POSIX world. Remote debug agents bloat the runtime. And scripting often breaks at the kernel boundary.
I wanted something that:
- Works over raw TCP sockets
- Supports plain-text, deterministic commands
- Is straightforward and simple to implement and use.
The Debugger Contract
I designed a simple protocol, implemented in the class VMKernelContract, which wraps a socket connection and sends control commands to the debug server.
Protocol Overview
| Command | Meaning |
| ------- | ------- |
| BP=<symbol>; | Set a breakpoint at the given symbol |
| B=1; | Trigger a software breakpoint |
| C=1; | Continue execution |
| D=1; | Detach and close the debugger session |
Note: C=0 is used to stop execution, as the debugging protocol supports both string and booleans.
The protocol is intentionally ASCII-based and can be scripted with any language or netcat.
Code Sample
Stripped down for clarity, here's how the VMKernelContract class implements the basic commands:
BOOL VMKernelContract::BreakAt(STLString symbol) noexcept {
// Prepare the command to set a breakpoint at the given symbol
// The command format is "BP=<symbol>;"
// This must be a non mangled symbol, as the server (here the kernel) mangles it to find the said symbol.
std::stringstream ss;
ss << "BP=" << symbol << ";";
// Let's assume we wrap the send function into a macro here... and that the wrapper returns a boolean
return ::send(this->m_socket, ss.str().data(), ss.str().size(), 0);
}Interested?
Quote me at founder@snu.systems, I'll be as reponsive as possible.