Menno Markus Personal Blog

Part 5: Conditional Breakpoints

Sat, 02 May 2026 00:00:05 +0000

The code for this part can be found here. Feeling lost? Consider starting at part 1.

With the content covered up to this point you might have an idea already how conditional or log points can be implemented. Simply wait for a breakpoint hit, read any variables you need from memory, and decide whether to break execution/log something to the console. Most debuggers are in fact implemented this way. However you may have noticed, these types of breakpoints really slow down your program. This is because even if the debugger decides to continue through, the breakpoint was still hit. Execution halted and a kernel round trip was performed.

A kernel round trip refers to how the debugee and debugger interact. Modern CPU’s and operating systems are build on the idea of different protection rings. The least privileged is ring 3, user space, where all our programs operate. The highest privilege is ring 0, the OS kernel.

All exceptions are first handled by the kernel. This means when the debugee hits a breakpoint it has to perform an expensive context switch and wake up the kernel. The kernel inspects the exception and notifies our debugger. But for the debugger to perform actions like ‘read process memory’ or ‘continue running’ it again has to wake up the kernel. Programs in user space are not allowed to touch other programs. Only the kernel can. So each of these actions is a costly operation up and down the protection rings.

To win back performance, can we perhaps avoid triggering an exception in the first place? We only want to break when our condition is true. Taking a cue from hot-reloaders we can! By injecting code into the debugee.

Setup

Our plan will be to replace the code we want to break at with a small piece of assembly that checks the condition. When the debugee executes this code, it will only run the int3 (software breakpoint instruction) when the condition is true. But before looking into that we need to make two small adjustments to code from previous parts.

First we ask for two more bits of data: the address of the value to watch, and the value to compare it against. Our conditional breakpoint will break only when the value to watch is a multiple of the value to compare. I.e. value_to_watch % value_to_compare == 0

// Received the signal the debugee is ready.
if (exception_code == 0xE0000001) {
    log.info("Set conditional breakpoint at address?", .{});
    var input_line_bare = try stdin.takeDelimiter('\n') orelse unreachable;
    var input_line = std.mem.trim(u8, input_line_bare, "\r");
    const break_at_address = try std.fmt.parseInt(usize, input_line, 0);

    log.info("Compare against counter at address?", .{});
    input_line_bare = try stdin.takeDelimiter('\n') orelse unreachable;
    input_line = std.mem.trim(u8, input_line_bare, "\r");
    const address_to_compare = try std.fmt.parseInt(usize, input_line, 0);

    log.info("Break when counter is a multiple of?", .{});
    input_line_bare = try stdin.takeDelimiter('\n') orelse unreachable;
    input_line = std.mem.trim(u8, input_line_bare, "\r");
    const value_to_compare = try std.fmt.parseInt(u32, input_line, 0);

    try setBreakpoint(
        process_id, 
        break_at_address, 
        address_to_compare, 
        value_to_compare
    );
}

Next we upgrade swapProcessByte to swapProcessBytes so we can read/write multiple bytes at once. I’ll skip the implementation of this but it should be easy to imagine.

pub fn swapProcessBytes(
    process_handle: win.HANDLE, 
    address: usize, 
    byte_count: comptime_int, 
    bytes_to_write: [byte_count]u8,
) ![byte_count]u8 { ... }

Injecting new functions

To add our condition we need to inject some assembly into the debugee. But we can’t just insert assembly in the middle of an already compiled function. Not only would we override the existing assembly, there might not even be enough bytes in the function to write out our condition.

Instead will write a short¹ jump instruction at the address to break at. This will jump to a new function we’ve injected into the debugee. Inside this function we’ll put our condition, the breakpoint and the instruction(s) the jump replaced. After executing that it will jump back to where we came from and continue like nothing happened.

The first step is to get some memory from inside the debugee where we can write our new function to. Remember from previous parts that all memory comes in pages. When your program wants some memory from the OS it allocates it through VirtualAllocEx. This returns the address of a new memory page.

Here MEM_RESERVE puts some page(s) aside for us, but not into physical RAM² yet. MEM_COMMIT than actually allocates them from physical memory³ so they become read/write-able. You’ll also recognise the PAGE_EXECUTE_READWRITE protection flags we add from previous parts. Lastly it’s worth pointing out that while we ask for only 255 bytes, dwSize is always rounded up to the page size.

pub fn setBreakpoint(
    process_id: win.DWORD, 
    break_at_address: usize,
    address_to_compare: usize,
    value_to_compare: u32,
) !void {
    // Ensure we obtain the process handle with memory read, write and page alloc permissions.
    const process_access = win.PROCESS_VM_READ | win.PROCESS_VM_WRITE | win.PROCESS_VM_OPERATION;
    const process_handle = win.OpenProcess(process_access, win.FALSE, process_id);
    defer _ = win.CloseHandle(process_handle);

    // Allocate memory inside the debugee process to inject the assembly 
    // of a new function into.
    var new_function_to_inject: [255]u8 = undefined;
    const new_function_addr = @intFromPtr(win.VirtualAllocEx(
        process_handle,                                 // hProcess
        null,                                           // lpAddress
        new_function_to_inject.len,                     // dwSize
        win.MEM_RESERVE | win.MEM_COMMIT,               // flAllocationType
        win.PAGE_EXECUTE_READWRITE,                     // flProtect
    ));

Next we replace the instruction(s) at the address to break at, with a jump instruction. We use a 64-bit RIP-relative indirect jump, which for reasons we discuss later you normally probably wouldn’t. But it’s an easy instruction to understand. When executed it will first read the value at [rip + 0], than jump to that value. rip is the current instruction pointer, so + 0 means read the value immediately after this.

In this case we write a jump to the start of our new function memory, and remember the bytes we overwrote.

    // Replace the code at the location of our breakpoint with a jump
    // instruction to the injected function.
    var jump_instruction = [14]u8{ 
        0xFF, 0x25, 0x00, 0x00, 0x00, 0x00,             // jmp qword, [rip + 0]
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // `addr_to_jump_to`
    };
    @memcpy(jump_instruction[6..], std.mem.asBytes(&new_function_addr));

    const original_bytes = try swapProcessBytes(
        process_handle, 
        break_at_address, 
        jump_instruction.len, 
        jump_instruction,
    );

Building our function

Next we have to write out our function body. Usually you can invoke a compiler, but we’ll create some artisanally⁴ hand-crafted assembly instead:

pub fn buildFunctionToInject(
    result_buffer: []u8,
    address_to_compare: u64, 
    value_to_compare: u32, 
    orginal_address: u64, 
    orginal_bytes: []const u8
) void {
    var code: std.ArrayList(u8) = .initBuffer(result_buffer);
    const add = std.ArrayList(u8).appendSliceAssumeCapacity;

    add(&code, &.{
    // Preserve the previous register values.
        0x50, 0x51, 0x52,                   // push rax, rcx, rdx
    });
    add(&code, &.{
    // Read the value stored at `address_to_compare` from memory.
        0x48, 0xB8,                         // mov rax, `address_to_compare`
    });
    add(&code, std.mem.asBytes(&address_to_compare));
    add(&code, &.{
        0x8B, 0x00,                         // mov eax, [rax]
    });
    add(&code, &.{
    // Load the value of `value_to_compare` into a register.
        0xB9,                               // mov ecx, `value_to_compare`
    });
    add(&code, std.mem.asBytes(&value_to_compare));
    add(&code, &.{
    // Calculate eax / ecx, the remainder will be stored in edx.
        0x31, 0xD2,                         // xor edx, edx
        0xF7, 0xF1,                         // div ecx

    // Check if edx is zero. Jump over the breakpoint if not.
        0x85, 0xD2,                         // test edx, edx
        0x75, 0x01,                         // jnz +1

    // Breakpoint.
        0xCC,                               // int3

    // Restore the preserved register values.
        0x5A, 0x59, 0x58,                   // pop rdx, rcx, rax
    });
    // Execute the code we replaced.
    add(&code, orginal_bytes);
    // Insert a jump back to the original code.
    add(&code, &.{ 
        0xFF, 0x25, 0x00, 0x00, 0x00, 0x00, // jmp qword, [rip + 0]
                                            // `addr_to_jump_to`
    });
    const return_addr = orginal_address + orginal_bytes.len;
    add(&code, std.mem.asBytes(&return_addr));
}

I won’t go too deep into the assembly and hopefully let the comments speak for themselves. Reading assembly could be it’s own series I’m not qualified to write. But have a look at the general structure.

Notice how we jump over the int3 instruction based on a condition. And also notice we how we add the replaced bytes after this followed by a jump back to the original code. Roughly translated you get:

const counter = address_to_compare.*;
const value = value_to_compare;
const division, const remainder = counter / value;
if (remainder == 0) @breakpoint();

originalInstructions();
return;

While this code works for our demo, a real debugger wouldn’t put the int3 before the register pop instructions. These registers might have contained values needed to inspect local variables before we entered this function. Therefore it should restore these register values before breaking. But we’ll prefer to keep things simple.

Finishing our breakpoint

This has been a lot, but we are now basically there. Back in our setBreakpoint function we now simply write the assembly snippet into the process memory.

    // Build the assembly to inject and write it into the debugees memory.
    buildFunctionToInject(
        &new_function_to_inject,
        address_to_compare,
        value_to_compare,
        break_at_address,
        &original_bytes,
    );

    _ = try swapProcessBytes(
        process_handle,
        new_function_addr,
        new_function_to_inject.len,
        new_function_to_inject,
    );

    log.info("Successfully set breakpoint at address 0x{X}", .{break_at_address});
}

You might wonder about the handleBreakpoint function we had in previous parts. But unlike other parts, when we hit our breakpoint, we can just continue through. There is no stepping or swapping of assembly required.

// A software breakpoint has been hit.
if (exception_code == win.EXCEPTION_BREAKPOINT) {
    if (!initial_breakpoint_hit) {
        initial_breakpoint_hit = true;
    } else {
        log.info("Breakpoint hit! Continue?", .{});
        _ = try stdin.takeDelimiter('\n') orelse unreachable;
    }
}

Results

This has been the most complex part yet. But has it all been worth it?.. Yes!

info(debugee): Address of doWork() == 0x7FF631041720
info(debugee): Address of counter == 0x7FF631109600
info(debugee): Waiting for debugger...
info(debugger): Set conditional breakpoint at address?
0x7FF631041720
info(debugger): Compare against counter at address?
0x7FF631109600
info(debugger): Break when counter is a multiple of?
5
info(debugger): Successfully set breakpoint at address 0x7FF631041720
info(debugee): Starting work!
info(debugger): Breakpoint hit! Continue?
y
info(debugee): 0
info(debugee): 1
info(debugee): 2
info(debugee): 3
info(debugee): 4
info(debugger): Breakpoint hit! Continue?
y
info(debugee): 5
info(debugee): 6
info(debugee): 7
info(debugee): 8
info(debugee): 9
info(debugger): Breakpoint hit! Continue?

That brings us to the numerous issues our demo here has. I mentioned earlier how a function might not have enough bytes to replace. And in fact it might not even have enough bytes for a jump instruction. That’s why using a 14-byte jump instruction is not great. It would be better to use something like a 5-byte 32-bit relative jump⁵. But that requires logic to ensure the page we VirtualAllocEx is close enough to the original code to make the jump. We might also need to depend on certain compiler settings or other tricks to make sure all functions have enough room for at least one jump instruction.

Furthermore, we can’t just replace assembly instructions like this. What if our jump ends in the middle of another instruction? The original instruction would get cut in half and become un-executable. The only reason it worked here is because I inspected the assembly of doWork() beforehand. A real debugger would use a disassembler to find the next instruction boundary and replace up to it.

Because our jump takes up multiple bytes, we also have to now care about code potentially jumping from elsewhere into the area we overwrote. With the int3 instruction it was very convenient it only takes up 1-byte.

We can go on about edge cases that would need to be handled. And ultimately this is why most debuggers don’t use this technique. But there are times where repeatedly halting is unacceptable. So I would love to see some of the big players throw their infinite piles of cash at the problem.

Footnotes

Part 4: Software Data Breakpoints

Sat, 02 May 2026 00:00:04 +0000

The code for this part can be found here. Feeling lost? Consider starting at part 1.

Data breakpoints allow us to break on data reads and writes. Most mainstream debuggers will stick to using hardware breakpoints for this, as discussed in part 2. However this limits us to the architecture, allowing only 4 breakpoints watching up to 8-bytes each on X86_64. If the debugger ever complained to you about this, you know how annoying it can be! Maybe you are trying to catch a memory stomp over on an array or observe when a vec3 structure is accessed!

In the previous part we overcame similar limitations for stepping though code by adding software instruction breakpoints. With some creativity we can implement a software equivalent for data breakpoints to. A feature I wished more debuggers would implement.

Guard pages

By now you’ll have noticed breakpoints almost always work by triggering an exception the debugger can catch. So what we need is an exception we can set on data access. Luckily there is a feature for this called guard pages. Recall from the previous part that all memory is split into pages, often 4Kb each. Each page has permissions on how they can be accessed. The PAGE_GUARD modifier will prevent any memory access and trigger an exception when tried. This is commonly used by the OS catch errors or implement dynamic growth of memory like the stack.

We use this to implement our data breakpoint, marking the page holding the data we want to watch as a guard page ¹. Do note that we can only mark entire 4Kb pages. Any access outside the data we want to watch but sharing the same page(s) will also trigger an exception. We’ll have to deal with these false positives when we go to handle our breakpoint.

Changing the page protection using VirtualProtectEx should be familiar from the previous part. VirtualQueryEx is new, but simply returns the page information used to add/remove the guard flag. Both these functions require a process handle with the correct permissions to call.

var breakpoint_address: usize = undefined;
var breakpoint_byte_count: usize = undefined;
var breakpoint_memory_info = std.mem.zeroes(win.MEMORY_BASIC_INFORMATION);

pub fn setBreakpoint(
    process_id: win.DWORD, 
    break_at_address: usize, 
    break_byte_count: usize
) !void {
    // Ensure we obtain the handle with page protection and query info permissions.
    const process_access = win.PROCESS_QUERY_INFORMATION | win.PROCESS_VM_OPERATION;
    const process_handle = win.OpenProcess(process_access, win.FALSE, process_id);
    defer _ = win.CloseHandle(process_handle);

    // Get the current page protection of the memory we want to watch.
    if (win.VirtualQueryEx(
        process_handle,                                     // hProcess
        @ptrFromInt(break_at_address),                      // lpAddress
        &breakpoint_memory_info,                            // lpBuffer
        @sizeOf(win.MEMORY_BASIC_INFORMATION),              // dwLength
    ) == win.FALSE) {
        return error.VirtualQueryExFailed;
    }

    // Change the page protection to add the guard page flag, prohibiting any access.
    var ignore: win.DWORD = undefined;
    if (win.VirtualProtectEx(
        process_handle,                                     // hProcess
        breakpoint_memory_info.BaseAddress,                 // lpAddress
        breakpoint_memory_info.RegionSize,                  // dwSize
        breakpoint_memory_info.Protect | win.PAGE_GUARD,    // flNewProtect
        &ignore,                                            // lpflOldProtect
    ) == win.FALSE) {
        return error.VirtualProtectFailed;
    }

    breakpoint_address = break_at_address;
    breakpoint_byte_count = break_byte_count;
    log.info("Successfully set breakpoint at address 0x{X}", .{break_at_address});
}

Guard page exceptions

When a guard page is accessed it raises the STATUS_GUARD_PAGE_VIOLATION exception. This exception is undocumented by Microsoft, but some searching around reveals it returns the same information as EXCEPTION_ACCESS_VIOLATION. The first element it holds is the type of access that caused the exception: read, write or execute. The second element gives the address trying to be accessed. We can use this to filter out any false positives not within the data we want to watch.

if (exception_code == win.STATUS_GUARD_PAGE_VIOLATION) {
    const has_params = debug_event.u.Exception.ExceptionRecord.NumberParameters >= 2;
    const access_type = debug_event.u.Exception.ExceptionRecord.ExceptionInformation[0];
    const access_addr = debug_event.u.Exception.ExceptionRecord.ExceptionInformation[1];

    const execute_access = 8;
    //const write_access = 1;
    //const read_access = 0;

    const start_addr = breakpoint_address;
    const end_addr = breakpoint_address + breakpoint_byte_count;
    const breakpoint_hit = start_addr <= access_addr and access_addr < end_addr;

    if (has_params and access_type != execute_access and breakpoint_hit) {
        log.info("Breakpoint hit! Continue?", .{});
        _ = try stdin.takeDelimiter('\n') orelse unreachable;
    }

    // If this is not our data watch, immediately continue.
    try handleBreakpoint(process_id, thread_id);
}

When the guard page exception triggers it automatically removes the guard page flag. This exception is classified as a fault, which if you remember means the instruction causing the exception has not completed executing. This means we can single step the CPU to allow through the instructions access to our page. In the previous part we showed how this was done with the trap flag.

Finally we re-apply the guard page flag so our breakpoint remains set.

pub fn handleBreakpoint(process_id: win.DWORD, thread_id: win.DWORD) !void {
    // Ensure we obtain the process handle with page protection permissions.
    const process_access = win.PROCESS_VM_OPERATION;
    const process_handle = win.OpenProcess(process_access, win.FALSE, process_id);
    defer _ = win.CloseHandle(process_handle);

    // The guard page flag is automatically removed upon triggering it.
    // Allow the CPU to execute the memory access instruction than re-apply the guard flag.

    const thread_handle, var thread_ctx = try getThreadContext(thread_id);
    {
        // Tell the CPU to only execute 1 instruction before raising an exception again.
        const trap_flag_bit = 0x00000100;
        thread_ctx.EFlags |= trap_flag_bit;
    }
    try setThreadContext(thread_handle, thread_ctx);

    // Execute the instruction.
    var debug_event = std.mem.zeroes(win.DEBUG_EVENT);
    _ = win.ContinueDebugEvent(process_id, thread_id, win.DBG_CONTINUE);
    _ = win.WaitForDebugEvent(&debug_event, win.INFINITE);

    std.debug.assert(debug_event.dwDebugEventCode == win.EXCEPTION_DEBUG_EVENT);
    const exception_code = debug_event.u.Exception.ExceptionRecord.ExceptionCode;
    std.debug.assert(exception_code == win.EXCEPTION_SINGLE_STEP);

    // Reapply the guard flag.
    var ignore: win.DWORD = undefined;
    if (win.VirtualProtectEx(
        process_handle,                                     // hProcess
        breakpoint_memory_info.BaseAddress,                 // lpAddress
        breakpoint_memory_info.RegionSize,                  // dwSize
        breakpoint_memory_info.Protect | win.PAGE_GUARD,    // flNewProtect
        &ignore,                                            // lpflOldProtect
    ) == win.FALSE) {
        return error.VirtualProtectFailed;
    }
    log.info("Resumed execution.", .{});
}

Result

To test if we were successful in lifting restriction, let’s increase our debugee counter variable from u64 (8-bytes) to u128 (16-bytes). A size hardware breakpoints can’t watch. Running the debugger…

info(debugee): Address of doWork() == 0x7FF73ABB1720
info(debugee): Address of counter == 0x7FF73AC79600
info(debugee): Waiting for debugger...
info(debugger): Set software data breakpoint at address?
0x7FF73AC79600
info(debugger): Memory byte size?
16
info(debugger): Successfully set breakpoint at address 0x7FF73AC79600
info(debugee): Starting work!
info(debugger): Breakpoint hit! Continue?
y
info(debugger): Resumed execution.
info(debugger): Breakpoint hit! Continue?
y
info(debugger): Resumed execution.
info(debugee): 0
info(debugger): Breakpoint hit! Continue?
y
info(debugger): Resumed execution.
info(debugger): Breakpoint hit! Continue?
y
info(debugger): Resumed execution.
info(debugger): Breakpoint hit! Continue?
y
info(debugger): Resumed execution.
info(debugger): Breakpoint hit! Continue?
y
info(debugger): Resumed execution.
info(debugger): Breakpoint hit! Continue?
y
info(debugger): Resumed execution.
info(debugger): Breakpoint hit! Continue?
y
info(debugger): Resumed execution.
info(debugee): 1
info(debugger): Breakpoint hit! Continue?

It works but… aren’t we hitting the breakpoint a lot? Lets consider the code we are running:

var counter: u128 = 0;
fn doWork() void {
    log.info("{}", .{counter});
    counter += 1;
}

You might break this down into 3 operations:

Read counter and log it.
Read counter and add 1 to it.
Write the result back to counter.

But there are 6 hits of our breakpoint for each call to doWork(). This starts to make sense if you consider that the compiler might had to split reading/writing a 128-bit value into two 64-bit accesses. Each access to counter involves first touching the lower 64-bits we are watching, than touching the higher 64-bits. A look at the actual assembly output confirms this theory.

So great! We managed to create software data breakpoints! Of course, just like with software instruction breakpoints, hardware breakpoints aren’t obsolete. A lot of debuggers still prefer them for data breakpoints. You might not be able to change page protection flags. And having to watch an entire 4Kb page can generate a lot of false positives, slowing down the program. But it’s certainly a useful tool to be aware of. Even in the absence of debugger support, it’s not difficult to implement this in our own programs.

In the next part we’ll look into how we can remove some of the overhead repeatedly breaking has.

Footnotes

Part 3: Software Instruction Breakpoints

Sat, 02 May 2026 00:00:03 +0000

The code for this part can be found here. Feeling lost? Consider starting at part 1.

Last time we managed to set our first breakpoint using hardware registers. While quite powerful, we could only set 4 of them at the same time. In fact, because we depend on hardware, if you are on ARM you might only get 2 or even just 1! This is why nearly all debuggers use software breakpoints instead. Or at least they do for instruction breakpoints. Data breakpoints are often still left to the hardware registers and their limitations. In this part we’ll look at implementing software instruction breakpoints. Than in the next part we’ll see if we can come up with a data breakpoint equivalent.

As implied by the name, software breakpoints are implemented in software by overriding part of the code we are executing. X86 has a special instruction for this called interrupt 3 or int3. When the CPU encounters an interrupt it looks up the corresponding entry in the “interrupt vector table” for a function to call. This is normally used for events such as “illegal instruction” or “invalid memory access”. But in the case of int3 the OS has placed a handler here that raises a breakpoint exception.

The int3 instruction is very convenient as it’s encoding is 0xCC ¹, occupying just 1 byte. This means we can override the start of any other instruction, turning it into a breakpoint when the CPU tries to execute it. On X86 instructions are between 1 and 16-bytes, so if int3 were 2-bytes or more we’d risk overriding part of the next instruction.

The plan will be that we write int3 to the address we want to break on. Than when the breakpoint exception hits we can step back the debugee, restore the original instruction and continue execution. We than reapply the int3 so the breakpoint hits again.

Overwriting memory

To replace an instruction with int3 we must read and write the memory of our debugee process. Windows provides ReadProcessMemory and WriteProcessMemory to do this. These might give a sense of deja vu. Similar to the previous part, when we tried to set the thread context, these functions require we open a privileged process handle. Additionally any instruction memory is likely marked execute-only as a security feature by the OS. We’ll first have to make it readable/writable before we can do anything.

Recall that the DEBUG_EVENT gave us the debugee process id ². We obtain a process handle with 3 permissions:

Allow reading memory.
Allow writing memory.
Allow operations on the address space.

Once done, we have to close the handle again. If you are not familiar with Zigs defer keyword, it just means: “Execute this code when leaving the scope.” This ensures we close the handle when the function returns.

const process_access = win.PROCESS_VM_READ | win.PROCESS_VM_WRITE | win.PROCESS_VM_OPERATION;
const process_handle = win.OpenProcess(process_access, win.FALSE, process_id);
defer _ = win.CloseHandle(process_handle);

The last permission requires some explanation. The OS allocates memory in pages, often 4Kb each. Even if you try to alloc something smaller, somewhere an allocator has to request a full page from the OS before dividing it up into smaller parts returned to you. Code too is also placed in a page. Each of these pages has a set of protection flags and PROCESS_VM_OPERATION gives us the privilege to change them.

This is exactly what we’ll do next using VirtualProtectEx. We’ll ensure the page has the PAGE_READWRITE flag before we attempt to do so. When done, we restore the original page permissions (using defer) so it’s as nothing happened.

Notice the Windows API asks us to pass the address as a pointer. It’s important to remember though, this address isn’t pointing into our memory space but that of the debugee instead. So it’s not valid to try and access it.

// Instruction memory is likely be protected, ensure we can read and write to the memory.
var old_page_protection: win.DWORD = undefined;
if (win.VirtualProtectEx(
    process_handle,         // hProcess
    @ptrFromInt(address),   // lpAddress
    @sizeOf(u8),            // dwSize
    win.PAGE_READWRITE,     // flNewProtect
    &old_page_protection,   // lpflOldProtect
) == win.FALSE) {
    return error.VirtualProtectFailed;
}

// Ensure we restore the memory protection after we are done writing to it.
var ignore: win.DWORD = undefined;
defer _ = win.VirtualProtectEx(
    process_handle,         // hProcess
    @ptrFromInt(address),   // lpAddress
    @sizeOf(u8),            // dwSize
    old_page_protection,    // flNewProtect
    &ignore,                // lpflOldProtect
);

We now have the permission to actually read and write memory. We’ll implement our function as swapProcessByte, replacing 1-byte at the target address and returning whatever we overwrote. This will allow us to read the original instruction byte, than replace it with int3.

var byte_read: u8 = undefined;
if (win.ReadProcessMemory(
    process_handle,         // hProcess
    @ptrFromInt(address),   // lpBaseAddress
    &byte_read,             // lpBuffer
    @sizeOf(u8),            // nSize
    null,                   // lpNumberOfBytesRead
) == win.FALSE) {
    return error.ReadProcessMemoryFailed;
}

if (win.WriteProcessMemory(
    process_handle,         // hProcess
    @ptrFromInt(address),   // lpBaseAddress
    &byte_to_write,         // lpBuffer
    @sizeOf(u8),            // nSize
    null,                   // lpNumberOfBytesWritten
) == win.FALSE) {
    return error.WriteProcessMemoryFailed;
}

Finally an important detail is to flush the instruction cache. Because code is stored in main memory it can be slow to access and decode. To speed up execution, the CPU will copy code into a fast instruction cache. If we don’t tell it the cache is outdated, it might not pick up on the changes we made.

_ = win.FlushInstructionCache(process_handle, @ptrFromInt(address), @sizeOf(u8));
return byte_read;

Setting a breakpoint

Using our new swapProcessByte function it becomes trivial to replace the implementation of setBreakpoint. We’ll write the int3 instruction and record the original byte and location we replaced.

const breakpoint_instruction: u8 = 0xCC; // Equal to int3 instruction.
var breakpoint_address: usize = undefined;
var orginal_instruction_byte: u8 = undefined;

pub fn setBreakpoint(process_id: win.DWORD, break_at_address: usize) !void {
    orginal_instruction_byte = try swapProcessByte(process_id, break_at_address, breakpoint_instruction);
    breakpoint_address = break_at_address;
    log.info("Successfully set breakpoint at address 0x{X}", .{break_at_address});
}

Handling a breakpoint hit

Handling the breakpoint is more complicated. Let’s start by replacing the EXCEPTION_SINGLE_STEP check for hardware breakpoints with the more appropriately named EXCEPTION_BREAKPOINT raised when int3 is hit. We also grab the address triggering the exception to check if it’s our breakpoint.

if (debug_event.dwDebugEventCode == win.EXCEPTION_DEBUG_EVENT) {
    const exception_code = debug_event.u.Exception.ExceptionRecord.ExceptionCode;
    const exception_addr = debug_event.u.Exception.ExceptionRecord.ExceptionAddress;

    // Received the signal the debugee is ready.
    if (exception_code == 0xE0000001) {
        // ...
        try setBreakpoint(process_id, break_at_address);
    }

    // A software breakpoint has been hit.
    if (exception_code == win.EXCEPTION_BREAKPOINT) {
        log.info("Breakpoint hit! Continue?", .{});
        _ = try stdin.takeDelimiter('\n') orelse unreachable;

        try handleBreakpoint(process_id, thread_id, exception_addr);
    }
}

To continue from our breakpoint we must restore and execute the original instruction we replaced.

pub fn handleBreakpoint(process_id: win.DWORD, thread_id: win.DWORD, exception_address: win.PVOID) !void {
    if (breakpoint_address != @intFromPtr(exception_address)) {
        return; // Make sure it's our breakpoint.
    }

    // Restore the original instruction so we can execute it.
    _ = try swapProcessByte(process_id, breakpoint_address, orginal_instruction_byte);

If you remember from the previous part, an exception is either a fault, trap or abort. The breakpoint exception is a trap, meaning it completed executing the int3 instruction. Therefore we must move the instruction pointer back 1 byte to execute the original instruction.

We must also re-apply the int3 after executing the original instruction, so the breakpoint is set again. X86 has another feature which can help, the trap flag found on the EFlags register. This tells the CPU to execute only 1 instruction before raising EXCEPTION_SINGLE_STEP again. This is the same exception our hardware breakpoints used, and it’s name makes a bit more sense now.

We use the GetThreadContext/SetThreadContext implemented in the previous part to modify our registers. Than execute and re-apply the breakpoint.

    const thread_handle, var thread_ctx = try getThreadContext(thread_id);
    {
        // Step back one instruction, to right before our breakpoint got executed.
        thread_ctx.Rip = breakpoint_address;

        // Tell the CPU to only execute 1 instruction before raising an exception again.
        const trap_flag_bit = 0x00000100;
        thread_ctx.EFlags |= trap_flag_bit;
    }
    try setThreadContext(thread_handle, thread_ctx);

    // Execute the restored instruction.
    var debug_event = std.mem.zeroes(win.DEBUG_EVENT);
    _ = win.ContinueDebugEvent(process_id, thread_id, win.DBG_CONTINUE);
    _ = win.WaitForDebugEvent(&debug_event, win.INFINITE);

    std.debug.assert(debug_event.dwDebugEventCode == win.EXCEPTION_DEBUG_EVENT);
    const exception_code = debug_event.u.Exception.ExceptionRecord.ExceptionCode;
    std.debug.assert(exception_code == win.EXCEPTION_SINGLE_STEP);

    // Reinsert the breakpoint.
    orginal_instruction_byte = try swapProcessByte(process_id, breakpoint_address, breakpoint_instruction);

    log.info("Resumed execution.", .{});
}

Result

Trying out this code you’ll notice something strange: we immediately hit a breakpoint that’s not ours! This is actually a Windows feature called the initial breakpoint. When Windows has loaded our program into memory, but before execution begins, it triggers a breakpoint. This is normally the point at which real debuggers can setup any breakpoints required. We’ll simply ignore it though.

// A software breakpoint has been hit.
if (exception_code == win.EXCEPTION_BREAKPOINT) {
    if (!initial_breakpoint_hit) {
        initial_breakpoint_hit = true;
    } else {
        log.info("Breakpoint hit! Continue?", .{});
        _ = try stdin.takeDelimiter('\n') orelse unreachable;

        try handleBreakpoint(process_id, thread_id, exception_addr);
    }
}

Now let’s see if our software breakpoint works…

info(debugee): Address of doWork() == 0x7FF6FC771720
info(debugee): Address of counter == 0x7FF6FC839600
info(debugee): Waiting for debugger...
info(debugger): Set software breakpoint at address?
0x7FF6FC771720
info(debugger): Successfully set breakpoint at address 0x7FF6FC771720
info(debugee): Starting work!
info(debugger): Breakpoint hit! Continue?
y
info(debugger): Resumed execution.
info(debugee): 0
info(debugger): Breakpoint hit! Continue?
y
info(debugger): Resumed execution.
info(debugee): 1
info(debugger): Breakpoint hit! Continue?

Identical to our hardware breakpoints, but without the same limitations! Of course hardware breakpoints still have their strengths. They don’t require modifying code (which might not be possible), work per thread and can be used as data breakpoints too. Therefore most debuggers allow you to set both types. But any mainstream debugger is likely to use software instruction breakpoints the most.

In the next part we’ll explore if we can replace data breakpoints with a software version too. We’ll into overcoming their hardware limitations too, allowing to place more than 4 at once and watch memory beyond 8 bytes.

Footers

Part 2: Hardware Breakpoints

Sat, 02 May 2026 00:00:02 +0000

The code for this part can be found here. Feeling lost? Consider starting at part 1.

In the previous post we setup a basic debugger structure. In this post we’ll finally get to the exiting part: setting a breakpoint! There are two types of breakpoints, hardware breakpoints and software breakpoints. This part will focus on hardware breakpoints and we’ll discuss what software breakpoints even are in the next post.

A hardware breakpoint, as the name implies, is implemented by the CPU hardware. The details differ per architecture, but on x86 this is controlled by the debug registers, of which there are 8. Registers DR0 through DR3 specify the address of a breakpoint. We’ll ignore DR4 and DR5 as they are generally unused. Register DR6 is the debug status register and DR7 the debug control register. We’ll explain further how each of them works below, but the Wikipedia page is a good reference if you need to know more beyond this post.

Manipulating registers

You probably know that a CPU executes on assembly and registers. Each thread in our program has a register context associated with it holding the current values of whatever the thread was executing. Even the location of the current assembly instruction being executed is stored in here, by the register named instruction program counter (RIP). This allows the OS to suspend execution by saving the context, than later resume running by restoring all the register values. If we take a peek at the X86_64 CONTEXT structure we can spot our debug registers among it’s fields.

pub const CONTEXT = struct {
    // ...
    EFlags: DWORD, // 32 bit register.
    //...
    Dr0: DWORD64, // 64 bit registers.
    Dr1: DWORD64,
    Dr2: DWORD64,
    Dr3: DWORD64,
    Dr6: DWORD64,
    Dr7: DWORD64,
    //...
    Rip: DWORD64,
    //...
};

If we want to set the debug registers, we’ll need a way to modify this context. Windows provides two functions for this GetThreadContext and SetThreadContext. These functions take a thread handle and return/apply the CONTEXT struct respectively. They are privilege functions though, not just anyone can call them. We need a thread handle with certain access rights.

You may recall from the previous part DEBUG_EVENTgave us a thread id ¹. We can use this to open a thread handle and ask for the get/set thread context permission. Whenever you open a handle, Windows requires you to call CloseHandle when done.

const thread_access = win.THREAD_GET_CONTEXT | win.THREAD_SET_CONTEXT;
const thread_handle = win.OpenThread(thread_access, win.FALSE, thread_id);
_ = win.CloseHandle(thread_handle);

Given the thread handle we ask for the thread context. Notice how we first set the ContextFlags field to CONTEXT_ALL. Windows reads this field to know which values of the struct we are asking for. It might skip certain fields depending on the flags set.

var thread_ctx = std.mem.zeroes(win.CONTEXT);
thread_ctx.ContextFlags = win.CONTEXT_ALL; // Get and set (nearly) all of the thread context.
if (win.GetThreadContext(thread_handle, &thread_ctx) == win.FALSE) {
    return error.GetThreadContextFailed;
}

We’ll be manipulating the context a lot so let’s create some wrapper functions. Repeatedly opening a handle isn’t exactly great, a real debugger should probably store it of to the side. But we’re far from a real debugger.

pub fn getThreadContext(thread_id: win.DWORD) !struct { win.HANDLE, win.CONTEXT } {
    const thread_access = win.THREAD_GET_CONTEXT | win.THREAD_SET_CONTEXT;
    const thread_handle = win.OpenThread(thread_access, win.FALSE, thread_id);

    var thread_ctx = std.mem.zeroes(win.CONTEXT);
    thread_ctx.ContextFlags = win.CONTEXT_ALL; // Get and set (nearly) all of the thread context.
    if (win.GetThreadContext(thread_handle, &thread_ctx) == win.FALSE) {
        _ = win.CloseHandle(thread_handle);
        return error.GetThreadContextFailed;
    }

    return .{ thread_handle, thread_ctx };
}

pub fn setThreadContext(thread_handle: win.HANDLE, thread_ctx: win.CONTEXT) !void {
    const success: win.WINBOOL = win.SetThreadContext(thread_handle, &thread_ctx);
    _ = win.CloseHandle(thread_handle);

    if (success == win.FALSE) {
        return error.SetThreadContextFailed;
    }
}

Setting a breakpoint

To set an instruction breakpoint we first must pick one of DR0 through DR3 to fill out the instruction address we want to break on, easy enough! But we also must enable the register through the DR7 control register. This register is a bitset with fields for each debug register. To visualise its layout lets make use of a Zig feature to specify the bit fields of a 64-bit integer:

pub const Dr7 = packed struct(u64) {
    local_enable_0: u1,
    global_enable_0: u1,
    local_enable_1: u1,
    global_enable_1: u1,
    local_enable_2: u1,
    global_enable_2: u1,
    local_enable_3: u1,
    global_enable_3: u1,

    unused0: u8,

    trigger_condition_0: u2,
    trigger_size_0: u2,
    trigger_condition_1: u2,
    trigger_size_1: u2,
    trigger_condition_2: u2,
    trigger_size_2: u2,
    trigger_condition_3: u2,
    trigger_size_3: u2,

    unused1: u32,
};

As you can see there are 3 unique bits of information here. A 1-bit enable flag, a 2-bit condition flag and a 2-bit size flag. The condition and size reveal a powerful feature of hardware registers. They can not only break on execution, but also on data read/writes!

Condition value 0 is for execution and when set the size must also be 0 to indicate 1-byte. Condition value 1 is for just data writes and 3 for data reads and writes. Size can than be set to either 0 through 3, indicating 1-byte, 2-bytes, 8-bytes or 4-bytes respectively ². We’ll only implement instruction breakpoints here, leaving data breakpoints as an excessive to the reader.

Recall that in the previous part we left of on implementing a setBreakpoint function. Let’s fill it in now! It’s worth noting we only set our breakpoint on a single thread. A real debugger would apply it across all threads, but we ignore multi threading.

pub fn setBreakpoint(thread_id: win.DWORD, break_at_address: usize) !void {
    const thread_handle, var thread_ctx = try getThreadContext(thread_id);

    thread_ctx.Dr0 = break_at_address;

    var debug_control: Dr7 = @bitCast(thread_ctx.Dr7);
    debug_control.local_enable_0 = 1;                   // Enable.
    debug_control.trigger_condition_0 = 0;              // Break on instruction execution.
    debug_control.trigger_size_0 = 0;                   // 1 byte.
    thread_ctx.Dr7 = @bitCast(debug_control);

    try setThreadContext(thread_handle, thread_ctx);
    log.info("Successfully set breakpoint at address 0x{X}", .{break_at_address});
}

Handling a breakpoint hit

When the CPU hits a hardware breakpoint it raises an exception, similar to our custom exception in the previous part. The exception raised is called EXCEPTION_SINGLE_STEP. This might seem like a strange name but will make more sense in the next part. You can probably already imagine what else it might be used for… Let’s add a case to catch this exception.

if (debug_event.dwDebugEventCode == win.EXCEPTION_DEBUG_EVENT) {
    const exception_code = debug_event.u.Exception.ExceptionRecord.ExceptionCode;

    // Received the signal the debugee is ready.
    if (exception_code == 0xE0000001) {
        // ...
        try setBreakpoint(thread_id, break_at_address);
    }

    // A hardware breakpoint has been hit.
    if (exception_code == win.EXCEPTION_SINGLE_STEP) {
        log.info("Breakpoint hit! Continue?", .{});
        _ = try stdin.takeDelimiter('\n') orelse unreachable;

        try handleBreakpoint(thread_id);
    }
}

So far we skipped over an important detail though. Exceptions can be classified as one of 3 categories:

Faults: happen before an instruction completes, continuing will try to re-execute.
Traps: happen after an instruction completes, continuing does not re-execute.
Aborts: happen upon unrecoverable errors.

Hardware breakpoints cause a fault exception, stopping before the instruction executes. With our breakpoint still set, resuming execution we would immediately break again! So you might imagine we’ll have to perform a complicated dance involving temporarily removing our breakpoint… and belief me we’ll get there… in another part! X86_64 actually has a feature for this, the resume flag. This 1-bit flag is found on the EFlags register and will disable hardware breakpoints for 1 instruction, than clear itself.

We can change the EFlags register the same way as before. We’ll also check the debug status register DR6 to confirm it was our breakpoint, DR0, that got hit.

pub fn handleBreakpoint(thread_id: win.DWORD) !void {
    const thread_handle, var thread_ctx = try getThreadContext(thread_id);

    const debug_status_dr0 = thread_ctx.Dr6 & 0x1;
    std.debug.assert(debug_status_dr0 == 1); // Check DR0 was hit by reading debug status bit 0.

    const resume_flag_bit = 0x00010000;
    thread_ctx.EFlags |= resume_flag_bit; // Disable breakpoints for 1 instruction.

    try setThreadContext(thread_handle, thread_ctx);
    log.info("Resumed execution.", .{});
}

Result

Let’s set a breakpoint on the start of the doWork() function. Try it out, and… we got a hit! Continuing through, we can see the function executes, incrementing counter, before entering our function in a loop again. Perfect!

info(debugee): Address of doWork() == 0x7FF72A1B1720
info(debugee): Address of counter == 0x7FF72A279600
info(debugee): Waiting for debugger...
info(debugger): Set hardware breakpoint at address?
0x7FF72A1B1720
info(debugger): Successfully set breakpoint at address 0x7FF72A1B1720
info(debugee): Starting work!
info(debugger): Breakpoint hit! Continue?
y
info(debugger): Resumed execution.
info(debugee): 0
info(debugger): Breakpoint hit! Continue?
y
info(debugger): Resumed execution.
info(debugee): 1
info(debugger): Breakpoint hit! Continue?

It should also be trivial to implement a data breakpoint watching for writes to counter instead.

There are however quite some limitations with hardware breakpoints. For one, we can only set a maximum of 4! If you ever got a debugger message complaining you can’t set more than 4 data breakpoints, you might start the see why. But most debuggers can at least set more than 4 instruction breakpoints? What’s going on? This is where software breakpoints come into play. We’ll take a look at these in the next part.

Footnotes

Part 1: Debugger Setup

Sat, 02 May 2026 00:00:01 +0000

Introduction

Any systems programmer probably uses a debugger near daily. So when I needed to write my own, I was surprised to find how little information is out there on how they work! Don’t get me wrong, I can highly recommend TimDbg’s series on Windows debugging and Sy Brands equivalent for Linux. They’re excellent for getting started! But my hope is to contribute by delving even deeper into topics.

In this mini series we’ll focus on just one aspect: breakpoints. This means we won’t be looking at setting up a proper debugger loop or parsing debug info. Parsing debug info along would be enough to fill a book! But if you are interested in that, maybe PdbPlus can provide some reference for Windows at least.

The code is aimed at Windows X86_64 as that’s what I’m familiar with, but any ideas should apply more generally. All code will be written in Zig 0.16, using nothing but the std-lib and Windows header. I won’t assume any in depth knowledge of Zig though. As long as you have familiarity with any C-like language you should be able to follow along.

The code for this part can be found here.
Alright enough talk!

The Debugee

First, have a look at the program we’ll be debugging throughout this series:

const std = @import("std");
const win = @import("windows");
const log = std.log.scoped(.debugee);

var counter: u64 = 0;
fn doWork() void {
    log.info("{}", .{counter});
    counter += 1;
}

pub fn main(init: std.process.Init) !void {
    log.info("Address of doWork() == 0x{X}", .{@intFromPtr(&doWork)});
    log.info("Address of counter == 0x{X}", .{@intFromPtr(&counter)});

    // Briefly wait to allow our debugger to setup breakpoints at the above addresses.
    // A real debugger would have gotten these from the debug data and setup on process creation.
    log.info("Waiting for debugger...", .{});
    win.RaiseException(0xE0000001, 0, 0, null);
    log.info("Starting work!", .{});

    while (true) {
        doWork();
        try std.Io.sleep(init.io, .fromSeconds(1), .real);
    }
}

For the most part it’s a simple counter within a 1 second loop. The RaiseException call stands out though. Catching exceptions is one of the main ways debuggers interact with a program. You might be familiar with “Stack Overflow”, “Access Violation” and “Divide By Zero” among others. But programs are allowed to define their own. Here we raise a custom exception code we can catch in the debugger.

The reason we do this is so we can figure out the memory addresses of doWork() and counter to initialise our debugger. You might have heard of address space layout randomization before. This is a security technique where the OS might place code and data at random addresses each run. Normally a debugger finds out where everything is placed by parsing the debug info. But as mentioned we won’t be delving into that topic, so use this slightly crude way instead.

Lets run our program to confirm it’s output.

info(debugee): Address of doWork() == 0x7FF720C71710
info(debugee): Address of counter == 0x7FF720D39600
info(debugee): Waiting for debugger...
info(debugee): Starting work!
info(debugee): 0
info(debugee): 1
info(debugee): 2

Attaching the debugger

To debug a program we’ll have to ask the OS. There are two ways to do this. If you want to attach to an already running program, Windows provides DebugActiveProcess. The alternative is to spawn the debugee ourselves through CreateProcess and pass either the DEBUG_PROCESS or DEBUG_ONLY_THIS_PROCESS creation flag. This will spawn a new process in the attached state.

We’ll be using the later method here. As with many Windows API functions there is an A (ansi string) and a W (wide string) variant. We’ll be using CreateProcessA for simplicity, but any serious debugger should aim to support Unicode wide strings. Using this our debugger can spawn a new process by invoking the command line we give it. You may realise that if we can spawn a new process, there is nothing stopping our debugee from doing the same. It is in fact rather common for processes to spawn other processes, think for example the console used to display output. We only want to focus on our debugee though so will be passing the DEBUG_ONLY_THIS_PROCESS flag. Passing DEBUG_PROCESS would cause our debugger to also attach to any further child processes spawned.

Also note we won’t be passing the CREATE_NEW_CONSOLE flag, so our debugee shares the output console with the debugger.

var launch_args = "debugee.exe".*;
var process_info = std.mem.zeroes(win.PROCESS_INFORMATION);
var startup_info = std.mem.zeroes(win.STARTUPINFOA);
startup_info.cb = @sizeOf(win.STARTUPINFOA);

if (win.CreateProcessA(
    null,                           // lpApplicationName (null, use command line instead).
    &launch_args,                   // lpCommandLine
    null,                           // lpProcessAttributes
    null,                           // lpThreadAttributes
    win.FALSE,                      // bInheritHandles
    win.DEBUG_ONLY_THIS_PROCESS,    // dwCreationFlags (Debug only this process, no child processes).
    null,                           // lpEnvironment
    null,                           // lpCurrentDirectory
    &startup_info,                  // lpStartupInfo
    &process_info,                  // lpProcessInformation
) == win.FALSE) {
    log.err("Unable to launch debugee process!", .{});
    return;
}

Watching debug events

With the debugger attached Windows will send us DEBUG_EVENT messages. To receive them we can call WaitForDebugEvent. When an event is raised the debugee is suspended to allow the debugger to respond. To continue the debugee program, ContinueDebugEvent must be called. Let’s setup a basic event handling loop with this:

while (true) {
    var debug_event = std.mem.zeroes(win.DEBUG_EVENT);
    if (win.WaitForDebugEvent(&debug_event, win.INFINITE) == win.FALSE) {
        log.err("Unexpected error waiting for debug event!", .{});
        break;
    }
    const process_id = debug_event.dwProcessId;
    const thread_id = debug_event.dwThreadId;

    // TODO: Do stuff...

    if (win.ContinueDebugEvent(process_id, thread_id, win.DBG_CONTINUE) == win.FALSE) {
        log.err("Unexpected error continuing from debug event!", .{});
        break;
    }
}

Looking at the ContinueDebugEvent you can see we pass in 3 parameters:

The first, process_id, should make some sense to you already. If debuggers can attach to multiple processes at once, we need to know which process was suspended and continue just that one. Of course we specified the DEBUG_ONLY_THIS_PROCESS flag, so know it will always be the same single process we are attached to.

The second, thread_id, specifies which thread to continue. In multi threaded programs only the thread generating the exception is suspended, the others continue running. This is an important detail! In a real debugger you probably want to suspend all threads. It would be weird if other threads continue to change memory while the user is watching it in the debugger. This could even lead to crashes if we try to modify values. To keep things simple though, we’ll only handle single threaded debugees here.

The third parameter is set to DBG_CONTINUE. When an exception is raised, we might not be the only ones trying to handle it. If we decide that actually, something else should catch this we can pass DBG_EXCEPTION_NOT_HANDLED. But in our case we will always continue running.

Handling exceptions

Looking at the DEBUG_EVENT structure there is a whole bunch of things we could respond to:

pub const DEBUG_EVENT = extern struct {
    dwDebugEventCode: DWORD,
    dwProcessId: DWORD,
    dwThreadId: DWORD,
    u: extern union {
        Exception:          EXCEPTION_DEBUG_INFO,
        CreateThread:       CREATE_THREAD_DEBUG_INFO,
        CreateProcessInfo:  CREATE_PROCESS_DEBUG_INFO,
        ExitThread:         EXIT_THREAD_DEBUG_INFO,
        ExitProcess:        EXIT_PROCESS_DEBUG_INFO,
        LoadDll:            LOAD_DLL_DEBUG_INFO,
        UnloadDll:          UNLOAD_DLL_DEBUG_INFO,
        DebugString:        OUTPUT_DEBUG_STRING_INFO,
        RipInfo:            RIP_INFO,
    },
};

For now, let’s just try and catch the custom exception we raised inside the debugee. You can probably guess this means listening for the EXCEPTION_DEBUG_EVENT. Buried within EXCEPTION_DEBUG_INFO, EXCEPTION_RECORD we find the ExceptionCode that was raised.

pub const EXCEPTION_RECORD = extern struct {
    ExceptionCode: DWORD,
    ExceptionFlags: DWORD,
    ExceptionRecord: *EXCEPTION_RECORD,
    ExceptionAddress: PVOID,
    NumberParameters: DWORD,
    ExceptionInformation: [15]ULONG_PTR,
};

When our custom exception code is detected we should ask the user where they would like to set the breakpoint.

if (debug_event.dwDebugEventCode == win.EXCEPTION_DEBUG_EVENT) {
    const exception_code = debug_event.u.Exception.ExceptionRecord.ExceptionCode;

    // Received the signal the debugee is ready.
    if (exception_code == 0xE0000001) {
        log.info("Set hardware breakpoint at address?", .{});
        const input_line_bare = try stdin.takeDelimiter('\n') orelse unreachable;
        const input_line = std.mem.trim(u8, input_line_bare, "\r");
        const break_at_address = try std.fmt.parseInt(usize, input_line, 0);

        // TODO: setBreakpoint(break_at_address);
    }
}

Going forward, we’ll be implementing setBreakpoint. But for now lets take a breather here and check we successfully caught the exception:

info(debugee): Address of doWork() == 0x7FF70B1F1720
info(debugee): Address of counter == 0x7FF70B2B9600
info(debugee): Waiting for debugger...
info(debugger): Set hardware breakpoint at address?
0x7FF70B1F1720
info(debugee): Starting work!
info(debugee): 0
info(debugee): 1
info(debugee): 2

Excellent! With all of that boilerplate out of the way, in the next part will implement our first type of breakpoint: hardware breakpoints. For now it’s important to realise the basic loop on how debugees raise exceptions that debuggers can catch and tell the debugee what to do.

Part 5: ParallelFor Using ParallelWriter With Thread Index

Wed, 18 Mar 2020 00:00:05 +0000

Introduction

Previously in this series we looked into creating our own custom native container and adding support for features such as deallocation on job completion and multiple ways to add parallel jobs support. In this part we will look into yet another way to add support for parallel writing using [NativeSetThreadIndex].

This article won’t use code from previous articles, but will instead implement a completely new native container. It is therefor assumed that you understand how to do this. If not, you can go back and read the previous articles in this series.

The result of the previous article can be found here.
The final result of this article can be found here.

1) NativeSummedFloat3 Setup

The container we will implement is called NativeSummedFloat3. This container holds a single float3, but allows multiple threads to add to it in parallel. This can be useful when for instance calculating the average position of a large set of entities.

In the code below we do all the basic setup for our custom container. But interesting to note is the amount of memory we allocate. We will be allocating a single cache line of each worker thread. This allows us to make our container thread safe. By having each thread write to it’s own part of memory, the cache line, there will never by multiple threads writing to the same memory. It also allows for better cache access, giving a performance optimization. The downside is that we will be allocating a lot of memory (8Kb in total as of writing). We can’t allocate anything less than a single cache line per worker thread, because the CPU will always load a single cache line when accessing data.

using System;
using System.Runtime.InteropServices;
using Unity.Burst;
using Unity.Collections;
using Unity.Collections.LowLevel.Unsafe;
using Unity.Jobs;
using Unity.Jobs.LowLevel.Unsafe;
using Unity.Mathematics;

[NativeContainer]
[NativeContainerSupportsDeallocateOnJobCompletion]
[StructLayout(LayoutKind.Sequential)]
public unsafe struct NativeSummedFloat3 : IDisposable
{
    [NativeDisableUnsafePtrRestriction] internal void* m_Buffer;

#if ENABLE_UNITY_COLLECTIONS_CHECKS
    internal AtomicSafetyHandle m_Safety;
    [NativeSetClassTypeToNullOnSchedule] internal DisposeSentinel m_DisposeSentinel;
#endif

    internal Allocator m_AllocatorLabel;

    public NativeSummedFloat3(Allocator allocator)
    {
        // Safety checks
#if ENABLE_UNITY_COLLECTIONS_CHECKS
        if (allocator <= Allocator.None)
            throw new ArgumentException("Allocator must be Temp, TempJob or Persistent", nameof(allocator));

        // There are other checks you might want to perform when working with generic containers and cache lines.
        /*
        if (!UnsafeUtility.IsBlittable<T>())
            throw new ArgumentException(string.Format("{0} used in NativeValue<{0}> must be blittable", typeof(T)));
        if (UnsafeUtility.SizeOf<T>() > JobsUtility.CacheLineSize)
            throw new ArgumentException(string.Format("{0} used in NativeValue<{0}> had a size of {1} which is greater than the maximum size of {2}", typeof(T), UnsafeUtility.SizeOf<T>(), JobsUtility.CacheLineSize));
        */

        DisposeSentinel.Create(out m_Safety, out m_DisposeSentinel, 0, allocator);
#endif

        // Allocate a cache line for each worker thread.
        m_Buffer = UnsafeUtility.Malloc(JobsUtility.CacheLineSize * JobsUtility.MaxJobThreadCount, JobsUtility.CacheLineSize, allocator);
        m_AllocatorLabel = allocator;
        Value = float3.zero;
    }

    // Allows NativeSummedFloat3 to be cast to float3.
    public static implicit operator float3(NativeSummedFloat3 value) { return value.Value; }

    /*
     * ... Next Code ...
     */

Lets get all the boring code out of the way by immediately adding the end part of our NativeSummedFloat3 struct. Again, more about how this code works can be found in previous parts of this series.

    /*
     * ... Previous Code ...
     */

    [WriteAccessRequired]
    public void Dispose()
    {
#if ENABLE_UNITY_COLLECTIONS_CHECKS
        if (!UnsafeUtility.IsValidAllocator(m_AllocatorLabel))
            throw new InvalidOperationException("The NativeSummedFloat3 can not be Disposed because it was not allocated with a valid allocator.");

        DisposeSentinel.Dispose(ref m_Safety, ref m_DisposeSentinel);
#endif

        // Free the allocated memory and reset our variables.
        UnsafeUtility.Free(m_Buffer, m_AllocatorLabel);
        m_Buffer = null;
    }

    public unsafe JobHandle Dispose(JobHandle inputDeps)
    {
#if ENABLE_UNITY_COLLECTIONS_CHECKS
        if (!UnsafeUtility.IsValidAllocator(m_AllocatorLabel))
            throw new InvalidOperationException("The NativeSummedFloat3 can not be Disposed because it was not allocated with a valid allocator.");

        // DisposeSentinel needs to be cleared on the main thread.
        DisposeSentinel.Clear(ref m_DisposeSentinel);
#endif

        NativeSummedFloat3DisposeJob disposeJob = new NativeSummedFloat3DisposeJob()
        {
            Data = new NativeSummedFloat3Dispose() { m_Buffer = m_Buffer, m_AllocatorLabel = m_AllocatorLabel }
        };
        JobHandle result = disposeJob.Schedule(inputDeps);

#if ENABLE_UNITY_COLLECTIONS_CHECKS
        AtomicSafetyHandle.Release(m_Safety);
#endif

        m_Buffer = null;
        return result;
    }
}

[NativeContainer]
internal unsafe struct NativeSummedFloat3Dispose
{
    [NativeDisableUnsafePtrRestriction] internal void* m_Buffer;
    internal Allocator m_AllocatorLabel;
    public void Dispose() { UnsafeUtility.Free(m_Buffer, m_AllocatorLabel); }
}

[BurstCompile]
internal struct NativeSummedFloat3DisposeJob : IJob
{
    internal NativeSummedFloat3Dispose Data;
    public void Execute() { Data.Dispose(); }
}

2) Single Thread Getter And Setter

Lets implement a getter and setter, that can only be accessed from a single thread. The getter here is interesting. We loop over each cache line and add the values together for the final result. We use ReadArrayElementWithStride because our array elements are the size of a cache line, but we’re only interested in the float3 stored at the beginning.

The setter first resets all cache lines to 0 and than add the value. We will have a look at these methods next.

    /*
     * ... Other Code ...
     */

    public float3 Value
    {
        get
        {
#if ENABLE_UNITY_COLLECTIONS_CHECKS
            AtomicSafetyHandle.CheckReadAndThrow(m_Safety);
#endif
            // Sum the values stored on each worker threads cache line.
            float3 result = UnsafeUtility.ReadArrayElement<float3>(m_Buffer, 0);
            for (int i = 1; i < JobsUtility.MaxJobThreadCount; i++)
                result += UnsafeUtility.ReadArrayElementWithStride<float3>(m_Buffer, i, JobsUtility.CacheLineSize);

            return result;
        }

        [WriteAccessRequired]
        set
        {
#if ENABLE_UNITY_COLLECTIONS_CHECKS
            AtomicSafetyHandle.CheckWriteAndThrow(m_Safety);
#endif
            Reset();
            AddValue(value);
        }
    }

    /*
     * ... Next Code ...
     */

3) Single Thread Methods

The AddValue and Reset methods access the cache lines in a similar manner as our getter. We don’t have to worry about multiple writers yet, so we can use WriteArrayElement and just write to the first cache line. For Reset however we need to use WriteArrayElementWithStride again because our elements are the size of a cache line.

    /*
     * ... Previous Code ...
     */

    [WriteAccessRequired]
    public void AddValue(float3 value)
    {
#if ENABLE_UNITY_COLLECTIONS_CHECKS
        AtomicSafetyHandle.CheckWriteAndThrow(m_Safety);
#endif
        // Add a value to the sum. We're writing from a single thread, so we'll write to the first cache line.
        float3 current = UnsafeUtility.ReadArrayElement<float3>(m_Buffer, 0);
        current += value;
        UnsafeUtility.WriteArrayElement(m_Buffer, 0, current);
    }

    [WriteAccessRequired]
    public void Reset()
    {
#if ENABLE_UNITY_COLLECTIONS_CHECKS
        AtomicSafetyHandle.CheckWriteAndThrow(m_Safety);
#endif
        // Reset each worker threads cache line to float3.zero.
        for (int i = 0; i < JobsUtility.MaxJobThreadCount; i++)
            UnsafeUtility.WriteArrayElementWithStride(m_Buffer, i, JobsUtility.CacheLineSize, float3.zero);
    }

    /*
     * ... Next Code ...
     */

4) Parallel Writer With Thread Index

Now for the fun part, parallel writing. We add code within the NativeSummedFloat3 struct for creating a parallel writer object, as explained in previous articles. But interesting to note is [NativeSetThreadIndex] and the m_ThreadIndex variable. Watch out as naming is important here! This variable will receive the thread index when its scheduled with a job. We than use that variable as index into the cache line to read an write from.

    /*
     * ... Previous Code ...
     */

    [NativeContainerIsAtomicWriteOnly]
    [NativeContainer]
    unsafe public struct ParallelWriter
    {
        [NativeDisableUnsafePtrRestriction] internal void* m_Buffer;

#if ENABLE_UNITY_COLLECTIONS_CHECKS
        internal AtomicSafetyHandle m_Safety;
#endif

        [NativeSetThreadIndex]
        internal int m_ThreadIndex;

        [WriteAccessRequired]
        public void AddValue(float3 value)
        {
#if ENABLE_UNITY_COLLECTIONS_CHECKS
            AtomicSafetyHandle.CheckWriteAndThrow(m_Safety);
#endif
            // Add a value to the sum. We're writing in parallel, so we'll write to the cache line assigned to this thread.
            float3 current = UnsafeUtility.ReadArrayElementWithStride<float3>(m_Buffer, m_ThreadIndex, JobsUtility.CacheLineSize);
            current += value;
            UnsafeUtility.WriteArrayElementWithStride(m_Buffer, m_ThreadIndex, JobsUtility.CacheLineSize, current);
        }
    }

    public ParallelWriter AsParallelWriter()
    {
        ParallelWriter writer;

#if ENABLE_UNITY_COLLECTIONS_CHECKS
        AtomicSafetyHandle.CheckWriteAndThrow(m_Safety);
        writer.m_Safety = m_Safety;
        AtomicSafetyHandle.UseSecondaryVersion(ref writer.m_Safety);
#endif
        writer.m_Buffer = m_Buffer;
        writer.m_ThreadIndex = 0; // Thread index will be set by the job schedular later.

        return writer;
    }

    /*
     * ... More Code ...
     */

Usage

That’s all we have to do! With this we have a created a custom native container that allows for parallel writing by making use of the thread index. The code below shows how we can use this to calculate the average position of all entities with a LocalToWorld component in the scene.

using Unity.Collections;
using Unity.Entities;
using Unity.Jobs;
using Unity.Mathematics;
using Unity.Transforms;

public class NativeSummedFloat3System : SystemBase
{
    private EntityQuery localToWorldQuery;

    protected override void OnUpdate()
    {
        NativeSummedFloat3 avgPosition = new NativeSummedFloat3(Allocator.TempJob);
        NativeSummedFloat3.ParallelWriter avgPositionParallelWriter = avgPosition.AsParallelWriter();

        // Sum together all positions of entities with a LocalToWorld component.
        JobHandle jobHandle = Entities.WithName("AvgPositionJob")
            .WithStoreEntityQueryInField(ref localToWorldQuery)
            .ForEach((in LocalToWorld localToWorld) =>
            {
                avgPositionParallelWriter.AddValue(localToWorld.Position);
            }).ScheduleParallel(default);

        jobHandle.Complete();

        // We store the query so we can calculate how many entities have the LocalToWorld component.
        int entityCount = localToWorldQuery.CalculateEntityCount();
        UnityEngine.Debug.Log(avgPosition.Value / entityCount);

        avgPosition.Dispose();
    }
}

Conclusion

In this article we wrote a new custom native container that used thread index for parallel writing by assigning each thread it’s own memory/cache line. This allowed us to create a float3 that can be written to in parallel. But we can also make this container generic, to allow for any value to be operated on in parallel (as long as they are smaller than a cache line). The generic version of this container can be found here along with an example on how to use it here

Part 4: Parallel Job Using ParallelWriter

Wed, 18 Mar 2020 00:00:04 +0000

Introduction

In the previous article in this series, Unity DOTS Custom Native Container Part 3: Parallel Job Using Min Max, we added support for parallel jobs. But these jobs were limited to writing to a single index of the array. In this article we will remove this limitation from our NativeIntArray by adding support for ParallelWriter. The article assumes basic (C#) multithreading knowledge.

The result of the previous article can be found here.
The final result of this article can be found here.

1) ParallelWriter Struct

First we must add a ParallelWriter struct within our NativeIntArray struct. This is essentially a new container that only allows writing to the array, but allows multiple threads to do so. The actual write operations are implemented using the Interlocked class. This class provides atomic operations. More information can be found here.

    /*
     * ... More Code ...
     */

    // Allow parallel writing through NativeIntArray.ParallelWriter in a parallel job.
    // No reading allowed.
    [NativeContainerIsAtomicWriteOnly]
    [NativeContainer]
    unsafe public struct ParallelWriter
    {
        // Copy pointer of the full container.
        [NativeDisableUnsafePtrRestriction] internal void* m_Buffer;
        internal int m_Length;

        // Copy the safty handle. The dispose sentinal doesn't need to be copied as no memory will be allocated within this struct.
#if ENABLE_UNITY_COLLECTIONS_CHECKS
        internal AtomicSafetyHandle m_Safety;
#endif
        // Copy length for convenience
        public int Length => m_Length;

        public int Increment(int index)
        {
            // Increment still needs to safety check for write permissions and index range.
#if ENABLE_UNITY_COLLECTIONS_CHECKS
            AtomicSafetyHandle.CheckWriteAndThrow(m_Safety);
            if (index < 0 || index > Length)
                throw new IndexOutOfRangeException(string.Format("Index {0} is out of range of '{1}' Length.", index, Length));
#endif
            // Increment is implemented as an atomic operation since it can be incremented by multiple threads at the same time.
            return Interlocked.Increment(ref *((int*)m_Buffer + index));
        }

        public int Decrement(int index)
        {
#if ENABLE_UNITY_COLLECTIONS_CHECKS
            AtomicSafetyHandle.CheckWriteAndThrow(m_Safety);
            if (index < 0 || index > Length)
                throw new IndexOutOfRangeException(string.Format("Index {0} is out of range of '{1}' Length.", index, Length));
#endif
            return Interlocked.Decrement(ref *((int*)m_Buffer + index));
        }

        public int Add(int index, int value)
        {
#if ENABLE_UNITY_COLLECTIONS_CHECKS
            AtomicSafetyHandle.CheckWriteAndThrow(m_Safety);
            if (index < 0 || index > Length)
                throw new IndexOutOfRangeException(string.Format("Index {0} is out of range of '{1}' Length.", index, Length));
#endif
            return Interlocked.Add(ref *((int*)m_Buffer + index), value);
        }
    }

    /*
     * ... More Code ...
     */

2) AsParallelWriter

We define a function to create a NativeIntArray.ParallelWriter out of our container. Its implementation listed below should be pretty straight forward.

    /*
     * ... Previous Code ...
     */

    public ParallelWriter AsParallelWriter()
    {
        ParallelWriter writer;

#if ENABLE_UNITY_COLLECTIONS_CHECKS
        AtomicSafetyHandle.CheckWriteAndThrow(m_Safety);
        writer.m_Safety = m_Safety;
        AtomicSafetyHandle.UseSecondaryVersion(ref writer.m_Safety);
#endif
        writer.m_Buffer = m_Buffer;
        writer.m_Length = m_Length;

        return writer;
    }

    /*
     * ... More Code ...
     */

Usage

Thats all we need to implement parallel writing. To prove that our container is in fact now capable handling multiple writers, lets implement something visually interesting. The job below picks a random index in the container and increments it’s value in parallel. Random indices are picked according to a normal distribution thats than drawn to the screen as a bar graph. This results in a Galton board!

using Unity.Burst;
using Unity.Collections;
using Unity.Entities;
using Unity.Jobs;
using Unity.Mathematics;

public class NativeIntArraySystem : SystemBase
{
    [BurstCompile]
    struct ParallelWriteNormalDistributionJob : IJobParallelFor
    {
        public Random random;
        public NativeIntArray.ParallelWriter array;

        public void Execute(int index)
        {
            // Calculate normal distribution.
            double u1 = 1.0 - random.NextDouble();
            double u2 = 1.0 - random.NextDouble();
            double randomStdNormal = math.sqrt(-2.0 * math.log(u1)) * math.sin(2.0 * math.PI * u2);
            double randomNormal = (array.Length / 2) + randomStdNormal * (array.Length / 8);

            // Use the normal distribution to pick an element to increment.
            int arrayIndex = math.clamp((int)randomNormal, 0, array.Length - 1);

            // Use our atomic operation.
            array.Increment(arrayIndex);
        }
    }

    protected override void OnUpdate()
    {
        NativeIntArray myArray = new NativeIntArray(100, Allocator.TempJob);

        // Fill myArray with normal distribution values.
        JobHandle jobHandle = new ParallelWriteNormalDistributionJob()
        {
            random = new Random((uint)UnityEngine.Random.Range(0, int.MaxValue)),
            array = myArray.AsParallelWriter()
        }.Schedule(10000, 64); // Run our job a 10000 times in batches of 64 (values chosen randomly).

        jobHandle.Complete();

        // Draw each element in myArray as a bar graph where it's value is the height of the bar.
        Job.WithName("DrawBarGraph")
            .WithReadOnly(myArray)
            .WithoutBurst()
            .WithCode(() =>
            {
                for (int i = 0; i < myArray.Length; i++)
                {
                    float barWidth = 1.0f;
                    float barHeight = (myArray[i] / 40.0f) * 10.0f;
                    DrawBar(new float2(i * barWidth, 0), new float2(barWidth, barHeight));
                }
            }).Run();


        myArray.Dispose();
    }

    private void DrawBar(float2 position, float2 size)
    {
        UnityEngine.Color color = UnityEngine.Color.red;
        float3 lowerBound = new float3(position.xy, 0);

        UnityEngine.Debug.DrawLine(lowerBound, lowerBound + new float3(size.x, 0, 0), color);
        UnityEngine.Debug.DrawLine(lowerBound, lowerBound + new float3(0, size.y, 0), color);
        UnityEngine.Debug.DrawLine(lowerBound, lowerBound + new float3(size.xy, 0)  , color);

        lowerBound += new float3(size.xy, 0);
        UnityEngine.Debug.DrawLine(lowerBound, lowerBound + new float3(-size.x, 0, 0), color);
        UnityEngine.Debug.DrawLine(lowerBound, lowerBound + new float3(0, -size.y, 0), color);
    }
}

Conclusion

This article showed how to add support for ParallelWriter. Normal concurrent data structure design applies, so we can implement our operations using the Interlocked class. One thing to note however is that our container is not a managed object, and can therefore not be locked to a thread. This means that all native containers need to be designed as lock free data structures. In the next part we will look into how we can use the thread index to implement a new lock free data structure.

Part 3: Parallel Job Using Min Max

Wed, 18 Mar 2020 00:00:03 +0000

Introduction

In previous parts of this series we looked into how to create a basic custom native container that can be used with jobs. This article will improve our NativeIntArray container to add support for parallel jobs. This is done by using a pattern where the job is split into ranges and each job is only allowed to operate on this range. This limits the array access to the index passed through Execute(int index). More information about how these jobs are scheduled behind the scenes can be found in the Unity documentation here.

The result of the previous article can be found here.
The final result of this article can be found here.

1) Enable Support

We add the [NativeContainerSupportsMinMaxWriteRestriction] tag to enable support for this kind of parallel job. We also have to create m_MinIndex and m_MaxIndex variables and initialize them with the entire range of our array. These variables are required for safety checking. Watch out, the naming and order of variables is very important here!

We will also use this opportunity to have a quick reminder of what our container roughly looked like: a simple array of integers.

using System;
using System.Diagnostics;
using System.Runtime.InteropServices;
using System.Threading;
using Unity.Burst;
using Unity.Collections;
using Unity.Collections.LowLevel.Unsafe;
using Unity.Jobs;

// This enables support for parallel job exection where each worker thread 
// is only allowed to operation on a range of indices between min and max.
[NativeContainerSupportsMinMaxWriteRestriction]
[NativeContainerSupportsDeallocateOnJobCompletion]
[NativeContainer]
[StructLayout(LayoutKind.Sequential)] 
public unsafe struct NativeIntArray : IDisposable
{
    [NativeDisableUnsafePtrRestriction] internal void* m_Buffer;
    internal int m_Length;

#if ENABLE_UNITY_COLLECTIONS_CHECKS
    // NativeContainerSupportsMinMaxWriteRestriction expects the passed ranges it can operate on to be checked for safety.
    // The range is passed to the container when an parallel job schedules it's batch jobs.
    internal int m_MinIndex;
    internal int m_MaxIndex;

    internal AtomicSafetyHandle m_Safety;
    [NativeSetClassTypeToNullOnSchedule] internal DisposeSentinel m_DisposeSentinel;
#endif

    internal Allocator m_AllocatorLabel;

    public NativeIntArray(int length, Allocator allocator, NativeArrayOptions options = NativeArrayOptions.ClearMemory){ /* More Code */ }

    static void Allocate(int length, Allocator allocator, out NativeIntArray array)
    {
        long size = UnsafeUtility.SizeOf<int>() * (long)length;

        /* More Code */

        array = default(NativeIntArray);
        array.m_Buffer = UnsafeUtility.Malloc(size, UnsafeUtility.AlignOf<int>(), allocator);
        array.m_Length = length;
        array.m_AllocatorLabel = allocator;

#if ENABLE_UNITY_COLLECTIONS_CHECKS
        // By default the job can operate over the entire range.
        array.m_MinIndex = 0;
        array.m_MaxIndex = length - 1;

        DisposeSentinel.Create(out array.m_Safety, out array.m_DisposeSentinel, 1, allocator);
#endif
    }

    /*
     * ... Next Code ...
     */

2) Range checking

The only other change we need to make to the code is to check if we are within range when accessing an element in the array. All other functions that access the array in this container use the operator [] to do so, therefore it is enough to add our range checks to this operator only.

    /*
     * ... Previous Code ...
     */

    // Remove calls to this function if safety is disabled.
    [Conditional("ENABLE_UNITY_COLLECTIONS_CHECKS")]
    private void CheckRangeAccess(int index)
    {
#if ENABLE_UNITY_COLLECTIONS_CHECKS
        // Check if we're within the range of indices that this parallel batch job operates on.
        if (index < m_MinIndex || index > m_MaxIndex)
        {
            if (index < Length && (m_MinIndex != 0 || m_MaxIndex != Length - 1))
                throw new IndexOutOfRangeException(string.Format(
                    "Index {0} is out of restricted IJobParallelFor range [{1}...{2}] in ReadWriteBuffer.\n" +
                    "ReadWriteBuffers are restricted to only read & write the element at the job index. " +
                    "You can use double buffering strategies to avoid race conditions due to " +
                    "reading & writing in parallel to the same elements from a job.",
                    index, m_MinIndex, m_MaxIndex));

            // This is not a parallel job but the index is still out of range.
            throw new IndexOutOfRangeException(string.Format("Index {0} is out of range of '{1}' Length.", index, Length));
        }
#endif
    }

    public int this[int index]
    {
        get
        {
#if ENABLE_UNITY_COLLECTIONS_CHECKS
            AtomicSafetyHandle.CheckReadAndThrow(m_Safety);
#endif
            CheckRangeAccess(index);
            return UnsafeUtility.ReadArrayElement<int>(m_Buffer, index);
        }

        [WriteAccessRequired]
        set
        {
#if ENABLE_UNITY_COLLECTIONS_CHECKS
            AtomicSafetyHandle.CheckWriteAndThrow(m_Safety);
#endif
            CheckRangeAccess(index);
            UnsafeUtility.WriteArrayElement(m_Buffer, index, value);
        }
    }

    /*
    * ... More Code ...
    */

Usage

And that all! We now have added support for deallocation on job completion to our NativeIntArray. An example of this is shown below.

using Unity.Burst;
using Unity.Collections;
using Unity.Entities;
using Unity.Jobs;
using Unity.Mathematics;

public class NativeIntArraySystem : SystemBase
{
    [BurstCompile]
    struct ParallelWriteRangeJob : IJobParallelFor
    {
        public Random random;
        // See the previous part on how to add support for [DeallocateOnJobCompletion].
        [DeallocateOnJobCompletion] public NativeIntArray array;

        public void Execute(int index)
        {
            array[index] = random.NextInt();
        }
    }
    
    protected override void OnUpdate()
    {
        NativeIntArray myArray = new NativeIntArray(1024, Allocator.TempJob);

        // Fill myArray with random values.
        JobHandle jobHandle = new ParallelWriteRangeJob()
        {
            random = new Random((uint)UnityEngine.Random.Range(0, int.MaxValue)),
            array = myArray
        }.Schedule(myArray.Length, 64, Dependency); // Schedule with a batch size of 64.

        Dependency = jobHandle;
    }
}

Conclusion

This article showed how to add support for parallel jobs using a pattern where the job is split into ranges. But a limitation of this pattern is that it does not allow for multiple jobs to write to the same index. In the next part we will look how we can make this possible by adding support for ParallelWriter.

Part 2: Deallocate On Job Completion

Wed, 18 Mar 2020 00:00:02 +0000

Introduction

In the previous article in this series, Unity DOTS Custom Native Container Part 1: The Basics, we looked into how we can create a bare basic custom native container for usage with the job system. In this article we will extend our NativeIntArray container to add support for usage with .WithDeallocateOnJobCompletion and [DeallocateOnJobCompletion].

The result of the previous article can be found here.
The final result of this article can be found here.

1) Enable Support

To enable support for deallocation on job completion we must add the [NativeContainerSupportsDeallocateOnJobCompletion] attribute to our container struct. We will also use this opportunity to have a quick reminder of what our container roughly looked like: a simple array of integers.

using System;
using System.Diagnostics;
using System.Runtime.InteropServices;
using System.Threading;
using Unity.Burst;
using Unity.Collections;
using Unity.Collections.LowLevel.Unsafe;
using Unity.Jobs;

// Enable support for ".WithDeallocateOnJobCompletion" and "[DeallocateOnJobCompletion]".
[NativeContainerSupportsDeallocateOnJobCompletion]
[NativeContainer] 
[StructLayout(LayoutKind.Sequential)] 
public unsafe struct NativeIntArray : IDisposable
{
    [NativeDisableUnsafePtrRestriction] internal void* m_Buffer;
    internal int m_Length;

#if ENABLE_UNITY_COLLECTIONS_CHECKS
    internal AtomicSafetyHandle m_Safety;
    [NativeSetClassTypeToNullOnSchedule] internal DisposeSentinel m_DisposeSentinel;
#endif

    internal Allocator m_AllocatorLabel;

    public NativeIntArray(int length, Allocator allocator, NativeArrayOptions options = NativeArrayOptions.ClearMemory) { /* More Code */ }

    static void Allocate(int length, Allocator allocator, out NativeIntArray array)
    {
        long size = UnsafeUtility.SizeOf<int>() * (long)length;

        /* More Code */

        array = default(NativeIntArray);
        // Allocate memory for our buffer.
        array.m_Buffer = UnsafeUtility.Malloc(size, UnsafeUtility.AlignOf<int>(), allocator);
        array.m_Length = length;
        array.m_AllocatorLabel = allocator;

        /* More Code */
    }

    /* More Code */

2) JobHandle Dispose

Next we will add a new dispose function to our container struct. This dispose function will not deallocate our container immediately, but will instead return a job handle that can be scheduled later. This is how deallocate on job completion works, by scheduling another job to do the cleanup once our job is completed.

    /*
     * ... More Code ...
     */

    public unsafe JobHandle Dispose(JobHandle inputDeps)
    {
#if ENABLE_UNITY_COLLECTIONS_CHECKS
        // DisposeSentinel needs to be cleared on the main thread.
        DisposeSentinel.Clear(ref m_DisposeSentinel);
#endif

        // Create a job to dispose of our container and pass a copy of our pointer to it.
        NativeCustomArrayDisposeJob disposeJob = new NativeCustomArrayDisposeJob()
        {
            Data = new NativeCustomArrayDispose() 
            { 
                m_Buffer = m_Buffer,
                m_AllocatorLabel = m_AllocatorLabel
            }
        };
        JobHandle result = disposeJob.Schedule(inputDeps);

#if ENABLE_UNITY_COLLECTIONS_CHECKS
        AtomicSafetyHandle.Release(m_Safety);
#endif
        
        m_Buffer = null;
        m_Length = 0;

        return result;
    }

    /*
     * ... More Code ...
     */

3) NativeCustomArrayDisposeJob And NativeCustomArrayDispose

As you may have noticed, inside our Dispose function we make use of two new structs. These need to be defined outside out container struct. NativeCustomArrayDispose is used to hold a copy of our container pointer and NativeCustomArrayDisposeJob will call Dispose. Whenever the job gets scheduled, it will internally make sure that no other job is reading or writing to our container.

/*
* ... More Code ...
*/

[NativeContainer]
internal unsafe struct NativeCustomArrayDispose
{
    // Relax the pointer safety so jobs can schedule with this struct.
    [NativeDisableUnsafePtrRestriction] internal void* m_Buffer;
    internal Allocator m_AllocatorLabel;

    public void Dispose()
    {
        // Free the allocated memory
        UnsafeUtility.Free(m_Buffer, m_AllocatorLabel);
    }
}

[BurstCompile]
internal struct NativeCustomArrayDisposeJob : IJob
{
    internal NativeCustomArrayDispose Data;

    public void Execute()
    {
        Data.Dispose();
    }
}

Usage

And that is it! We now have added support for parallel jobs to our NativeIntArray. An example of this is shown below.

using Unity.Burst;
using Unity.Collections;
using Unity.Entities;
using Unity.Jobs;
using Unity.Mathematics;

public class NativeIntArraySystem : SystemBase
{
    protected override void OnUpdate()
    {
        NativeIntArray myArray = new NativeIntArray(100, Allocator.TempJob);
        Job.WithName("NativeIntArrayJob")
            .WithDeallocateOnJobCompletion(myArray)
            .WithCode(() =>
            {
                for (int i = 0; i < myArray.Length; i++)
                    myArray.Increment(i);
            }).Run();
    }
}

Conclusion

This article showed how to add support for .WithDeallocateOnJobCompletion and [DeallocateOnJobCompletion]. In the next part(s) we will continue to add features to our NativeIntArray by supporting parallel jobs.

Part 1: The Basics

Wed, 18 Mar 2020 00:00:01 +0000

Introduction

Native containers are used for data communication between jobs. Unity already provides a set of native containers in their Collections package, such as NativeList, NativeQueue, NativeHashMap, etc. But when you need something more custom, you can write your own native container.

In this article we will write such a custom container that can be used with jobs. In subsequent articles we will look into adding more advanced features to this container such as adding support for parallel jobs. These articles will not be about how to write a good container type, but rather will seek to demonstrate all the features that can be implemented when writing a custom native container. This article expects basic knowledge of pointers and memory management.

The final result of this article can be found here.)

NativeIntArray

The container we will be implementing is called NativeIntArray. It is a fixed size array of integers, essentially the same as NativeArray<int>. We will purposely use such a simple example so we can focus on the actual native container implementation. The basic structure of this container is shown below. We will override parts of it to turn into a native container.

// We will be working with pointers, so our struct needs to be unsafe.
public unsafe struct NativeIntArray
{
    internal void* m_Buffer; // Array to hold our integers.
    internal int m_Length;

    public NativeIntArray(int length) { /* Allocate memory... */ }

    public int this[int index] // Getter and setter using NativeIntArray[index].
    {
        get { return *((int*)m_Buffer + index); }
        set { *((int*)m_Buffer + index) = value; }
    }

    public int Increment(int index) { return ++this[index]; }

    public int Decrement(int index) { return --this[index]; }

    public int Add(int index, int value) { return (this[index] += value); } 

    public int Length => m_Length;
}

1) Member Variables

Lets first define all the member variables and some of the attributes to turn our struct into a native container. The naming and order of variables is very important here! Make sure to copy it exactly so you do not get weird results. The rest of the code is explained through comments. Do not be afraid if you do not understand what each line does exactly, having a rough idea of what each part does is more important.

using System;
using System.Diagnostics;
using System.Runtime.InteropServices;
using System.Threading;
using Unity.Burst;
using Unity.Collections;
using Unity.Collections.LowLevel.Unsafe;
using Unity.Jobs;

// Needed to mark as a native container.
[NativeContainer] 
// Ensure our memory layout is the same as the order of our variables.
[StructLayout(LayoutKind.Sequential)] 
public unsafe struct NativeIntArray : IDisposable
{
    // Relax the pointer safety so jobs can schedule with this container.
    [NativeDisableUnsafePtrRestriction] internal void* m_Buffer;
    internal int m_Length;

    // This macro makes sure safety features can be disabled for better performance.
#if ENABLE_UNITY_COLLECTIONS_CHECKS
    // Handle to tell if operations such as reading and writing can be performed safely.
    internal AtomicSafetyHandle m_Safety;

    // Handle to tell if the container has been disposed.
    // This is a managed object. It can be passed along as the job can't dispose the container, 
    // but needs to be (re)set to null on schedule to prevent job access to a managed object.
    [NativeSetClassTypeToNullOnSchedule] internal DisposeSentinel m_DisposeSentinel;
#endif

    // Keep track of which memory was allocated (Allocator.Temp/TempJob/Persistent).
    internal Allocator m_AllocatorLabel;

    /*
     * ... Next Code ...
     */

2) Allocating Memory

Next we will write out constructor and Allocate function. The constructor will use the Allocate function to allocate memory for our buffer. The code is again explained through added comments.

    /*
     * ... Previous Code ...
     */

    public NativeIntArray(int length, Allocator allocator, NativeArrayOptions options = NativeArrayOptions.ClearMemory)
    {
        Allocate(length, allocator, out this);

        // Set the memory block to 0 if requested.
        if ((options & NativeArrayOptions.ClearMemory) == NativeArrayOptions.ClearMemory)
            UnsafeUtility.MemClear(m_Buffer, (long)length * UnsafeUtility.SizeOf<int>());
    }

    static void Allocate(int length, Allocator allocator, out NativeIntArray array)
    {
        // Calculate how many bytes are needed.
        long size = UnsafeUtility.SizeOf<int>() * (long)length;

        // Check if this is a valid allocation.
#if ENABLE_UNITY_COLLECTIONS_CHECKS
        if (allocator <= Allocator.None)
            throw new ArgumentException("Allocator must be Temp, TempJob or Persistent", nameof(allocator));

        if (length < 0)
            throw new ArgumentOutOfRangeException(nameof(length), "Length must be >= 0");

        if (size > int.MaxValue)
            throw new ArgumentOutOfRangeException(nameof(length), $"Length * sizeof(int) cannot exceed {(object)int.MaxValue} bytes");

        // There are other checks you might want to perform when working with generic containers.
        /* 
        if (!UnsafeUtility.IsBlittable<T>())
           throw new ArgumentException(string.Format("{0} used in NativeCustomArray<{0}> must be blittable", typeof(T)));

        if (!UnsafeUtility.IsValidNativeContainerElementType<T>())
            throw new InvalidOperationException($"{typeof(T)} used in NativeCustomArray<{typeof(T)}> must be unmanaged (contain no managed types) and cannot itself be a native container type.");
        */
#endif

        array = default(NativeIntArray);
        // Allocate memory for our buffer.
        array.m_Buffer = UnsafeUtility.Malloc(size, UnsafeUtility.AlignOf<int>(), allocator);
        array.m_Length = length;
        array.m_AllocatorLabel = allocator;

#if ENABLE_UNITY_COLLECTIONS_CHECKS
        // Create a dispose sentinel to track memory leaks. 
        // An atomic safety handle is also created automatically.
        DisposeSentinel.Create(out array.m_Safety, out array.m_DisposeSentinel, 1, allocator);
#endif
    }

    /*
     * ... Next Code ...
     */

3) Read/Write Access

Now we can finally write the functionality to change values in our array. The code for this is pretty straight forward, if not simpler than the original.

    /*
     * ... Previous Code ...
     */

    public int this[int index]
    {
        get
        {
#if ENABLE_UNITY_COLLECTIONS_CHECKS
        	AtomicSafetyHandle.CheckReadAndThrow(m_Safety);
#endif
            return UnsafeUtility.ReadArrayElement<int>(m_Buffer, index);
        }

        [WriteAccessRequired]
        set
        {
#if ENABLE_UNITY_COLLECTIONS_CHECKS
        	AtomicSafetyHandle.CheckWriteAndThrow(m_Safety);
#endif
            UnsafeUtility.WriteArrayElement(m_Buffer, index, value);
        }
    }

    public int Increment(int index) { return ++this[index]; }

    public int Decrement(int index) { return --this[index]; }

    public int Add(int index, int value) { return (this[index] += value); }

    public int Length => m_Length;

    /*
     * ... Next Code ...
     */

4) Disposing

There is one last thing we must not forget before we can use our container, and that is Dispose. Dispose can be called to cleanup an free our memory after we are done with our container.

    /*
     * ... Previous Code ...
     */

    public void Dispose()
    {
#if ENABLE_UNITY_COLLECTIONS_CHECKS
        if (!UnsafeUtility.IsValidAllocator(m_AllocatorLabel))
            throw new InvalidOperationException("The NativeArray can not be Disposed because it was not allocated with a valid allocator.");

        DisposeSentinel.Dispose(ref m_Safety, ref m_DisposeSentinel);
#endif

        // Free the allocated memory and reset our variables.
        UnsafeUtility.Free(m_Buffer, m_AllocatorLabel);
        m_Buffer = null;
        m_Length = 0;
    }
}

Usage

That is it! We now have created a NativeIntArray that can be used with jobs like below.

using Unity.Burst;
using Unity.Collections;
using Unity.Entities;
using Unity.Jobs;
using Unity.Mathematics;

public class NativeIntArraySystem : SystemBase
{
    protected override void OnUpdate()
    {
        NativeIntArray myArray = new NativeIntArray(100, Allocator.TempJob);
        Job.WithName("NativeIntArrayJob").WithCode(() =>
        {
            for (int i = 0; i < myArray.Length; i++)
                myArray.Increment(i);
        }).Run();

        myArray.Dispose();
    }
}

Conclusion

This article showed all the steps involved to create a bare basic native container. But you might have already noticed that a few very useful features are missing. For instance .WithDeallocateOnJobCompletion and [DeallocateOnJobCompletion] will throw an error that our container does not support this incredibly useful feature. We will implement missing features in the next parts of this series.

Untity DOTS Editor

Sun, 23 Feb 2020 00:00:00 +0000

Introduction

Knowing how the Unity editor can support you while developing with DOTS is important to speed up the workflow and get debugging information. In this post I will go over what editor features are available for DOTS.

To follow along with this post you will needed the following packages:

Entities: Installing this package and its dependencies will add everything needed to develop with DOTS, such as the burst compiler and job system.
DOTS Editor: While optional, this package will add extra editor features for DOTS which will be covered here.

Entity Debugger

Can be found under: Window > Analysis > Entity Debugger

The entity debugger gives information about the state of the world. It can show you which entities exist, what components they contain, which systems are running on them and which chunks are used. We will go over each part of the entity debugger.

1. World selection:

Allows you to select the world to show the containing entities and systems of. You can select Show Full Player Loop to show all worlds and Show Inactive Systems to also show systems that are not running.

2. System details:

This section will allow you to view all system groups and the systems they contain. Notice that systems are listed in order of execution.
For each system it gives useful performance information in milliseconds spend on the main thread. You can also disable individual systems to prevent them from running. Selecting a system will show you the entities and components the system runs on, and allows you to view chunk information.

3. Entity inspector:

This shows you all the entities that match the specified system query.
Notice that components are coloured based on read only, read write or subtractive (can not contain), but expect this to be expanded on in the future as currently not all query types have a colour code.
Selecting an entity will show you its component values in the inspector, shown here in the image on the right.

4. Chunk info:

Chunks in DOTS can be very confusing at first, so to properly explain this section I will first give an simplified explanation of chunks. A chunk is a pre-allocated 16kB block of memory, which has an archetype that defines which components are in the chunk. Based on this the chunk calculates how many entities of this archetype it can hold. When entities are added they are put in the chunk until its full at which point a new chunk is created.

The chunk info section shows you how many archetypes match a system query and how many chunks each archetype has. At the bottom it than shows how full these chunks are in a histogram.

In this image we match 1 archetype which has 3 chunks. A chunk of this archetype can hold 104 entities. 2 chunks are currently full (bar on the right) and 1 chunk holds 13 entities (bar on the left).

Live Link Mode

Can be found under: DOTS > Live Link Mode

Live link allows to convert to DOTS while in edit mode. For live link to work you need to add your objects in a subscene: Hierarchy > Right click > New SubScene From Selection. Selecting an object in the subscene will you its ECS components, and the object can now be found in the entity debugger. Keeping the subscene open during play mode will now also allow you to make changes to the entity without restarting.

There are two scene view modes for live link. SceneView: Live Game State makes the scene view show the final converted result in edit mode turning the scene into a hybrid rendered scene. SceneView: Editing State will instead keep the editor renderer and not apply the full conversion. This mode will renderer gizmos, but any changes to the entity that happen at conversion will not be applied.

Burst Inspector

Can be found under: Jobs > Burst > Open Inspector...

The burst inspector is a useful tool for low level performance optimizations. Most people will not be able or need to read the compiler output. However the LLVM Optimization Diagnostics can still be interesting to look at. It gives compiler info in a more human readable manner, which can be used to for instance check if your code is getting vectorized.

Here you can see that the compiler marked line 35 as unable to vectorize.

After a change in the code line 35 is marked as vectorized.

DOTS Compiler

Can be found under: DOTS > DOTS Compiler > Open Inspector...

The DOTS Compiler can show DOTS generated code. This means its able to show you the code generated when using the [GenerateAuthoringComponent] tag. But not only that, it’s also capable of showing the code generated from Entities.ForEach or Job.WithCode code. This can be really useful to see what the you final job actually looks like.

Others

To further control the compilation process you can enable and disable multiple setting having to do with debugging and safety under Jobs. These speak for themselves so I will not be going in depth any further.

To speed up the load time when pressing play in the editor, you can disable scene and domain reload under: Edit > Player Settings > Editor > Enter Play Mode Settings. This is not directly connected to DOTS in any way. But its main downsides largely do not apply to DOTS code, while still giving you the benefits of improved speed. For more information see the documentation.