Lab 6 - Debugging in Assembly

Lab goals:

Pre-lab

Background: Why Debug at the Assembly Level?

Debugging at the assembly level becomes crucial when higher-level debugging tools fall short. Compiler optimizations, undefined behavior, or hardware-level bugs can cause surprising issues that are only evident in the generated machine code. In this lab, you'll walk through real bugs in x86-64 assembly and learn how to trace program behavior at the instruction level; you'll be inspecting compiled C programs and debugging them by analyzing their machine code and stack behavior. You'll also review how arguments are passed using registers, how stack frames are managed, and how the call and return instructions modify the execution flow.

To prepare, revisit these key concepts from lecture:

Be sure to review how the call and ret instructions manipulate the stack, and how arguments are passed using registers like rdi and rsi. You’ll be using this knowledge heavily during the lab.

GitHub Repository for This Lab

To obtain your private repo for this lab, please point your browser to the starter code for the lab at:

https://classroom.github.com/a/YT2BvSdh
Follow the same steps as for previous labs and assignments to to create your repository on GitHub and to then clone it onto CLEAR. The directory for your repository for this lab will be
debugging-in-assembly-name
where name is your GitHub userid.

In-lab

In this lab, you will debug several programs by inspecting their corresponding x86-64 assembly. These programs are intentionally simple at the C level but contain subtle bugs that manifest in the machine code. Your goal is to use gdb and your understanding of assembly to identify and fix these bugs. Each exercise walks you through the investigation process step by step. You will be reading both C and assembly side by side, interpreting control flow, register usage, and assembly layout. This is excellent practice for building a deep understanding of how C maps to machine-level execution.

Exercise 1 — Calculate the Maximum

Open maximum.c and build it using the Makefile. Run the program once to see its behavior, then read the short function carefully and compare what the C source claims to do with what actually happens at runtime. The intent is to return the maximum of two integers, but the control-flow is subtly flipped so the function returns the minimum instead. After observing the printed result, examine the emitted assembly (open the generated .s file and trace the compare-and-branch sequence that decides which value is moved into %eax as the return value. In gdb, use layout asm and single-step through the cmpl and the conditional jump; notice how the path taken contradicts the intended “max” logic and places the smaller argument into %eax. Make a minimal edit to the C conditional so that the control flow matches the mathematical definition of maximum, rebuild, and step again to verify that the correct path is taken and the larger value is returned. 

#include <stdio.h>

/* Simply calculate the maximum
 * of two integers.
 */
int calculate_max(int a, int b)
{
    if (b < a)
        return a;
    else
        return b;
}

int main(void) {
    
    int a, b, max;
    
    a = 10;
    b = 5;
    max = calculate_max(a, b);

    printf("%d\n", max);
    
    return 0;
}

Once you have corrected the conditional, rebuild and rerun to confirm that the program now prints the larger value. As you step the fixed version in the assembly view, map the cmpl operands and the specific conditional jump (jge or similar) to the exact C comparison you wrote, and convince yourself that the updated path reliably selects the true maximum for any pair of inputs.

Exercise 2 — Why is the Output Still Correct?

In this exercise you will examine addition.c. This file computes the sum of integers from zero up to a bound. At first glance it is perfectly correct, and you should build and run it exactly as the Makefile specifies to confirm the expected result. 

#include <stdio.h>

    /* Computes the sum of integers from 0 up to n (inclusive).
     *
     * Example: sum_to_n(5) = 0 + 1 + 2 + 3 + 4 + 5 = 15
     */
    int sum_to_n(int n)
    {
        /* Keep the below line commented for the initial compilation */
        // asm("movl $2, %%edi" ::: "%edi");
        
        int i;
        int total = 0;

        for (i = 0; i <= n; i++) {
            total += i;
        }

        return total;
    }

    int main(void)
    { 
        printf("%d\n", sum_to_n(5));

        return 0;
    }

After running it once, go back into the source and locate the single inline-assembly line that overwrites %edi. This is the register that passes the value of n to the function and it is being overwritten before n is even used. Uncomment that line, rebuild, and run again. You should still see the correct result, which may feel surprising. To understand why, look at the generated assembly output in the corresponding .s file. The compiler treats your inline assembly as clobbering %edi and therefore reloads the live value of the parameter n from its home before using it in the loop bound and updates. In gdb, step instruction by instruction with layout asm and observe how the loop compare instructions use the recovered parameter value, not the register you overwrote.

The lesson here is that clobbering %edi in this way does not have the effect you might expect, because the compiler protects the actual parameter. To deepen your understanding, try performing the same experiment with other registers. What if you overwrite the register that %edi is copied to? You will discover that many of these edits leave the program unaffected. You do not have to submit anything for this exercise.

Exercise 3 — Sensor Readings

Open sensor.c and build it with the Makefile. The file is presented as a tiny sensor logger that should produce five samples and quit, printing a “Starting…” line, five numbered readings, and then “Logging complete.” Look at the assembly output in the corresponding .s file and use gdb to figure out what is the bug in the code. 

#include <stdio.h>

/* The function reads a temperature sensor value
 * based on the location information that is supplied.
 *
 * Expected output:
 * Starting sensor logger...
 *   Sample 1: 1.00
 *   Sample 2: 3.00
 *   Sample 3: 3.00
 *   Sample 4: 5.00
 *   Sample 5: 6.00
 *   Logging complete.
 */
int read_sensor_value(int loc) {
    
    double temp = loc + loc/2 - loc/3;
    return temp;
}

int main(void) {
    int loc = 1;

    printf("Starting sensor logger...\n");

    // Intent: log temperature for five locations
    while (loc < 6) {
        double temp = read_sensor_value(loc);
        printf("Sample %d: %.2f\n", loc, temp);
    loc++;
    }

    printf("Logging complete.\n");
    return 0;
}

After you fix the bug, run the corrected program under the debugger and watch how the induction variable flows through the compare and branch. With the fix in place, you should see the compare succeed for the first five iterations, the loop body execute and print a line each time, and then the jump not taken on the sixth check so that control falls through to the final message.

Exercise 4 — Advanced Telemetry

The final file, telemetry.c, is a larger example that simulates analyzing a fixed dataset by computing rolling averages and producing a checksum. You can build it with the Makefile. At first glance the program runs and prints plausible results, but there are several subtle bugs that you should diagnose by looking at the emitted assembly and carefully tracing the loops. Once fixed, rebuild and rerun to verify consistent, sensible output.

Debugging Tips Summary

Post-lab

Submission Checklist

Before you submit, double-check that you have thoroughly completed all four exercises. You should understand the root cause of each bug at the assembly level and demonstrate that your fix eliminates the error while preserving correct program behavior. Your GitHub repository must include the following:

Submission Instructions

Once you’ve completed the exercises, be sure to git add your modified C files. Then commit and push to your GitHub repository. All submissions are due by 11:55 PM on Sunday, 10/5.