This is a simple example on writing, compiling, and debugging Solaris 64-bit x86 assembly language with a C program.
This is also referred to as "AMD64" assembly. The term "AMD64" is used in an inclusive sense to refer to all X86 64-bit processors, whether AMD Opteron family or Intel 64 processor family. Both run Solaris x86.
I'm keeping this example simple mainly to illustrate how everything comes together—compiler, assembler, linker, and debugger when using assembly language.
The example I'm using here is a C program that calls an assembly language program passing a C string. The assembly language program takes the C string and calls printf() with it to print the string.
AMD64 Register Usage
But first let's review the use of AMD64 registers.
AMD64 has several 64-bit registers, some special purpose (such as the stack pointer) and others general purpose. By convention, Solaris follows the AMD64 ABI in register usage, which is the same used by Linux, but different from Microsoft Windows in usage (such as which registers are used to pass parameters). This blog will only discuss conventions for Linux and Solaris.
The following chart shows how AMD64 registers are used.
The first six parameters to a function are passed through registers.
If there's more than six parameters, parameter 7 and above are pushed on the stack before calling the function. The stack is also used to save temporary "stack" variables for use by a function.
Instruction Pointer points to the current instruction
Frame Pointer (saved stack pointer pointing to parameters on stack)
Function Parameter 1
Function Parameter 2
Function Parameter 3
Function Parameter 4
Function Parameter 5
Function Parameter 6
Function return value
Temporary registers (need not be saved before used)
%rbx, %r12, %r13, %r14, %r15
Temporary registers, but must be saved before use and
restored before returning from the current function
(usually with the push and pop instructions).
32-, 16-, and 8-bit registers
To access the lower 32-, 16-, or 8-bits of a 64-bit register use the following:
Least significant 32-bits
Least significant 16-bits
Least significant 8-bits
There's other registers present, such as the 64-bit %mm registers, 128-bit %xmm registers, 256-bit %ymm registers, and 512-bit %zmm registers. Except for %mm registers, these registers may not present on older AMD64 processors.
The following is the source for a C program, helloas1.c, that calls an assembly function, hello_asm().
$ cat helloas1.c
The assembly function called above, hello_asm(), is defined below.
$ cat helloas2.s
In the assembly source above, the C skeleton code under "#if defined(lint)" is optionally used for lint to check the interfaces with your C program--very useful to catch nasty interface bugs. The "asm_linkage.h" file includes some handy macros useful for assembly, such as ENTRY_NP(), used to define a program entry point, and SET_SIZE(), used to set the function size in the symbol table.
The function hello_asm calls C function printf() by passing two parameters, Parameter 1 (P1) is a printf format string, and P2 is a string variable. The function begins by moving %rdi, which contains Parameter 1 (P1) passed hello_asm, to printf()'s P2, %rsi. Then it sets printf's P1, the format string, by loading the address of the format string in %rdi, P1. Finally it calls printf.
After returning from printf, the hello_asm function returns itself.
Instructions starting with dot "." are pseudo instructions. Pseudo instructions don't generate code but control things such as alignment (.align), externals (.extern). string data (.ascii), and byte data (.byte). Labels are external symbols unless they begin with a dot "." (such as ".printf_string" in this example).
Larger, more complex assembly functions usually do more setup than the example above. If a function is returning a value, it would set %rax to the return value. Also, it's typical for a function to save the %rbp and %rsp registers of the calling function and to restore these registers before returning. %rsp contains the stack pointer and %rbp contains the frame pointer. Here is the typical function setup and return sequence for a function:
Use the Solaris cc command to compile both C and assembly source, and to pre-process assembly source. You can also use GNU gcc instead of cc to compile, if you prefer. The "-m64" option tells the compiler to compile in 64-bit address mode (instead of 32-bit).
$ cc -m64 -o helloas2-cpp.s -D_ASM -E helloas2.s
MDB is the Solaris system debugger. It can also be used to debug user programs, including assembly and C. The following example runs the above program, hello-asm, under control of the debugger.
In the example below I load the program, set a breakpoint at the assembly function hello_asm, display the registers and the first parameter, step through the assembly function, and continue execution.
$ mdb hello-asm# Start the debugger
In the example above, at the start of function hello_asm(), I display the stack contents with "$C", display the registers contents with "$r", then disassemble the current function with "::dis".
The first function parameter, which is a C string, is passed by reference with the string address in %rdi (see the register usage chart above).
The address is 0x400cf8, so I print the value of the string with the "/S" MDB command: "0x0000000000400cf8/S".
I can also print the contents at an address in several other formats.
Here's a few popular formats. For more, see the mdb(1) man page for details.
Finally, I step through each machine instruction with the "[" command, which steps over functions. If I wanted to enter a function, I would use the "]" command. Then I continue program execution with ":c", which continues until the program terminates.
Here's a brief cheat sheet of some of the more common MDB commands useful for assembly debugging.
There's an entire set of macros and more powerful commands, especially some for debugging the Solaris kernel, but that's beyond the scope of this example.
X86 assembly language comes in two formats—one used by Intel and Microsoft DOS and Windows, and the other used by ATT UNIX and UNIX-like systems (including Solaris and Linux). Here's a chart illustrating the differences:
|Intel/Microsoft syntax||ATT/UNIX/Solaris/Linux syntax|
As you can see the main difference is the operands are reverse. Intel uses "destination, source" and ATT uses "source, destination". Other differences is ATT prefixes literals with "$" and registers with "%". Intel uses a "h" suffix to designate hexadecimal and ATT uses "0x" prefix. Intel uses "[register+offset]", addressing and ATT uses "offset(register)". A simple way to help translate from Intel to ATT syntax is to assemble the source into an object file and use Solaris dis(1) to disassemble the object.