Segmentation fault in assembly when multiplying registers?

Question

I was trying to convert the following C code into assembly. Here is the C code:

typedef struct {
    int x;
    int y;
} point;


int square_distance( point * p ) {
    return p->x * p->x + p->y * p->y;  
}

My assembly code is as follows:

square_distance:

.LFB23:
    .cfi_startproc
    movl    (%edi), %edx
    imull   %edx, %edx
    movl    4(%edi), %eax
    imull   %eax, %eax
    addl    %edx, %eax
    ret
    .cfi_endproc

I get a segmentation fault when I try to run this program. Could someone please explain why? Thanks! I would be grateful!

What makes you think `%edi` points to the `p` parameter? Have a look at — Jabberwocky, Apr 09 '20 at 07:04
Isn't the first parameter to the function passed in the edi/rdi register? — , Apr 09 '20 at 07:06
It said so in the question. That "assume" the first parameter to the function is passed in the %edi register. — , Apr 09 '20 at 07:14
BTW you didn't say anything in the question about assuming that the first parameter is in %edi — Jabberwocky, Apr 09 '20 at 07:21
@Jabberwocky: the other possibility is that this is being assembled as 64-bit code and is truncating a 64-bit pointer in RDI to 32-bit by using a 32-bit addressing mode. (Which assembles with an address-size override prefix.) — Peter Cordes, Apr 09 '20 at 09:16
@Jabberwocky: Without the OP explaining how they built this into an executable, we don't know. Although it looks like they copy/pasted compiler output, given the `.LFB23:` which nobody would write by hand, and cfi directives which no beginner would. So maybe they were trying to port 64-bit compiler output to 32-bit instead of just compiling with `-m32`. I didn't feel the need to write yet another guide to x86 calling conventions in general as an answer to cover the bases for this non-[mcve]; Agner Fog has already done that: https://www.agner.org/optimize/calling_conventions.pdf — Peter Cordes, Apr 09 '20 at 09:20

Jabberwocky · Accepted Answer · 2020-04-09T09:15:48.110

Your code is 32 bit code (x86) but you apply the calling convention used with 64 bit code (x64). This can obviously not work.

The x86 calling convention is passing all parameters on the stack.

The x64 calling convention is passing the first parameter in rdi, the second in rsi, the third in rdx, etc. (I'm not sure which registers are used if there are more than 3 parameters, this might also depend on your platform).

Your code is presumably more or less correct for x64 code, that would be something like this:

square_distance:
    movl    (%rdi), %edx
    imull   %edx, %edx
    movl    4(%rdi), %eax
    imull   %eax, %eax
    addl    %edx, %eax
    ret

With x86 code the parameters are passed on the stack and the corresponding correct code would be something like this:

square_distance:
    movl    4(%esp), edx
    movl    (%edx), eax
    imull   eax, eax
    movl    4(%edx), edx
    imull   edx, edx
    addl    edx, eax
    ret

In general the Calling conventions subject is vast, there are other calling conventions depending on the platform and even within the same platform different calling conventions can exist in certain cases.

[Windows vs. everything else](https://stackoverflow.com/questions/4429398/why-does-windows64-use-a-different-calling-convention-from-all-other-oses-on-x86) uses different regs even for the first 3 args, and only x86-64 System V passes args in RDI, RSI, RDX, ... The problem here isn't 32-bit code, just 32-bit address-size, because pointers are 64-bit. — Peter Cordes, Oct 07 '21 at 22:38

score 2 · Answer 2 · answered Apr 09 '20 at 08:37

2

Just want to supplement Jabberwocky answer. Because my reputation is not enough to comment.

The way of passing paraments when calling functions (also known as calling convention) are different from architectures and operating systems(OS). You can find out many common calling conventions from this wiki

From the wiki we can know that The x64 calling convention on *nix is passing the first six parameters through RDI, RSI, RDX, RCX, R8, R9 registers, while others through stack.

answered Apr 09 '20 at 08:37

wxk1997

131
1
6

1

There are other register-arg x86 calling conventions besides x86-64 System V. For example Windows x64, and Windows 32-bit thiscall and fastcall. And on \*nix even gcc 32-bit regparm (`__attribute__((regparm(3))) sqr_dst(point *);`). Yes i386 System V uses the old inefficient stack-args method exclusively, but your answer seems to imply *everything* else is like that. – Peter Cordes Apr 09 '20 at 09:12

Segmentation fault in assembly when multiplying registers?

2 Answers2

Linked

Related