Casting a function pointer to another type

As far as the C standard is concerned, if you cast a function pointer to a function pointer of a different type and then call that, it is undefined behavior. See Annex J.2 (informative):

The behavior is undefined in the following circumstances:

  • A pointer is used to call a function whose type is not compatible with the pointed-to
    type (6.3.2.3).

Section 6.3.2.3, paragraph 8 reads:

A pointer to a function of one type may be converted to a pointer to a function of another
type and back again; the result shall compare equal to the original pointer. If a converted
pointer is used to call a function whose type is not compatible with the pointed-to type,
the behavior is undefined.

So in other words, you can cast a function pointer to a different function pointer type, cast it back again, and call it, and things will work.

The definition of compatible is somewhat complicated. It can be found in section 6.7.5.3, paragraph 15:

For two function types to be compatible, both shall specify compatible return types127.

Moreover, the parameter type lists, if both are present, shall agree in the number of
parameters and in use of the ellipsis terminator; corresponding parameters shall have
compatible types. If one type has a parameter type list and the other type is specified by a
function declarator that is not part of a function definition and that contains an empty
identifier list, the parameter list shall not have an ellipsis terminator and the type of each
parameter shall be compatible with the type that results from the application of the
default argument promotions. If one type has a parameter type list and the other type is
specified by a function definition that contains a (possibly empty) identifier list, both shall
agree in the number of parameters, and the type of each prototype parameter shall be
compatible with the type that results from the application of the default argument
promotions to the type of the corresponding identifier. (In the determination of type
compatibility and of a composite type, each parameter declared with function or array
type is taken as having the adjusted type and each parameter declared with qualified type
is taken as having the unqualified version of its declared type.)

127) If both function types are ‘‘old style’’, parameter types are not compared.

The rules for determining whether two types are compatible are described in section 6.2.7, and I won’t quote them here since they’re rather lengthy, but you can read them on the draft of the C99 standard (PDF).

The relevant rule here is in section 6.7.5.1, paragraph 2:

For two pointer types to be compatible, both shall be identically qualified and both shall be pointers to compatible types.

Hence, since a void* is not compatible with a struct my_struct*, a function pointer of type void (*)(void*) is not compatible with a function pointer of type void (*)(struct my_struct*), so this casting of function pointers is technically undefined behavior.

In practice, though, you can safely get away with casting function pointers in some cases. In the x86 calling convention, arguments are pushed on the stack, and all pointers are the same size (4 bytes in x86 or 8 bytes in x86_64). Calling a function pointer boils down to pushing the arguments on the stack and doing an indirect jump to the function pointer target, and there’s obviously no notion of types at the machine code level.

Things you definitely can’t do:

  • Cast between function pointers of different calling conventions. You will mess up the stack and at best, crash, at worst, succeed silently with a huge gaping security hole. In Windows programming, you often pass function pointers around. Win32 expects all callback functions to use the stdcall calling convention (which the macros CALLBACK, PASCAL, and WINAPI all expand to). If you pass a function pointer that uses the standard C calling convention (cdecl), badness will result.
  • In C++, cast between class member function pointers and regular function pointers. This often trips up C++ newbies. Class member functions have a hidden this parameter, and if you cast a member function to a regular function, there’s no this object to use, and again, much badness will result.

Another bad idea that might sometimes work but is also undefined behavior:

  • Casting between function pointers and regular pointers (e.g. casting a void (*)(void) to a void*). Function pointers aren’t necessarily the same size as regular pointers, since on some architectures they might contain extra contextual information. This will probably work ok on x86, but remember that it’s undefined behavior.

Leave a Comment

tech