From bc0c28d4d384418c73dd41f5934e95193551dd73 Mon Sep 17 00:00:00 2001 From: Folkert de Vries Date: Wed, 1 Jul 2026 17:54:38 +0200 Subject: [PATCH 1/7] add `extern "custom"` RFC --- text/0000-extern-custom.md | 160 +++++++++++++++++++++++++++++++++++++ 1 file changed, 160 insertions(+) create mode 100644 text/0000-extern-custom.md diff --git a/text/0000-extern-custom.md b/text/0000-extern-custom.md new file mode 100644 index 00000000000..93c97f6a652 --- /dev/null +++ b/text/0000-extern-custom.md @@ -0,0 +1,160 @@ +- Feature Name: `abi_custom` +- Start Date: 2026-07-01) +- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000) +- Rust Issue: [rust-lang/rust#140829)](https://github.com/rust-lang/rust/issues/140829) + +# Summary +[summary]: #summary + +An `extern "custom" fn` is a function with a custom ABI that is unknown to rust. Often these are low-level functions that pass arguments in different registers than any standard calling convention. + +```rust +#[unsafe(naked)] +pub unsafe extern "custom" fn __aeabi_uidivmod() { + core::arch::naked_asm!( + "push {{lr}}", + "sub sp, sp, #4", + "mov r2, sp", + "bl {trampoline}", + "ldr r1, [sp]", + "add sp, sp, #4", + "pop {{pc}}", + trampoline = sym crate::arm::__udivmodsi4 + ); +} + +unsafe extern "custom" { + fn __fentry__(); +} +``` + +### History + +* https://github.com/rust-lang/rust/issues/140566 +* https://github.com/rust-lang/rust/issues/140829 +* https://github.com/rust-lang/rust/pull/140770 +* https://github.com/rust-lang/rust/pull/158504 + +# Motivation +[motivation]: #motivation + +In some low-level scenarios we must define naked functions with a custom ABI. This comes up in `rust-lang/compiler-builtins` (e.g. for `__rust_probestack` and `__aeabi_uidivmod`), and also in systems programming (e.g. when defining `__fentry__`, a symbol used the mcount mechanism). + +The current solution is often to use `extern "C"`, but that is misleading: the function does not use the C calling convention, and calling it as if it were is almost certainly causing UB. + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +Because rust doesn't know what calling convention to use, an `extern "custom"` function can only be called via inline assembly or FFI. + +``` +error: functions with the "custom" ABI cannot be called + --> :5:5 + | +5 | bar(); + | ^^^^^ + | +note: an `extern "custom"` function can only be called using inline assembly +``` + +An `extern "custom"` function definition must be a naked function: + +``` +error: items with the "custom" ABI can only be declared externally or defined via naked functions + --> :10:1 + | +10 | unsafe extern "custom" fn bar() { + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + | +help: convert this to an `#[unsafe(naked)]` function + | +10 + #[unsafe(naked)] +11 | unsafe extern "custom" fn bar() { + | +``` + +An `extern "custom"` function definition must be unsafe. The intent here is that a safety comment is written on how this function may be called. + +``` +error: functions with the "custom" ABI must be unsafe + --> :10:1 + | +10 | extern "custom" fn bar() { + | ^^^^^^^^^^^^^^^^^^^^^^^^ + | +help: add the `unsafe` keyword to this definition + | +10 | unsafe extern "custom" fn bar() { + | ++++++ +``` + +In an `extern "custom"` block, functions cannot be marked as `safe`: + +``` +error: foreign functions with the "custom" ABI cannot be safe + --> :16:5 + | +16 | safe fn foobar(); + | ^^^^^^^^^^^^^^^^^ + | +help: remove the `safe` keyword from this definition + | +16 - safe fn foobar(); +16 + fn foobar(); +``` + +An `extern "custom"` function cannot have any arguments or a return type: + +``` +error: invalid signature for `extern "custom"` function + --> :6:31 + | +6 | unsafe extern "custom" fn foo(a: i32) -> i32 { + | ^^^^^^ ^^^ + | + = note: functions with the "custom" ABI cannot have any parameters or return type +help: remove the parameters and return type + | +6 - unsafe extern "custom" fn foo(a: i32) -> i32 { +6 + unsafe extern "custom" fn foo() { + | +``` + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +See https://github.com/rust-lang/reference/pull/2300. + +# Drawbacks +[drawbacks]: #drawbacks + +No specific drawback. + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +https://github.com/rust-lang/rust/issues/140566 + +The name has already been debated. The name `unknown` has been mentioned, but to write the implementation you really do need to know the ABI. It is custom in the sense that rustc does not know about it, but the author definitely does. + +# Prior art +[prior-art]: #prior-art + +https://github.com/rust-lang/rust/issues/140566#issuecomment-2846205457 + +We do have some other calling conventions that cannot be called using rust's function calling syntax: + +- extern "*-interrupt` +- extern "gpu-kernel" + +Those however do have a known ABI, it's just that semantically it does not make sense to call them from a rust program. + +# Unresolved questions +[unresolved-questions]: #unresolved-questions + +None currently. + +# Future possibilities +[future-possibilities]: #future-possibilities + +None currently. From b38e056fec949f6ec105342aea74fa80bfa39d96 Mon Sep 17 00:00:00 2001 From: Folkert de Vries Date: Wed, 1 Jul 2026 19:52:32 +0200 Subject: [PATCH 2/7] changes after review --- text/0000-extern-custom.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/text/0000-extern-custom.md b/text/0000-extern-custom.md index 93c97f6a652..af1e05fbbe9 100644 --- a/text/0000-extern-custom.md +++ b/text/0000-extern-custom.md @@ -6,9 +6,14 @@ # Summary [summary]: #summary -An `extern "custom" fn` is a function with a custom ABI that is unknown to rust. Often these are low-level functions that pass arguments in different registers than any standard calling convention. +An `extern "custom" fn` is a function with a custom ABI that is unknown to rust. Often these are low-level functions that pass arguments in different registers than any standard calling convention, so using this helps rustc block you from using this in any place where rustc would need to understand that calling convention. + ```rust +/// # SAFETY +/// +/// - Expects the dividend and the divisor in r0 and r1. +/// - Returns the quotient in r0 and the remainder in r1. #[unsafe(naked)] pub unsafe extern "custom" fn __aeabi_uidivmod() { core::arch::naked_asm!( From 231b22fa2514dbd8c70fbd2a01b12d2a74d7ccdf Mon Sep 17 00:00:00 2001 From: Folkert de Vries Date: Thu, 2 Jul 2026 21:22:05 +0200 Subject: [PATCH 3/7] Apply suggestions from code review Co-authored-by: Trevor Gross --- text/0000-extern-custom.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/text/0000-extern-custom.md b/text/0000-extern-custom.md index af1e05fbbe9..1aa37ca8458 100644 --- a/text/0000-extern-custom.md +++ b/text/0000-extern-custom.md @@ -35,10 +35,10 @@ unsafe extern "custom" { ### History -* https://github.com/rust-lang/rust/issues/140566 -* https://github.com/rust-lang/rust/issues/140829 -* https://github.com/rust-lang/rust/pull/140770 -* https://github.com/rust-lang/rust/pull/158504 +* Proposal: https://github.com/rust-lang/rust/issues/140566 +* Tracking issue: https://github.com/rust-lang/rust/issues/140829 +* Implementation: https://github.com/rust-lang/rust/pull/140770 +* Stabilization PR: https://github.com/rust-lang/rust/pull/158504 # Motivation [motivation]: #motivation @@ -142,6 +142,7 @@ https://github.com/rust-lang/rust/issues/140566 The name has already been debated. The name `unknown` has been mentioned, but to write the implementation you really do need to know the ABI. It is custom in the sense that rustc does not know about it, but the author definitely does. +Arguments and return types are forbidden because they are not consumed in any way. The compiler has no way of validating them against the function body, similar to other naked functions, but the functions can never be invoked directly so they serve no purpose when calling. The intent is that correct parameter passing and returning will be covered in documentation. # Prior art [prior-art]: #prior-art From eba79be9b44d0ec092561848749ea75fdc4d78bf Mon Sep 17 00:00:00 2001 From: Eric Huss Date: Wed, 1 Jul 2026 16:00:52 -0700 Subject: [PATCH 4/7] Fix abi_custom header levels This updates section headers to use Markdown 2nd level headings. As part of https://github.com/rust-lang/rfcs/pull/3883 we switched the template to not use level-1 headings --- text/0000-extern-custom.md | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/text/0000-extern-custom.md b/text/0000-extern-custom.md index 1aa37ca8458..7679231afed 100644 --- a/text/0000-extern-custom.md +++ b/text/0000-extern-custom.md @@ -3,7 +3,7 @@ - RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000) - Rust Issue: [rust-lang/rust#140829)](https://github.com/rust-lang/rust/issues/140829) -# Summary +## Summary [summary]: #summary An `extern "custom" fn` is a function with a custom ABI that is unknown to rust. Often these are low-level functions that pass arguments in different registers than any standard calling convention, so using this helps rustc block you from using this in any place where rustc would need to understand that calling convention. @@ -40,14 +40,14 @@ unsafe extern "custom" { * Implementation: https://github.com/rust-lang/rust/pull/140770 * Stabilization PR: https://github.com/rust-lang/rust/pull/158504 -# Motivation +## Motivation [motivation]: #motivation In some low-level scenarios we must define naked functions with a custom ABI. This comes up in `rust-lang/compiler-builtins` (e.g. for `__rust_probestack` and `__aeabi_uidivmod`), and also in systems programming (e.g. when defining `__fentry__`, a symbol used the mcount mechanism). The current solution is often to use `extern "C"`, but that is misleading: the function does not use the C calling convention, and calling it as if it were is almost certainly causing UB. -# Guide-level explanation +## Guide-level explanation [guide-level-explanation]: #guide-level-explanation Because rust doesn't know what calling convention to use, an `extern "custom"` function can only be called via inline assembly or FFI. @@ -125,17 +125,17 @@ help: remove the parameters and return type | ``` -# Reference-level explanation +## Reference-level explanation [reference-level-explanation]: #reference-level-explanation See https://github.com/rust-lang/reference/pull/2300. -# Drawbacks +## Drawbacks [drawbacks]: #drawbacks No specific drawback. -# Rationale and alternatives +## Rationale and alternatives [rationale-and-alternatives]: #rationale-and-alternatives https://github.com/rust-lang/rust/issues/140566 @@ -143,7 +143,9 @@ https://github.com/rust-lang/rust/issues/140566 The name has already been debated. The name `unknown` has been mentioned, but to write the implementation you really do need to know the ABI. It is custom in the sense that rustc does not know about it, but the author definitely does. Arguments and return types are forbidden because they are not consumed in any way. The compiler has no way of validating them against the function body, similar to other naked functions, but the functions can never be invoked directly so they serve no purpose when calling. The intent is that correct parameter passing and returning will be covered in documentation. -# Prior art + +## Prior art +======= [prior-art]: #prior-art https://github.com/rust-lang/rust/issues/140566#issuecomment-2846205457 @@ -155,12 +157,12 @@ We do have some other calling conventions that cannot be called using rust's fun Those however do have a known ABI, it's just that semantically it does not make sense to call them from a rust program. -# Unresolved questions +## Unresolved questions [unresolved-questions]: #unresolved-questions None currently. -# Future possibilities +## Future possibilities [future-possibilities]: #future-possibilities None currently. From 5573cea528adcbabb8129b6504d47b1bce14e582 Mon Sep 17 00:00:00 2001 From: Eric Huss Date: Wed, 1 Jul 2026 16:01:39 -0700 Subject: [PATCH 5/7] Rename 3980 extern-custom --- text/{0000-extern-custom.md => 3980-extern-custom.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename text/{0000-extern-custom.md => 3980-extern-custom.md} (100%) diff --git a/text/0000-extern-custom.md b/text/3980-extern-custom.md similarity index 100% rename from text/0000-extern-custom.md rename to text/3980-extern-custom.md From 206ea924ca783f1b270945cd2924555e03caf608 Mon Sep 17 00:00:00 2001 From: Eric Huss Date: Wed, 1 Jul 2026 16:02:24 -0700 Subject: [PATCH 6/7] Set 3980 RFC PR link --- text/3980-extern-custom.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/3980-extern-custom.md b/text/3980-extern-custom.md index 7679231afed..6bcaa066dfc 100644 --- a/text/3980-extern-custom.md +++ b/text/3980-extern-custom.md @@ -1,6 +1,6 @@ - Feature Name: `abi_custom` - Start Date: 2026-07-01) -- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000) +- RFC PR: [rust-lang/rfcs#3980](https://github.com/rust-lang/rfcs/pull/3980) - Rust Issue: [rust-lang/rust#140829)](https://github.com/rust-lang/rust/issues/140829) ## Summary From 6dbabd0bf9396b0c324e5f5814f68ddb6162ee51 Mon Sep 17 00:00:00 2001 From: Folkert de Vries Date: Thu, 23 Jul 2026 22:34:08 +0200 Subject: [PATCH 7/7] changes after T-lang meeting --- text/3980-extern-custom.md | 55 +++++++++++++++++++++++++++++++++----- 1 file changed, 49 insertions(+), 6 deletions(-) diff --git a/text/3980-extern-custom.md b/text/3980-extern-custom.md index 6bcaa066dfc..30e061366a1 100644 --- a/text/3980-extern-custom.md +++ b/text/3980-extern-custom.md @@ -1,4 +1,4 @@ -- Feature Name: `abi_custom` +- Feature Name: `abi_custom` - Start Date: 2026-07-01) - RFC PR: [rust-lang/rfcs#3980](https://github.com/rust-lang/rfcs/pull/3980) - Rust Issue: [rust-lang/rust#140829)](https://github.com/rust-lang/rust/issues/140829) @@ -6,7 +6,7 @@ ## Summary [summary]: #summary -An `extern "custom" fn` is a function with a custom ABI that is unknown to rust. Often these are low-level functions that pass arguments in different registers than any standard calling convention, so using this helps rustc block you from using this in any place where rustc would need to understand that calling convention. +An `extern "custom" fn` is a function with a custom ABI that is unknown to rust. Often these are low-level functions that pass arguments in different registers than any standard calling convention, so using `extern "custom"` helps rustc block you from using a function in a place where rustc would need to understand that calling convention. ```rust @@ -38,6 +38,7 @@ unsafe extern "custom" { * Proposal: https://github.com/rust-lang/rust/issues/140566 * Tracking issue: https://github.com/rust-lang/rust/issues/140829 * Implementation: https://github.com/rust-lang/rust/pull/140770 +* Restrictions on function pointer types: https://github.com/rust-lang/rust/pull/159780 * Stabilization PR: https://github.com/rust-lang/rust/pull/158504 ## Motivation @@ -50,7 +51,7 @@ The current solution is often to use `extern "C"`, but that is misleading: the f ## Guide-level explanation [guide-level-explanation]: #guide-level-explanation -Because rust doesn't know what calling convention to use, an `extern "custom"` function can only be called via inline assembly or FFI. +Because rust doesn't know what calling convention to use, an `extern "custom"` function can only be called via inline assembly or FFI. ``` error: functions with the "custom" ABI cannot be called @@ -78,7 +79,7 @@ help: convert this to an `#[unsafe(naked)]` function | ``` -An `extern "custom"` function definition must be unsafe. The intent here is that a safety comment is written on how this function may be called. +An `extern "custom"` function definition must be unsafe. The intent here is that a safety comment is written on how this function may be called. ``` error: functions with the "custom" ABI must be unsafe @@ -125,6 +126,23 @@ help: remove the parameters and return type | ``` +We also error on missing `unsafe` or presence of argument and return types in `extern` blocks and function pointer types. + +``` +error: invalid signature for `extern "custom"` function + --> $DIR/bad-custom.rs:13:40 + | +LL | type MustbeUnsafe = extern "custom" fn(i64) -> i64; + | ^^^ ^^^ + | + = note: functions with the "custom" ABI cannot have any parameters or return type +help: remove the parameters and return type + | +LL - type MustbeUnsafe = extern "custom" fn(i64) -> i64; +LL + type MustbeUnsafe = extern "custom" fn(); + | +``` + ## Reference-level explanation [reference-level-explanation]: #reference-level-explanation @@ -145,7 +163,6 @@ The name has already been debated. The name `unknown` has been mentioned, but to Arguments and return types are forbidden because they are not consumed in any way. The compiler has no way of validating them against the function body, similar to other naked functions, but the functions can never be invoked directly so they serve no purpose when calling. The intent is that correct parameter passing and returning will be covered in documentation. ## Prior art -======= [prior-art]: #prior-art https://github.com/rust-lang/rust/issues/140566#issuecomment-2846205457 @@ -165,4 +182,30 @@ None currently. ## Future possibilities [future-possibilities]: #future-possibilities -None currently. +### Support `Never` as a return type + +A return type of `-> !` indicates that the function diverges. + +That a function diverges is useful to know, and relevant for the very low-level use cases for `extern "custom"`. However, like +arbirary argument and return types (see below), the type is never actually validated, and the signature might go out of sync with +the implementation. + +So far, no actual use of `-> !` has come up, so we leave it as a future possibility. + +### Support argument and return types + +Argument and return types are not validated by the compiler whatsoever + +- like any other `#[naked]` function, argument/return types are not validated against the body +- argument/return types are not validated at the call site (because inline assembly is used) + +For those reasons, argument and return types are omitted in the current RFC. + +The only place where the argument/return types would be checked is when an `extern "custom"` function pointer +is unified again another function pointer type. A newtype wrapper can be used to differentiate different +`extern "custom"` function pointers that carry a particular semantic meaning. + +However, the point has been raised that the argument and return types can serve as documentation. On the other hand, the +signature can be misleading: the whole point of `extern "custom"` is that no known ABI is followed. + +In summary, we leave space for argument and return types to be supported in the future when a compelling use case presents itself.