-
Notifications
You must be signed in to change notification settings - Fork 69
Add EEP for partially applied functions #82
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
b1cf16a
b4bd510
ddbb415
87a044a
63c6de4
da43090
3afd091
dba3977
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,320 @@ | ||
| Author: José Valim <jose(dot)valim(at)dashbit(dot)co> | ||
| Status: Active | ||
| Type: Process | ||
| Created: 09-Dec-2025 | ||
| Post-History: | ||
| **** | ||
| EEP XX: Partially applied functions | ||
| ---- | ||
|
|
||
| Abstract | ||
| ======== | ||
|
|
||
| This EEP proposes an alternative to "MFArgs" (Module-Function-Arguments): | ||
| three-element tuples where the first element is a module, the second is | ||
| a function name, and the third is a list of arguments. The proposed | ||
| alternative preserves the desired properties of MFArgs while being more | ||
| ergonomic and with none of MFArgs limitations. | ||
|
|
||
| Rationale | ||
| ========= | ||
|
|
||
| Today, the use of MFArgs are pervasive in Erlang. Generally speaking, | ||
| an API accepts MFArgs either as tuples or as three distinct arguments. | ||
| MFArgs can be invoked as-is but quite often they have additional arguments | ||
| prepended, as shown below: | ||
|
|
||
| ```erlang | ||
| {Mod, Fun, Args} = MFArgs | ||
| apply(Mod, Fun, [SomeValue | Args]). | ||
| ``` | ||
|
|
||
| One of the main reasons MFArgs exist is because anonymous functions | ||
| which close over an existing environment can only be serialized across | ||
| nodes or be persisted to disk if they preserve the same module version. | ||
| Therefore, when dealing with distribution, disk persistence, or hot code | ||
| upgrades, it is essential to use MFArgs instead. Additionally, configuration | ||
| files do not support anonymous functions, and MFArgs are the main option. | ||
|
|
||
| Due to those limitations, many functions in Erlang/OTP and also in | ||
| libraries need to provide two APIs, one that accepts function types | ||
| and another for MFArgs. | ||
|
|
||
| Despite their wide spread use, MFArgs come with several downsides: | ||
|
|
||
| 1. It is unclear which arity of the function will actually be invoked. | ||
| For example, `{some_mod, some_fun, [Arg1, Arg2]}` may have an | ||
| argument prepended when invoked, so what is invoked in practice | ||
| is `fun some_mod:some_fun/3`; | ||
|
|
||
| 2. Due to the above, they don't play well with `xref` or "go to | ||
| definition" used by editors; | ||
|
|
||
| 3. They are hard to evolve. Imagine you define an API that accepts | ||
| `{some_mod, some_fun, [Arg1, Arg2]}` and you prepend one argument. | ||
| In the future, users request for another agument to be prepended. | ||
| Using anonymous functions, you could use `is_function(Fun, Arity)` | ||
| to determine how many arguments are expected. With MFArgs, you can | ||
| use `erlang:function_exported/3`, but it may have false positives | ||
| (as in a higher arity function may exist for other purposes); | ||
|
|
||
| 4. They cause duplication in APIs, as APIs need to accept both `Fun` | ||
| and `MFArgs` as arguments; | ||
|
|
||
| 5. As we attempt to statically type Erlang programs, MFArgs offer | ||
| limited opportunities for static verification, which either | ||
| becomes the source of dynamism (so errors that could be caught | ||
| statically must now be handled at runtime) or leads to false | ||
| positives (requiring developers to rewrite their code); | ||
|
|
||
| Solution | ||
| ======== | ||
|
|
||
| Erlang should provide a construct for partially applied functions. | ||
| Partially applied functions use the `fun Mod:Fun(...Args)` notation, | ||
| where arguments can also be placeholders given by the `_` variable. | ||
|
|
||
| We will break down the syntax in the following section. For now, | ||
| let's see an example: | ||
|
|
||
| 1> Fun = fun maps:get(username, _). | ||
| 2> Fun(#{username => "Joe"}). | ||
| "Joe" | ||
|
|
||
| While the proposed notation does provide syntactical affordances, | ||
| the most important aspect is that the function preserves its remote | ||
| name and arguments within the runtime. This means the partially | ||
| applied function can be passed across nodes or written to disk, | ||
| even if the module that defines the function changes versions. | ||
|
|
||
| Furthermore, partially applied functions can replace `MFArgs`, | ||
| removing all ambiguity about its behaviour. For example, imagine | ||
| the configuration below: | ||
|
|
||
| ```erlang | ||
| {some_config, {some_mod, some_fun, [answer, 42]}}. | ||
| ``` | ||
|
|
||
| If `some_config` is invoked with additional arguments, those | ||
| arguments are not specified in the configuration definition itself, | ||
| therefore it is unclear which arity of `some_mod:some_fun` will | ||
| be invoked. But with partially applied functions, the number of | ||
| arguments is always clear, "go to definition" works, as do config | ||
| files and static typing: | ||
|
|
||
| ```erlang | ||
| {some_config, fun some_mod:some_fun(_, answer, 42)}. | ||
| ``` | ||
|
|
||
| In practice, they solve all the downsides of `MFArgs` listed above: | ||
|
|
||
| 1. The arity is always clear; | ||
|
|
||
| 2. `xref` or "go to definition" can be unambiguously implemented; | ||
|
|
||
| 3. It is possible to handle different arities via | ||
| `is_function(Fun, Arity)` checks; | ||
|
|
||
| 4. There is no longer a need for MFArgs, functions are all you need; | ||
|
|
||
| 5. They can be statically checked; | ||
|
|
||
| Syntax Specification | ||
| ==================== | ||
|
|
||
| The syntax of partially applied functions will be: | ||
|
|
||
| ```erlang | ||
| fun some_fun(...Args) | ||
| fun some_mod:some_fun(...Args) | ||
| ``` | ||
|
|
||
| Where `Args` can be zero, one, or many arguments. Arguments can | ||
| be any expression or the `_` variable. The `_` variable denotes | ||
| placeholders, which are arguments that have not yet been provided | ||
| (the use of `_` is a proposal, the exact notation can be changed, | ||
| see below for additional syntax considerations). | ||
|
|
||
| A partially applied function may have zero or more placeholders. | ||
| The number of placeholders dictate the arity of the function and | ||
| they are provided in order. For example, this is a two arity | ||
| function: | ||
|
|
||
| ```erlang | ||
| %% two arity function | ||
| fun hello(_, world, _) | ||
| ``` | ||
|
|
||
| The placeholder must always appear in the position of an argument, | ||
| it cannot be nested inside a construct. The following is not allowed: | ||
|
|
||
| ```erlang | ||
| fun hello({_, world}, _) | ||
| ``` | ||
|
|
||
| Furthermore, all arguments that are not placeholders are evaluated | ||
| **before** the function. Therefore, the following call: | ||
|
|
||
| ```erlang | ||
| spawn(fun ?MODULE:server_loop(self(), #{})) | ||
| ``` | ||
|
|
||
| is equivalent to: | ||
|
|
||
| ```erlang | ||
| spawn(begin | ||
| Arg1 = self(), | ||
| Arg2 = #{}, | ||
| fun() -> ?MODULE:server_loop(Arg1, Arg2) end | ||
| end). | ||
| ``` | ||
|
|
||
| This is important because the role of this feature extends beyond syntactic | ||
| sugar: it allows Erlang developers to glance at the code and, as long | ||
| as it uses `fun some_mod:some_fun/Arity` or `fun some_mod:some_fun(...Args)`, | ||
| they know they can be persisted. This information could also be used by | ||
| static analyzers and other features to lint code around distribution | ||
| properties. | ||
|
|
||
| Bound variables as functions | ||
| ---------------------------- | ||
|
|
||
| It is also possible to partially apply a module/function pair | ||
| given by bound variables: | ||
|
|
||
| ```erlang | ||
| fun Mod:Fun(username, _) | ||
| ``` | ||
|
|
||
| It is also possible to partially apply another function: | ||
|
|
||
| ```erlang | ||
| GetUsername = fun maps:get(username, _), | ||
| fun GetUsername(SomeMap). | ||
| ``` | ||
|
|
||
| The code above returns a zero-arity function that returns the | ||
| `username` of `SomeMap` when applied. | ||
|
|
||
| > `fun SomeFun/SomeArity` is not valid today, and such syntax | ||
| > will remain invalid as it is orthogonal to this proposal. | ||
|
|
||
| Visual cluttering | ||
| ----------------- | ||
|
|
||
| Given Erlang also supports named functions, the differences | ||
| between named functions, partially applied functions, and regular | ||
| `Function/Arity` may be too small: | ||
|
|
||
| ```erlang | ||
| foo(Y) -> Y-1. | ||
| bar(X) -> | ||
| F1 = fun Foo(X) -> X+1 end, % Arity 1 | ||
| F2 = fun foo(X), % Arity 0 | ||
| F3 = fun foo/1, % Arity 1 | ||
| {F1(X), F2(), F3(X)}. | ||
| ``` | ||
|
|
||
| In case this is deemed a restriction, different options could be | ||
| considered: | ||
|
|
||
| * Require all partially applied functions to have at least one `_`, | ||
| forbidding `fun foo(X)` or `fun some_mod:some_fun(Args)`. This does | ||
| add a syntactical annoyance but it does not remove any capability | ||
| as any function without placeholder can be written as a zero-arity | ||
| function; | ||
|
Comment on lines
+221
to
+225
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't like that one. The restriction would read like "... unless your partial application results in full application", which IMO is a stumbling block that at least I wouldn't want. |
||
|
|
||
| * Only allow remote partially applied functions, so `fun foo(_, ok)` | ||
| is invalid, but `fun some_mod:foo(_, ok)` is accepted. Unfortunately, | ||
| this may lead to developers doing external calls when a local call | ||
| would suffice; | ||
|
Comment on lines
+227
to
+230
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This does not look practical to me. Or useful. Or a good way. Or whatever 🤷♀️ |
||
|
|
||
| * Require partially applied functions to explicitly list the arity too, | ||
| hence `fun foo(X)` has to be written as: `fun foo(X)/0`. | ||
| `fun maps:get(username, _)` as `fun maps:get(username, _)/1`. | ||
| If the version with arity is preferred, then the `fun` prefix could | ||
| also be dropped, if desired, as there is no ambiguity (but it may | ||
| be preserved for clarity) | ||
|
|
||
| Alternative Solutions | ||
| ===================== | ||
|
|
||
| The solution above chose to extend the existing `fun` syntax and use | ||
|
josevalim marked this conversation as resolved.
|
||
| `_` as a placeholder. The exact placeholder syntax can be modified. | ||
|
|
||
| Note this EEP focuses on language changes, rather than runtime changes, | ||
| because support within configuration files is a key goal of this proposal. | ||
| This means an API that worked exclusively at runtime would not tackle all | ||
| of the use cases handled by the existing MFArgs. | ||
|
|
||
| With that in mind, we discuss some alternatives below. | ||
|
|
||
| `{Fun, Args}` Pairs | ||
| ------------------- | ||
|
|
||
| One alternative is to support `{fun some_mod:some_fun/3, [Arg1, Arg2]}`. | ||
| This does improve a few things, as it makes the arity clear and "go to | ||
| definition" also works, but it still requires duplication across APIs, | ||
| as they need to support both regular functions and `{Fun, Args}` pairs. | ||
|
|
||
| `{Fun, Args}` would likely allow us to type check the return type, but | ||
| the argument types could only be partially validated. | ||
|
|
||
| Additional Data Types | ||
| --------------------- | ||
|
|
||
| Additional data types could also be introduced, for example, a | ||
| "serializable function" record which would be internally represented as | ||
| a MFArgs. Its major advantage is that it would not need changes to the | ||
| runtime and those working on type systems could type check these new | ||
| records accordingly. However, they would still force library developers | ||
| to define duplicate APIs that accept both serializable and regular | ||
| functions. | ||
|
|
||
| Of course, we could change `is_function/2`, `apply/2`, and friends to | ||
| support this additional data type but, if we are ultimately changing | ||
| the Erlang runtime, I'd argue it is simpler and more productive to add | ||
| the serialization properties to functions, as done in this proposal, | ||
| than adding a new construct. | ||
|
|
||
| Cuts from erlando | ||
| ----------------- | ||
|
|
||
| The [`erlando`](https://github.com/rabbitmq/erlando) project offered | ||
| the ability to partially apply functions (and also data structures). | ||
|
|
||
| In particular, `erlando` does not require the `fun` prefix, so one can | ||
| write: | ||
|
|
||
| ```erlang | ||
| maps:get(username, _) | ||
| ``` | ||
|
|
||
| The lack of a prefix makes it harder to spot when a function is created | ||
| and also leads to visual ambiguity, such as in the code below: | ||
|
Comment on lines
+293
to
+294
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ... which kind of drives home my point made re dropping or keeping the
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @Maria-12648430 it has been a long discussion, so I believe your comment means you prefer the |
||
|
|
||
| ```erlang | ||
| list_to_binary([1, 2, math:pow(2, _)]) | ||
| ``` | ||
|
|
||
| Their documentation clarifies that it is always shallow (hence it applies | ||
| to `math:pow/2`). | ||
|
|
||
| This proposal also specifies that all arguments must be evaluated before | ||
| the function is captured, so the functions can be persisted across nodes. | ||
| `erlando` does not implement such behaviour. | ||
|
|
||
| Copyright | ||
| ========= | ||
|
|
||
| This document is placed in the public domain or under the CC0-1.0-Universal | ||
| license, whichever is more permissive. | ||
|
|
||
| [EmacsVar]: <> "Local Variables:" | ||
| [EmacsVar]: <> "mode: indented-text" | ||
| [EmacsVar]: <> "indent-tabs-mode: nil" | ||
| [EmacsVar]: <> "sentence-end-double-space: t" | ||
| [EmacsVar]: <> "fill-column: 70" | ||
| [EmacsVar]: <> "coding: utf-8" | ||
| [EmacsVar]: <> "End:" | ||
| [VimVar]: <> " vim: set fileencoding=utf-8 expandtab shiftwidth=4 softtabstop=4: " | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Foostarting with a capital is it a typo? or I'm missing something?Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's the (unused) name of the local function bound to
F1.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. That name is only valid within the fun Foo. It is typically used for recursion from within the fun.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was misguided by foo repetition.
But does it matter in this context if its a named function or anonymous?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I intentionally misguided by showing that the only difference between
fun Foo(X)...andfun foo(X)...is the capital letter, and wherefoo(X)is a fun body (or reference, or call, to a named function), andFoo(X)is a fun header (for an anonymous fun). What tells them apart is what follows, that is an arrow->vs end of expression (not an arrow).The third one;
fun foo/1is also a fun reference (to a named function), and here it is indicated by the arity marker/1that it is a reference.So I suggested that the new fun reference
fun foo(X)(anonymous reference to named function) might look much like the start of an anonymous fun definitionfun Foo(X) -> body end.