-
Notifications
You must be signed in to change notification settings - Fork 4
Draft: Revamped API #111
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
SolarLiner
wants to merge
16
commits into
main
Choose a base branch
from
refactoring/design-v2
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Draft: Revamped API #111
Changes from 11 commits
Commits
Show all changes
16 commits
Select commit
Hold shift + click to select a range
6423735
wip: remove input/output specific traits and make code compile again
SolarLiner 51a94d4
fix: examples and CoreAudio backend
SolarLiner 6cd037e
fix: compile errors in alsa backend
SolarLiner 836bf79
fix: alsa and pipewire backends work
SolarLiner 2c17962
wip: update ALSA and Pipewire backends
SolarLiner e5cc1e2
feat(coreaudio): do input/output detection for devices
SolarLiner 6e98f93
fix(examples): fix compilation errors
SolarLiner db5441a
refactor: remove `SendEverywhereButOnWeb` as it was useless (web will…
SolarLiner d1eeea2
feat: `StreamConfig` methods that use `DeviceType`
SolarLiner fb59144
fix: add back `buffer_size_range` to `AudioDevice` after rebase
jacksongoode 71fdec8
test: fix compilation errors
SolarLiner 164d19b
wip(wasapi): stream format negociation
SolarLiner 75d65b8
wip: move to workspace structure
SolarLiner 582825f
wip: wasapi backend
SolarLiner 5dfaf3c
fix(wasapi): forgot unversioned files
SolarLiner 7522d21
Merge branch 'main' into refactoring/design-v2
SolarLiner File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| { | ||
| "rust-analyzer.cargo.features": [ | ||
| "pipewire" | ||
| ] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Merely specifying the number of channels cannot adequately take advantage of all the features of some APIs. Pipewire and JACK both give names to ports. This API would require all the ports to share a common name, differentiated only by number like
client:port_1,client:port_2,client:port_3. PortAudio does this and it has always frustrated JACK users who expect meaningful names for ports so they can route them as they please in a separate patchbay application.Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also along these lines, putting all channels for input/output into a single buffer would be cumbersome for developers of applications that expose lots of named ports. With the JACK API, the application creates structs for each port which get accessed in the callback. See https://github.com/RustAudio/rust-jack/blob/main/examples/playback_capture.rs. I recommend to create an API along these lines as I find the JACK API quite intuitive to use (at least with the Rust bindings). For backends that do not support naming ports, creating a port struct would simply increment the number of channels and the name string would be ignored.
As an example of what a pain it would be for application developers to not be able to access ports by name, here's qpwgraph with Ardour open with a very simple project that only has a single track:

There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As a JACK and Pipewire user, I fully understand the pain and I have been thinking of ways to let users specify port names. The problem being that
interflow(like PortAudio) is an abstraction library, so it cannot fully map all platform-specific features. But I don't wantinterflowto be a "lowest common denominator" library either, and the whole design with "drivers as structs" allows people to specifically target one, get concrete types, allowing specific configuration where it can be found, while also remaining to stay generic with "good enough" defaults.I could also make it available as part of the stream configuration type, but that would significantly increase the configuration complexity for an option that I would argue, most people wouldn't use in the first place (either out of ignorance or because they don't even develop against backends that support port naming).
The other problem is that if you don't know that you can name your ports, you aren't going to do it. A lot of cross-platform software will not bother doing it anyway even with the feature available; it's not only a library but also a user problem.
Finally, JACK and Pipewire are fundamentally different than other platforms because the user is creating the node and its topology, vs. WASAPI or CoreAudio where the platform dictates what you can connect to. It's hard to blend the two together in the first place, and so the latter use case wins by virtue of being the most often used one.
All in all, the current solution in my head is to keep the generic path light on configuration, and instead allowing people who want to deal with backend-specific configuration to do so directly. GUI applications (as an example) will also require specific UI design to reflect specific configuration details, so I'm not even sure it makes sense to abstract much of that away in the first place, except for providing an "easy mode".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
re. the use of named ports, the order of declaration would be kept in the order of audio buffers in the audio callback, do you could set up an enum to do the naming, and case to
usizewhen indexing the audio buffers; I am going to explicitely disallow relying on string types to index ports because it's error prone and has very poor type safety.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That could work.
Another big difference between Pipewire and JACK versus APIs that directly connect to hardware is that JACK ports can exist independently of whether they are in active use. It's common for JACK applications to create various ports without programmatically connecting them to any other nodes and leaving that to the user or a session manager application. So as an application developer, I might want to create 4 input and 20 output ports, but by default only connect 2 of those output ports to a hardware output. Not sure how to account for this in a cross platform API, but I hope such considerations can be accounted for early in the API design before they get excluded.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's hard to make such specific situations part of the generic use-case where the assumption is that the audio callback that is opened is connected to a device, rather than having an additional nodal abstractions. Again, it's not just an issue of technology but also of expected usage ; most people not being on Linux and specifically not knowing about the inner workings of JACK/Pipewire means they will expect the traditional workflow. I don't want to make this use case weaker in
interflowfor the sole reason that it might make it easier to target JACK while staying in that workflow.In all cases, if you want to use
interflowfor the library design but still directly control the JACK node interface, you'll be able to do that eventually by using the JACK driver explicitely.Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interesting idea; we'll see if such an approach can really expose all the features of each API. What I want to avoid is situations like the discussion on that old Mixxx bug where developers are caught between having to choose an oversimplified cross platform abstraction that doesn't really meet users' needs or forgoing the cross platform abstraction and using a platform-specific API directly. I would lean towards a graceful degradation approach of designing the abstractions around the most featureful backends with sensible fallbacks for backends that don't support all features; it seems you're leaning towards a progressive enhancement approach instead.
Indeed, that's why I'd argue for graceful degradation over progressive enhancement. Read through that old discussion on the Mixxx bug and look how hard it was for users to communicate to unfamiliar developers what the behavior should be, or that there even was a problem to be solved. That's why I'd favor designing the API around the most featureful backends, so that developers who are not so familiar with them don't neglect them, for example by failing to provide port names.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have begun the work on an extension-based API, allowing user-defined traits and objects to be queried at runtime, and in a dyn-safe way. For example, here are additional traits for "named channels" and config enumeration, trait implementations can be registered and user code can query those traits from the context of a type-erased (i.e., through
dyn DeviceProxy) device.This would allow the JACK and PipeWire backends to implement custom "dynamic port" and "port name" extensions, and user code that would like to use those, would be able to query for the extension at runtime.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh interesting idea. The link to the diff in your comment isn't working though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It does for me, but in any case you can look at
lib.rsininterflow-corein that branch, the additional traits are extension points. There is also an example of platform-specific extensions in theinterflow-wasapicrate, thedevice.rsimplements and registers aDefaultForRoleextension that allows specifying a WASAPI Role and get the associated default device ID.