Add real-time barcode scanning API and iOS backend implementation by phildini · Pull Request #4458 · beeware/toga

phildini · 2026-06-13T05:07:13Z

Add Camera.start_scanning(), stop_scanning(), is_scanning() and on_detection callback to toga-core, with BarcodeFormat enum supporting 7 barcode types. Full implementation in the Dummy backend for testing. iOS backend uses AVCaptureSession with AVCaptureMetadataOutput for native barcode detection. Cocoa and Android backends raise NotImplementedError stubs.

Includes 14 new core tests (100% coverage), updated documentation, and towncrier change fragment.

Add the API surface area and the iOS backend for allowing real-time barcode processing off a camera stream. The iOS libraries support this natively, and toga already has a camera widget, this is exposing more features in toga. I believe Android and macOS also expose this, but need to do more research.

So, there's a lot of functionality that gets unlocked by exposing these APIs, but my immediate need is being able to do token auth / token exchange for a BeeWare app by having the hosting site generate a QR code that contains a token the app can use as part of an auth flow.

Refs #3675.

PR Checklist:

I will abide by the BeeWare Code of Conduct
I have read and have followed the CONTRIBUTING.md file
This PR was generated or assisted using an AI tool

Assisted-by: OpenCode with DeepSeek 4 Flash

Add Camera.start_scanning(), stop_scanning(), is_scanning() and on_detection callback to toga-core, with BarcodeFormat enum supporting 7 barcode types. Full implementation in the Dummy backend for testing. iOS backend uses AVCaptureSession with AVCaptureMetadataOutput for native barcode detection. Cocoa and Android backends raise NotImplementedError stubs. Includes 14 new core tests (100% coverage), updated documentation, and towncrier change fragment.

phildini · 2026-06-13T05:34:39Z

There's some indication that the API wanted for this on Android is https://developers.google.com/ml-kit/vision/barcode-scanning/code-scanner

Add test_barcode_format_str and test_barcode_format_values to exercise all BarcodeFormat members and their __str__ methods.

Replace static code type tests with parametrized test exercising each BarcodeFormat individually through the full scan API, plus a combined test for all types. The str() and member tests are now integrated into a single enumeration check.

phildini · 2026-06-14T02:45:08Z

I really think the test errors here are transient and not related to this PR, but please correct me if I'm wrong

freakboy3742 · 2026-06-14T06:13:30Z

Argh... I think the iOS failure is due to beeware/briefcase-iOS-Xcode-template#75 being merged (part of the whole macOS-26 logging issue).

freakboy3742 · 2026-06-14T08:48:14Z

Ok - I've resolved the issue that was causing problems; the test suite is now passing, but has some coverage gaps.

freakboy3742

This is exciting stuff! Thanks for the PR.

A few comments inline; most of which are fairly minor. The biggest source of concern is the removal of explanatory comments; it's not clear why these have been stripped. There's also the test failure due to some coverage misses.

I haven't run this actual code yet - for the purposes of testing, it would be helpful if there the example/hardware app had an example of barcode scanning - maybe a new tab that scans barcodes on demand and presents a multiline text widget that is populated with found barcodes, cleared each time a session is started?

I have two other architectural questions that don't need to be implemented as part of this PR, but I'd like to have at least an indicative answer to make sure we're not boxing ourselves into a corner:

Is this API compatible with Android's scanner? I can't think of any reason it wouldn't be, but I'd prefer to have a light confirmation before we lock this API in.
This API displays a view controller to display a camera view, and that definitely makes sense as a simple API for getting a barcode. However, I can also imagine that a user might want to have a custom view where they are in control of the visualisation of the camera.

I'm thinking in particular of the "multi-scan" case - I might want a GUI where the top half of the screen is a camera preview, and the bottom half is the list of most recently scanned codes. In that context, I can imagine we might need something like a "CameraPreview" widget, with the ability to connect that widget to scanning processes.

I imagine the setup for that would be to pass a CameraPreview widget to start_scanning(); the UI constructed by the hardware service is then "the UI that is displayed if you don't give me an explcit camera preview". That would presumably also have analog with take_photo() so that users could have their own custom inline camera preview for taking photos. If you've got any other suggestions on how this might be structured, or why this might not be possible, I'd be interested in hearing your thoughts.

freakboy3742 · 2026-06-15T06:40:07Z

+    def start_scanning(
+        self,
+        device: CameraDevice | None = None,
+        code_types: list[BarcodeFormat] | None = None,


As a convenience, should we also accept BarcodeFormat here? I would have expected the most common use case is "scan for QR Code", rather than "Scan for QR code or aztec code or Code128 barcode or ...".

Would you prefer that over code_types?

I wasn't suggesting changing the argument name - I was only suggesting adding the typing and argument handling shim so that start_scanning(QR) is a legal usage, equivalent to start_scanning([QR]).

start_scanning(code_types=QR) is a little weird because of inconsistent pluralization... but if code_types is the first argument, then you won't need to ever write code_types, and maybe that isn't as obvious as a weirdness?

freakboy3742 · 2026-06-15T07:36:38Z

+        capture_metadata_output = _scan_symbols()["capture_metadata_output"]
+        for output in session.outputs():
+            if output.isKindOfClass_(capture_metadata_output):
+                output.setMetadataObjectsDelegate_queue_(self._scan_delegate, None)


Rubicon provides a cleaner syntax for multi-argument ObjC methods:

Suggested change

output.setMetadataObjectsDelegate_queue_(self._scan_delegate, None)

output.setMetadataObjectsDelegate(self._scan_delegate, queue=None)

Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>

As a convenience for the common use case of scanning for a single barcode type (e.g. QR), start_scanning() now accepts a bare BarcodeFormat value in addition to a list. A single value is normalized to a one-element list before being passed to the backend.

Replace NotImplementedError stubs on macOS with real AVCaptureMetadataOutput implementation via TogaCameraScannerDelegate and TogaCameraScannerWindow. Add AVCaptureMetadataOutput, AVMetadataMachineReadableCodeObject, and 7 AVMetadataObjectType* constants to cocoa AVFoundation bindings. Restore # for classes that need to be monkeypatched for testing comment and commented-out native_video_quality() function in iOS backend. The code_types parameter now accepts a single BarcodeFormat value as a convenience shorthand. Update docs and changes fragment to reflect macOS support and single-value acceptance.

Follow the standard codebase pattern by declaring ObjCClass and objc_const symbols directly in libs/av_foundation.py instead of using a lazy dict lookup via _scan_symbols(). The BARCODE_FORMAT_MAP is now a static module-level dict using the directly imported AVMetadataObjectType constants.

Restore NSCameraUsageDescription, request_permission thread, take_photo configuration, delegate, and presentation comments that were lost during the rewrite of the iOS camera backend.

mhsmith · 2026-06-17T20:48:40Z

Is this API compatible with Android's scanner? I can't think of any reason it wouldn't be, but I'd prefer to have a light confirmation before we lock this API in.

My knowledge of this is way out of date, and there may be a better option now, but last time I worked on an Android QR code scanner, the best option I found was the external zxing library:

This would require the app to specifically add a dependency in order to use the feature, as some of Toga's other features already do.

freakboy3742

Nice work getting the Cocoa implementation working. A few review comments inline.

There's a lot of coverage gaps being reported by CI; there's also a lot of code marked as ignored. Some of that will be unavoidable because of the nature of simulating cameras - but we need to structure the code to minimize what we can't cover to just the content that is actually tied to camera hardware.

There are also some outstanding items from my previous review, including some inline comments about mapping ObjC APIs, and a requested extension to the Hardware example app for demonstration purposes.

freakboy3742 · 2026-06-22T06:53:31Z


+## Scanning for Barcodes
+
+The camera can be used to scan QR codes and other barcode types in real-time. Scanning is supported on iOS, macOS, and in the Dummy (test) backend.


We don't call out individual platform quirks (or availability) in the docs - they should describe the feature presuming it exists:

Suggested change

The camera can be used to scan QR codes and other barcode types in real-time. Scanning is supported on iOS, macOS, and in the Dummy (test) backend.

The camera can be used to scan QR codes and other barcode types in real-time.

freakboy3742 · 2026-06-22T06:55:40Z

    - Android: The `android.permission.CAMERA` permission must be declared.
- The iOS simulator implements the iOS Camera APIs, but is not able to take photographs. To test your app's Camera usage, you must use a physical iOS device.
+- The iOS simulator implements the iOS Camera APIs, but is not able to take photographs or scan barcodes. To test your app's Camera usage, you must use a physical iOS device.
+- Barcode scanning is currently available on iOS, macOS, and in the Dummy (test) backend. Other backends will raise `NotImplementedError`.


We don't reference the Dummy backend - it's purely a testing artefact.

Suggested change

- Barcode scanning is currently available on iOS, macOS, and in the Dummy (test) backend. Other backends will raise `NotImplementedError`.

- Availability of barcode scanning is currently limited to iOS and macOS.

freakboy3742 · 2026-06-22T06:58:07Z

        # granted from a different (inaccessible) thread, so it isn't picked up by
        # coverage.
-        def permission_complete(result) -> None:
+        def permission_complete(result) -> None:  # pragma: no cover


Why has this gained a no-cover?

freakboy3742 · 2026-06-22T07:00:09Z

+        """A handler to invoke when a barcode is detected during scanning.
+
+        The callback receives the camera as the first argument, and the detected content
+        as a keyword argument: ``on_detection(camera, content=content)``.


This is in ReST format, not Markdown

Suggested change

as a keyword argument: ``on_detection(camera, content=content)``.

as a keyword argument: `on_detection(camera, content=content)`.

freakboy3742 · 2026-06-22T07:00:22Z

+        If scanning was started with ``continuous=True``, the callback will be invoked
+        each time a barcode is detected. If ``continuous=False`` (the default), the


More ReST:

Suggested change

If scanning was started with ``continuous=True``, the callback will be invoked

each time a barcode is detected. If ``continuous=False`` (the default), the

If scanning was started with `continuous=True`, the callback will be invoked

each time a barcode is detected. If `continuous=False` (the default), the

This looks to be a recurring theme for the rest of this file.

freakboy3742 · 2026-06-22T07:02:39Z

+
+        result = ScanResult(None)
+        self._impl.start_scanning(
+            result, device=device, code_types=code_types, continuous=continuous


As a code style thing, once we get to a point where Ruff is breaking code onto a new line to get space for the arguments, we tend to go directly to "1 argument per line", rather than the intermediate "all args on one standalone line" format:

Suggested change

result, device=device, code_types=code_types, continuous=continuous

result,

device=device,

code_types=code_types,

continuous=continuous,

One argument per line might be more readable when the arguments are complex expressions, but forcing it for simple situations like this doesn't seem to accomplish anything except reducing the amount of code you can see on screen at once.

freakboy3742 · 2026-06-22T07:05:10Z

+    all_types = list(BarcodeFormat)
+    assert len(all_types) == 7
+    names = {str(t) for t in all_types}
+    assert names == {"Qr", "Code128", "Ean13", "Ean8", "Pdf417", "Aztec", "Data_Matrix"}


I can see what this is testing, but it's not a very strong test as there's no validation of constant to mapping. This might be better handled as a test parameterised over each of the enumerated values, looking for a specific str representation of each.

freakboy3742 · 2026-06-22T07:14:12Z

+                    self.camera._handle_scan(content)
+
+
+class TogaCameraScannerWindow(toga.Window):  # pragma: no cover


This is a big block to have marked no cover. If TogaCameraWindow is an indicator, only creating the session should be marked no-cover; everything else should be reachable in tests.

To that end ... it seems like there's a lot of overlapping content in the TogaCamera window. Is there any way to merge those two classes so there's a single UI for camera activity?

freakboy3742 · 2026-06-22T07:17:20Z

        self.camera.preview_windows.remove(self)


+class TogaCameraScannerDelegate(NSObject):  # pragma: no cover


Does this need to be a standalone class? Could we use the same AVSession subclass as the photo object uses, set it to be it's own delegate, migrate the body of the delegate method to a utility method on the Window, and use that to provide a point were we can just mock the Session object and everything else is tested explicitly?

mhsmith · 2026-06-22T08:27:38Z

+        device: CameraDevice | None = None,
+        code_types: BarcodeFormat | list[BarcodeFormat] | None = None,
+        on_detection: Callable | None = None,
+        continuous: bool = False,


We haven't been consistent about this elsewhere in Toga, but when a method has optional arguments with no obvious order, making them keyword-only tends to make the calling code more readable, especially when they're simple types like numbers or booleans. In this case, I'd say that applies to on_detection and continuous.

mhsmith · 2026-06-22T08:29:00Z

+        If ``continuous`` is ``False`` (the default), scanning stops automatically after
+        the first detection, and the returned ``ScanResult`` resolves with the detected
+        content string. If ``continuous`` is ``True``, scanning continues until
+        :meth:`stop_scanning` is called, and the ``ScanResult`` resolves with ``None``.


Part of this duplicates the continuous docstring, and the rest should be moved to the :returns: docstring.

phildini added 2 commits June 13, 2026 16:03

Add tests for BarcodeFormat enum to reach 100% constants coverage

67134a5

Add test_barcode_format_str and test_barcode_format_values to exercise all BarcodeFormat members and their __str__ methods.

Copilot started work on behalf of phildini June 14, 2026 00:43 View session

Defer iOS scan bindings until scanner startup

d555088

Copilot finished work on behalf of phildini June 14, 2026 00:55

freakboy3742 requested changes Jun 15, 2026

View reviewed changes

phildini and others added 6 commits June 15, 2026 12:47

Update changes/camera-scanning.feature.md

fcc8343

Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>

Restore explanatory comments in iOS test probe

ea52118

Restore all explanatory comments removed from iOS backend

8ff7b03

Restore NSCameraUsageDescription, request_permission thread, take_photo configuration, delegate, and presentation comments that were lost during the rewrite of the iOS camera backend.

freakboy3742 reviewed Jun 16, 2026

View reviewed changes

Comment thread changes/camera-scanning.feature.md Outdated

Use PR-numbered changenote with reviewer's suggested wording

c6e5035

phildini requested a review from freakboy3742 June 18, 2026 18:10

freakboy3742 requested changes Jun 22, 2026

View reviewed changes

mhsmith reviewed Jun 22, 2026

View reviewed changes

	output.setMetadataObjectsDelegate_queue_(self._scan_delegate, None)
	output.setMetadataObjectsDelegate(self._scan_delegate, queue=None)


		## Scanning for Barcodes

		The camera can be used to scan QR codes and other barcode types in real-time. Scanning is supported on iOS, macOS, and in the Dummy (test) backend.

	- Barcode scanning is currently available on iOS, macOS, and in the Dummy (test) backend. Other backends will raise `NotImplementedError`.
	- Availability of barcode scanning is currently limited to iOS and macOS.

	as a keyword argument: ``on_detection(camera, content=content)``.
	as a keyword argument: `on_detection(camera, content=content)`.

		If scanning was started with ``continuous=True``, the callback will be invoked
		each time a barcode is detected. If ``continuous=False`` (the default), the

		self.camera._handle_scan(content)


		class TogaCameraScannerWindow(toga.Window): # pragma: no cover

		self.camera.preview_windows.remove(self)


		class TogaCameraScannerDelegate(NSObject): # pragma: no cover

Uh oh!

Uh oh!

Conversation

phildini commented Jun 13, 2026 • edited by freakboy3742 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Checklist:

Uh oh!

phildini commented Jun 13, 2026

Uh oh!

phildini commented Jun 14, 2026

Uh oh!

freakboy3742 commented Jun 14, 2026

Uh oh!

freakboy3742 commented Jun 14, 2026

Uh oh!

freakboy3742 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mhsmith commented Jun 17, 2026

Uh oh!

freakboy3742 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

phildini commented Jun 13, 2026 •

edited by freakboy3742

Loading