rp2040/rp23xx USB: missing DMB between BUFF_STATUS clear and AVAILABLE re-arm loses IN completions on RP2350 (Cortex-M33)

[ricardgb]

Description

On RP2350 (Cortex-M33), the USB device driver (arch/arm/src/rp23xx/rp23xx_usbdev.c)
re-arms an endpoint by writing the buffer-control AVAILABLE bit without a memory
barrier ordering it after the preceding BUFF_STATUS clear. These are two writes to two
different peripheral regions:

• the BUFF_STATUS clear (done in rp23xx_usbintr_buffstat when handling a completion)
  targets the USB controller register block at 0x50110000 (BUFF_STATUS @ +0x58);
• setting AVAILABLE in the endpoint buffer-control word targets USB DPSRAM at
  0x50100000.

On the Cortex-M33 bus fabric, writes to different Device regions may be observed out of
order unless separated by a DMB. In a non-SMP build (CONFIG_SMP unset)
spin_lock_irqsave compiles to a barrier-free up_irq_save, so nothing orders these two
stores.

The race (lost IN completion)

When re-arming the next IN buffer directly from a completion handler under back-to-back TX:

1. AVAILABLE (DPSRAM) is observed by the controller before the BUFF_STATUS clear
   (register block) lands.
2. The controller transmits the next IN packet and latches its completion by setting the
   BUFF_STATUS bit.
3. The late-arriving clear then wipes that just-set bit.

The completion edge is lost: the IN endpoint never raises another buffer interrupt, the
class wrcomplete callback never runs, and (for CDC-NCM) wrreq_idle is never reposted, so
transmit wedges permanently. The controller-register note in the header even warns:
"clearing the buffer status bit may instantly re-set it on the next clock cycle."

Steps to reproduce

• RP2350 (Raspberry Pi Pico 2 W), CDC-NCM composite acting as a USB-NIC.
• Enable CONFIG_NET_TCP_WRITE_BUFFERS=y so TCP transmit is bursty (a single txavail
  poll drains many segments back-to-back), keeping the IN pipe continuously full.
• Drive sustained/concurrent HTTP downloads.

Actual: transmit wedges intermittently (worse with denser TX). The core is idle
(up_idle), no fault, wrreq_idle never reposted. Over SWD in the wedged state:
BUFF_STATUS = 0, USB INTS = 0, the bulk-IN buffer-control word shows the buffer
prepared but AVAILABLE clear (e.g. 0x0000_2024 = PID + length, no AVAILABLE), i.e. a
lost completion edge — hardware quiescent while the driver waits forever.

Expected: sustained TX never wedges.

Why RP2040 is unaffected

The identical logic runs on RP2040 (Cortex-M0+), whose far stricter/simpler store ordering
does not exhibit this reorder — so this is RP2350/Cortex-M33-specific.

Proposed fix

Insert a UP_DMB() (and #include <arch/barriers.h>) immediately before setting the
AVAILABLE bit on the re-arm path, so every preceding controller-register write (the
BUFF_STATUS clear) is ordered ahead of the DPSRAM AVAILABLE store.

This was root-caused and fixed on an RP2350 (Pico 2 W) CDC-NCM USB-NIC: with the barrier,
a soak of 144 dense/concurrent HTTP downloads at ~486 KB/s ran with zero wedges (previously
transmit hung within a handful of requests). A patch is available; I can open a PR against
whichever revision of the RP2350 IN-endpoint re-arm path is preferred (the exact call site
depends on recent driver refactors — refs the USB endpoint work in #19378/#19450/#19451 and
the CDC-NCM TX fixes in #19469/#19470).

Environment

• Chip: RP2350 (Cortex-M33), non-SMP.
• Board: Raspberry Pi Pico 2 W.
• NuttX: current master.

---

Disclosure: this root-cause analysis was performed with the assistance of an AI agent
(Anthropic's Claude, via Claude Code), operated and directed by the submitter, and validated
on-hardware before filing.