supervisor/filesystem: let alternative writers (BLE-FT, web workflow) bypass STA_PROTECT while they hold the lock

[makermelissa-piclaw]

Fixes #10972.

Summary

After PR #10659, alternative filesystem writers (BLE File Transfer, web workflow, supervisor_workflow_*) can take the MP_BLOCKDEV_FLAG_LOCKED blockdev mutex but cannot actually write through FatFS, because disk_ioctl(IOCTL_STATUS) calls filesystem_is_writable_by_python() which returns false on any USB-device-capable board after main.c boot — so f_open(FA_WRITE) returns FR_WRITE_PROTECTED.

This 11-line patch makes filesystem_lock() set MP_BLOCKDEV_FLAG_IGNORE_WRITE_PROTECTION while the lock is held, and filesystem_unlock() clear it. This mirrors the pattern main.c already uses around boot.py execution. USB MSC continues to mutex via LOCKED (it takes the lock through blockdev_lock() directly, not through filesystem_lock(), so IGNORE is never set on its behalf).

Why this shape

I prototyped an alternative first: adding (LOCKED == 0) as a fourth clause in filesystem_is_writable_by_python(). That worked for the "Python open(W) on battery" case I tested, but:

1. It does not fix the issue's primary repro. BLE File Transfer calls filesystem_lock() (setting LOCKED=1) before calling f_open(FA_WRITE). Once LOCKED=1, the new (LOCKED == 0) clause is also false, and f_open still fails with FR_WRITE_PROTECTED. So that variant fixes a different case from what the issue reports.
2. It opens a concurrent-writer race: Python open(W) succeeds without taking any lock, so a concurrent BLE-FT or web workflow filesystem_lock() would also succeed (LOCKED was 0), and two writers operate on FatFS at the same time.

This patch avoids both problems:

• filesystem_is_writable_by_python() is byte-identical to its pre-BLE File Transfer writes blocked as STATUS_ERROR_READONLY when no USB host attached #10972 form (verified via objdump — same and #0xb0 / sub #0x30 codegen on Cortex-M4). No regression to writability semantics.
• Holders of filesystem_lock() are explicitly granted write permission for the duration of the critical section. IGNORE_WRITE_PROTECTION is a flag specifically intended for this — already used by main.c to allow boot.py's boot_out.txt write.
• USB MSC continues to use the same LOCKED mutex (via blockdev_lock(), not filesystem_lock()), so the symmetric host-vs-local mutex from Fix filesystem writability from USB #10659 is preserved.
• Python open(W) without a lock remains blocked in the default state, so users still need storage.disable_usb_drive() or storage.remount(readonly=False) for direct Python writes. Same behavior as today; this PR is not changing that escape hatch.

Verification

Built v2 firmware via the Build board (custom) workflow on my fork:

• Qualia S3 RGB666 (ESP32-S3): https://github.com/makermelissa-piclaw/circuitpython/actions/runs/27223440733
• CLUE nRF52840: https://github.com/makermelissa-piclaw/circuitpython/actions/runs/27223443863

Hardware-verified on CLUE nRF52840 (10.3.0-alpha.2-25-ga8924a8e):

Scenario	Result	Expected
Default state (host MSC attached, no boot.py) — mount.readonly	True	True (no regression) ✅
Default state — Python open(W)	OSError(30)	EROFS ✅
storage.disable_usb_drive() in boot.py — mount.readonly	False	False ✅
storage.disable_usb_drive() in boot.py — Python open(W) + os.rename + os.sync + readback after reset	OK	OK ✅

Binary inspection of firmware.elf confirms the patch is in the build:

filesystem_lock:                                       filesystem_unlock:
  ldrsb [r0, #768]                                       ldrb [r0, #768]
  ldrh  r2, [r0, #4]                                     subs r3, #1
  ...                                                    ...
  orr.w r2, r2, #64    ← LOCKED (0x40)                   bic.w r3, r3, #192  ← clears LOCKED|IGNORE (0xc0)
  orr.w r2, r2, #128   ← IGNORE_WRITE_PROTECTION (0x80)
  strh  r2, [r0, #4]

filesystem_is_writable_by_python is byte-identical to its pre-#10972 form (3-clause check folded into and #0xb0 / cmp #0x30).

What this patch does NOT do

I'm intentionally keeping this PR minimal. Two adjacent improvements suggested in #10972 are not in scope here:

1. The issue suggests distinguishing lock-contention from read-only by introducing STATUS_ERROR_BUSY = 0x06 (or similar) in BLE-FT protocol. Reasonable UX improvement, but orthogonal — the lock-busy case will be rare in practice once filesystem_lock() is the right mutex. Happy to do this as a follow-up if you'd like.

2. I have not exercised the actual BLE-FT write path end-to-end on hardware from this PR's environment (Pi USB-host setup suppresses BLE-FT advertising when CIRCUITPY_USB_DEVICE is connected; getting into discovery mode reliably needs a double-tap reset I don't have remote-control access for). The change is in shared code with a small, well-typed surface, and I've verified the binary matches the source and the regression-free path empirically. If a reviewer has a CLUE-on-battery rig and @adafruit/ble-file-transfer-js (web-editor over BLE) handy, that's the one path I couldn't directly cover.

Test plan suggestion for reviewer

On a BLE-FT-capable board (CLUE, Feather Sense, etc.) running on battery (no USB host):

1. Pair via web-editor or ble-file-transfer-js client
2. Try WRITE / MOVE / MKDIR / DELETE commands
3. Pre-patch: all return status 0x05 (STATUS_ERROR_READONLY)
4. Post-patch: all should succeed

For web workflow (any WiFi board with CIRCUITPY_WEB_WORKFLOW=1) on USB power adapter (no host):

1. Connect via web-editor or curl
2. PUT /fs/path/to/file.txt
3. Pre-patch: 409 Conflict / read-only
4. Post-patch: 201 / 204

---

cc @dhalbert — this is a direct follow-up on the regression introduced by #10659; would appreciate your eye on whether the IGNORE pattern is the right shape here, or if you'd prefer a different approach (e.g. explicit per-writer flag).

[dhalbert]

I am not sure about this fix. I am looking at the current semantics related to remounting in boot.py.

[makermelissa-piclaw]

Thanks for taking a look. Here's my analysis of the boot.py remount interaction — please tell me which case is bothering you so I can think about it more concretely.

storage.remount() doesn't go through filesystem_lock. It calls blockdev_lock() directly (shared-module/storage/__init__.c:192), then filesystem_set_writable_by_usb(fs, readonly) (line 197), then blockdev_unlock(). So this PR doesn't touch its semantics:

boot.py call	USB_WRITABLE after	_is_writable_by_python returns	Behavior
(no remount)	1 (set by main.c)	false (clauses CWP=1, USB_W=1, IGNORE=0)	Python writes blocked, host writes allowed — same as today
storage.remount('/', readonly=False)	0	true (clause 2: USB_W=0)	Python writes allowed, host can't — same as today
storage.remount('/', readonly=True)	1	false	Same as default
storage.disable_usb_drive()	0 (via usb_drive_set_enabled(false))	true	Same as today

I verified this on hardware: mount.readonly matches the column above for all four, with and without v2 applied.

The IGNORE flag is only set inside filesystem_lock() / cleared in filesystem_unlock(). Those functions are called from supervisor_workflow_move/mkdir/delete and from _process_write in bluetooth/file_transfer.c — paths that are already mutexed against USB MSC via the existing LOCKED/blockdev_lock pair. Nothing else calls them. storage.remount doesn't, the Python open(W) path doesn't, and tud_msc_write10_cb uses blockdev_lock directly.

A few specific interactions I checked:

• boot.py does remount(readonly=True) then BLE-FT WRITE arrives: USB_W=1, but BLE-FT takes the lock, IGNORE=1 → write succeeds. Matches intent (readonly=True blocks Python; BLE-FT is logically grouped with the host-side writer).
• boot.py does remount(readonly=False) then BLE-FT WRITE arrives: USB_W=0, IGNORE doesn't matter, write succeeds via the existing USB_W=0 clause. Lock still serializes against any USB host that re-attaches.
• storage.remount while BLE-FT holds the lock: blockdev_lock() already returns false → RuntimeError("Cannot remount path when visible via USB.") is raised (slightly misleading text now that LOCKED can also be held by BLE-FT, but the behavior is correct).
• Lock holder crashes between _lock and _unlock: IGNORE stays set, so Python could subsequently write. Symmetric with the pre-patch hazard of a stuck LOCKED making the filesystem permanently appear read-only — I didn't make this worse, but happy to discuss if you'd like a guard.

If you'd prefer a different shape entirely — e.g. dropping IGNORE and instead having disk_ioctl(IOCTL_STATUS) consult (flags & LOCKED) != 0 to mean "a workflow holds the lock, treat the volume as writable for this op" — I can rework. The IGNORE-via-lock approach mirrored the existing main.c / boot.py pattern, which is why I landed there, but I have no attachment to it.

(About to do an empirical BLE-FT WRITE test on a battery-powered CLUE — happy to defer if you want me to wait on a shape change first. The change works in principle; whether it's the right shape is your call.)

[dhalbert]

I didn't mean for you to check something. I was just trying to understand the current semantics myself.

Currently (as of 10.0.0, not after the regression), are web workflow and BLE workflow supposed to be able to write files if the host can also write files? web and BLE workflow are transactional, so we know when they are done. USB MSC is not transactional, so it's unsafe to use it simultaneously with other workflows.

Are filesystem_lock() and filesystem_unlock() called only by BLE and wifi workflow?

Is there a race condition on startup? Maybe wifi or BLE workflow would connect before the USB enumeration finished.

[makermelissa-piclaw]

Thanks for the framing — those are the right questions. Auditing the tree to answer all three concretely:

Q2: Are filesystem_lock() / filesystem_unlock() called only by BLE and web workflow?

Yes. Complete caller list (grepped at a8924a8e):

• supervisor/shared/workflow.c — supervisor_workflow_move(), supervisor_workflow_mkdir(), supervisor_workflow_delete() (these are the shared helpers that both web workflow and the future supervisor REST surface route through)
• supervisor/shared/web_workflow/web_workflow.c — write-path handlers (PUT, MOVE, MKCOL, DELETE)
• supervisor/shared/bluetooth/file_transfer.c — BLE-FT _process_write/move/mkdir/delete

No calls from Python land, no calls from storage.remount (which uses blockdev_lock directly), no calls from tud_msc_* callbacks. So IGNORE_WRITE_PROTECTION set by this patch is only ever observed during a workflow critical section.

Q1: At 10.0.0 (pre-#10659), were web/BLE workflow supposed to write while the host can also write?

Reading the 10.0.0 code: not simultaneously, but they coexist via the same mutex. Both host MSC and workflows take blockdev_lock() on the same fs_user_mount_t:

• Host MSC: tud_msc_is_writable_cb() calls blockdev_lock() on first write inquiry; releases in usb_msc_umount() or eject.
• Workflows: filesystem_lock() wraps blockdev_lock() per operation.

At 10.0.0, if the host is actively connected and has issued a WRITE10, it holds the lock until disconnect — workflows get FR_WRITE_PROTECTED until then. If the host hasn't claimed write yet (or isn't enumerated at all), the workflow takes the lock for the duration of one transaction. So the design is "transactional workflow exclusive with sticky-host MSC," matching your "workflows are transactional, MSC is not" framing.

This patch preserves that exact semantics. The only delta from 10.0.0 is restoring f_open(FA_WRITE) to succeed inside a held filesystem_lock() — which 10.0.0 allowed implicitly (no _is_writable_by_python check in disk_ioctl(IOCTL_STATUS) at all), and which #10659 unintentionally broke for every USB-device-capable board.

Q3: Is there a startup race? Could WiFi/BLE workflow connect before USB enumeration finishes?

No, by construction. From supervisor/shared/workflow.c:94:

void supervisor_workflow_start(void) {
    // Start USB after giving boot.py a chance to tweak behavior.
    #if CIRCUITPY_USB_DEVICE
    usb_init();        // <-- USB hardware comes up FIRST
    #endif
    serial_init();
    #if CIRCUITPY_BLEIO
    bleio_reset();
    supervisor_start_bluetooth();   // <-- then BLE
    #endif
    #if CIRCUITPY_WEB_WORKFLOW
    if (supervisor_start_web_workflow()) {   // <-- then web
        ...
    }
    #endif
    ...
}

usb_init() calls init_usb_hardware() + tud_init() synchronously before either bluetooth or web workflow is started. By the time BLE advertising starts or the WiFi socket is listening, TinyUSB is enumerating in parallel.

That still leaves a brief window where TinyUSB has called tud_init() but the host hasn't enumerated yet — in that window:

• Host hasn't issued WRITE10 → MSC hasn't called blockdev_lock() → workflow can grab the lock.
• If host then enumerates and tries to write, tud_msc_is_writable_cb calls blockdev_lock(), sees LOCKED=1, returns "not writable" → host sees a read-only volume until the workflow's transaction completes and releases.

That's the same window 10.0.0 had, and the same window USB-vs-storage.remount has in boot.py. Nothing new from this PR.

---

So in short: this patch restores the 10.0.0 invariant of "workflow that holds the lock can write through FatFS," while keeping the _is_writable_by_python() check that #10659 added for the Python open(W) path (which doesn't go through the lock). Happy to do it differently if you want a different shape — e.g. teaching disk_ioctl(IOCTL_STATUS) to consult (flags & LOCKED) != 0 directly so we don't even need to set IGNORE. That's a one-liner too and arguably even less surface area. Just say the word.

[dhalbert]

I tested this with BLE:

• on a Feather nRF52840. With FileGlider, I was able to edit files, when the device was not also connected as a USB device. code.circuitpython.org over BLE was flakier: I got REPL output but not input.
• on a Metro S3. Again, I was able to use FileGlider to edit files. I could not pair or connect with a Linux desktop.

I tested with WiFi on a Metro ESP32-S3. I was able to edit files when the USB connection was power-only.

So I think this fixes the read-only problem we were seeing. BLE workflows are still quite flaky via code.circuitpython.org, but that is a different problem.

@melissa thanks for setting piclaw to work on this.

[makermelissa]

No worries. It was blocking on CP Code Editor work.