Whitebox Plugin - Transfer Manager¶

Manages file transfers from external devices (cameras, etc.) with a two-phase workflow: download from device, then process (transcode/stitch/convert). Tracks persistent state, emits real-time progress to the frontend, and handles retries and crash recovery automatically.

Installation¶

poetry add whitebox-plugin-transfer-manager

Capabilities¶

Provides	Requires
`transfer-manager`	`device`

Exposed to Other Plugins¶

Plugin Classes (`get_plugin_classes_map`)¶

Accessible via import_whitebox_plugin_class():

Key	Type	Description
`transfer.TransferManager`	`TransferManager` instance	Singleton service for queueing, dispatching, and tracking transfers
`transfer.TransferAdapter`	`Protocol` class	Interface that device plugins implement for download/processing
`transfer.RemoteFile`	`dataclass` class	Represents a file on a remote device

Models (`model_registry`)¶

Key	Model	Description
`transfer.Transfer`	`Transfer`	Persistent record for each file transfer

Frontend Components (`exposed_component_map`)¶

Category	Key	Component	Description
`service-component`	`transfer-service`	`TransferServiceComponent`	Background service that listens for WebSocket transfer events, updates the Zustand store, and triggers flight session re-fetches on completion
`transfer-manager`	`transfer-panel`	`TransferPanel`	UI panel showing active/completed transfers with progress bars, cancel, and retry actions

Frontend Slots (`slot_component_map`)¶

Slot	Component
`transfer.panel`	`TransferPanel`

State Stores (`state_store_map`)¶

Key	Module	Description
`transfer`	`stores/transfer`	Zustand store tracking transfer state (`transfers`, `activeCount`, `upsertTransfer`, `removeTransfer`, `clearCompleted`, `dismissAll`)

Daemon¶

Command: daemon_transfer_manager

Runs as a BaseDaemonCommand that orchestrates all transfers.

Subscribed Events¶

Event	Action
`observation.flight.end`	Dispatch any pending transfers
`observation.device.connection_status.update`	On reconnect, dispatch pending transfers for that device
`command.transfer.cancel`	Cancel a transfer (sets Redis flag for in-progress workers)
`command.transfer.retry`	Retry a failed/cancelled transfer
`command.transfer.queue_batch`	Create transfer records from a batch of files and dispatch

Periodic Checks (every 5s)¶

Stale transfer recovery: Resets transfers stuck in active states with no DB update for 2+ minutes (worker likely dead)
Dispatch pending: Picks up PENDING transfers respecting concurrency limits

Startup Recovery¶

Resets any in-progress transfers (DOWNLOAD_STARTED, DOWNLOAD_COMPLETED, PROCESSING_STARTED) back to PENDING
Clears retry_after on PENDING transfers that were in backoff when daemon stopped

Emitted Events¶

All events are sent to the management WebSocket group.

Event	When
`observation.transfer.queued`	Transfer created or reset to pending
`observation.transfer.download.started`	Download begins
`observation.transfer.download.progress`	Download progress (throttled: 1s or 5% delta)
`observation.transfer.download.completed`	Download finished
`observation.transfer.processing.started`	Processing begins
`observation.transfer.processing.progress`	Processing progress (throttled: 1s or 5% delta)
`observation.transfer.processing.completed`	Processing finished, FlightSessionRecording created
`observation.transfer.download.failed`	Download failed permanently
`observation.transfer.processing.failed`	Processing failed permanently
`observation.transfer.retry_scheduled`	Transfer scheduled for retry with backoff
`observation.transfer.cancelled`	User cancelled transfer
`observation.transfer.paused`	User paused transfer

Registering an Adapter¶

Device plugins expose adapters via their plugin class:

# In device plugin's whitebox plugin class
def get_transfer_adapters(self):
    from .transfer_adapter import MyDeviceTransferAdapter
    return [("my_device_codename", MyDeviceTransferAdapter())]

The daemon discovers these on startup; workers discover lazily on first use.

TransferManager API¶

Queueing¶

from plugin.registry import import_whitebox_plugin_class

transfer_manager = import_whitebox_plugin_class("transfer.TransferManager")
RemoteFile = import_whitebox_plugin_class("transfer.RemoteFile")

# Queue a batch of files (sync, for use in RQ tasks or daemon)
files = [RemoteFile(path="/DCIM/VID_001.insv", size=1_000_000)]
transfers = transfer_manager.queue_batch_sync(
    device_connection=conn,
    files=files,
    download_dir=Path("/media/transfers/downloads"),
    processed_dir=Path("/media/transfers/processed"),
    associate_with=flight_session,
)

Concurrency¶

Setting	Default	Description
Max concurrent global	4	Total active transfers across all devices
Max concurrent per device	2	Active transfers per device connection

Retry Logic¶

Transient errors (network, device) retry with exponential backoff: 2s, 4s, 8s, ..., capped at 60s
Retry window: 7 days from initial queue time
Safety cap: 500 retries max
Non-transient errors (disk full, corrupt file) do not retry