Natives Architecture

@oh-my-pi/pi-natives is a three-layer stack:

1. TypeScript wrapper/API layer exposes stable JS/TS entrypoints.
2. Addon loading/validation layer resolves and validates the .node binary for the current runtime.
3. Rust N-API module layer implements performance-critical primitives exported to JS.

This document is the foundation for deeper module-level docs.

Implementation files

• packages/natives/src/index.ts
• packages/natives/src/native.ts
• packages/natives/src/bindings.ts
• packages/natives/src/embedded-addon.ts
• packages/natives/scripts/build-native.ts
• packages/natives/scripts/embed-native.ts
• packages/natives/package.json
• crates/pi-natives/src/lib.rs

Layer 1: TypeScript wrapper/API layer

packages/natives/src/index.ts is the public barrel. It groups exports by capability domain and re-exports typed wrappers rather than exposing raw N-API bindings directly.

Current top-level groups:

• Search/text primitives: grep, glob, text, highlight
• Execution/process/terminal primitives: shell, pty, ps, keys
• System/media/conversion primitives: image, html, clipboard, system-info, work

packages/natives/src/bindings.ts defines the base interface contract:

• NativeBindings starts with shared members (cancelWork(id: number))
• module-specific bindings are added by declaration merging from each module’s types.ts
• Cancellable standardizes timeout and abort-signal options for wrappers that expose cancellation

Guaranteed contract (API-facing): consumers import from @oh-my-pi/pi-natives and use typed wrappers.

Implementation detail (may change): declaration merging and internal wrapper layout (src/<module>/index.ts, src/<module>/types.ts).

Layer 2: Addon loading and validation

packages/natives/src/native.ts owns runtime addon selection, optional extraction, and export validation.

Candidate resolution model

• Platform tag is "${process.platform}-${process.arch}".
• Supported tags are currently:
  • linux-x64
  • linux-arm64
  • darwin-x64
  • darwin-arm64
  • win32-x64
• x64 can use CPU variants:
  • modern (AVX2-capable)
  • baseline (fallback)
• Non-x64 uses the default filename (no variant suffix).

Filename strategy:

• Release: pi_natives.<platform>-<arch>.node
• x64 variant release: pi_natives.<platform>-<arch>-modern.node and/or ...-baseline.node
• Dev: pi_natives.dev.node (preferred when PI_DEV is set)

Platform-specific variant detection

For x64, variant selection uses:

• Linux: /proc/cpuinfo
• macOS: sysctl machdep.cpu.leaf7_features / machdep.cpu.features
• Windows: PowerShell check for System.Runtime.Intrinsics.X86.Avx2

PI_NATIVE_VARIANT can explicitly force modern or baseline.

Binary distribution and extraction model

packages/natives/package.json includes both src and native in published files. The native/ directory stores prebuilt platform artifacts.

For compiled binaries (PI_COMPILED or Bun embedded runtime markers), loader behavior is:

1. Check versioned user cache path: <getNativesDir()>/<packageVersion>/...
2. Check legacy compiled-binary location:
   • Windows: %LOCALAPPDATA%/pisces (fallback %USERPROFILE%/AppData/Local/pisces)
   • non-Windows: ~/.local/bin
3. Fall back to packaged native/ and executable directory candidates

If an embedded addon manifest is present (embedded-addon.ts generated by scripts/embed-native.ts), native.ts can materialize the matching embedded binary into the versioned cache directory before loading.

Validation and failure modes

After require(candidate), validateNative(...) verifies required exports (for example grep, glob, highlightCode, PtySession, Shell, getSystemInfo, getWorkProfile, invalidateFsScanCache).

Failure paths are explicit:

• Unsupported platform tag: throws with supported platform list
• No loadable candidate: throws with all attempted paths and remediation hints
• Missing exports: throws with exact missing names and rebuild command
• Embedded extraction errors: records directory/write failures and includes them in final load diagnostics

Guaranteed contract (API-facing): addon load either succeeds with a validated binding set or fails fast with actionable error text.

Implementation detail (may change): exact candidate search order and compiled-binary fallback path ordering.

Layer 3: Rust N-API module layer

crates/pi-natives/src/lib.rs is the Rust entry module that declares exported module ownership:

• clipboard
• fd
• fs_cache
• glob
• glob_util
• grep
• highlight
• html
• image
• keys
• prof
• ps
• pty
• shell
• system_info
• task
• text

These modules implement the N-API symbols consumed and validated by native.ts. JS-level names are surfaced through the TS wrappers in packages/natives/src.

Guaranteed contract (API-facing): Rust module exports must match the binding names expected by validateNative and wrapper modules.

Implementation detail (may change): internal Rust module decomposition and helper module boundaries (glob_util, task, etc.).

Ownership boundaries

At architecture level, ownership is split as follows:

• TS wrapper/API ownership (packages/natives/src)
  • public API grouping, option typing, and stable JS ergonomics
  • cancellation surface (timeoutMs, AbortSignal) exposed to callers
• Loader ownership (packages/natives/src/native.ts)
  • runtime binary selection
  • CPU variant selection and override handling
  • compiled-binary extraction and candidate probing
  • hard validation of required native exports
• Rust ownership (crates/pi-natives/src)
  • algorithmic and system-level implementation
  • platform-native behavior and performance-sensitive logic
  • N-API symbol implementation that TS wrappers consume

Runtime flow (high level)

1. Consumer imports from @oh-my-pi/pi-natives.
2. Wrapper module calls into singleton native binding.
3. native.ts selects candidate binary for platform/arch/variant.
4. Optional embedded binary extraction occurs for compiled distributions.
5. Addon is loaded and export set is validated.
6. Wrapper returns typed results to caller.

Glossary

• Native addon: A .node binary loaded via Node-API (N-API).
• Platform tag: Runtime tuple platform-arch (for example darwin-arm64).
• Variant: x64 CPU-specific build flavor (modern AVX2, baseline fallback).
• Wrapper: TS function/class that provides typed API over raw native exports.
• Declaration merging: TS technique used by module types.ts files to extend NativeBindings.
• Compiled binary mode: Runtime mode where the CLI is bundled and native addons are resolved from extracted/cache paths instead of only package-local paths.
• Embedded addon: Build artifact metadata and file references generated into embedded-addon.ts so compiled binaries can extract matching .node payloads.
• Validation gate: validateNative(...) check that rejects stale/mismatched binaries missing required exports.