# Mnemosyne

A sensor-fusion runtime that hot-plugs new input modalities into a frozen trained core. A new sensor is integrated by fine-tuning only a small adapter (~22 K params) while the core and fuser stay frozen, so adding a modality does not require retraining or redeploying the model.

- **Runtime:** CPU-only; no GPU required. Inference under 200 MB RAM (int8-quantized).
- **Requirements:** Python + PyTorch; the GGUF C loader (`c_loader/`) builds with any C compiler.
- **Determinism:** single seed (`mnemosyne.SEED = 1337`) seeds torch/numpy/random; data windows derive from `(seed_base, sample_index)`, so identical seeds reproduce identical runs.

## Install & train

```bash
pip install torch   # CPU build is sufficient

# Phase-1 warm start: train core + fuser + two known modalities (rf, spectrogram)
python -m mnemosyne.train          # writes checkpoints/

# Evaluate a never-seen modality end to end
python -m mnemosyne.demo --eval-radar
python -m mnemosyne.demo --eval-gesture
```

## Core API

### FuserLoom

The whole fusion system: backbone + fuser + ports + output head.

```python
from mnemosyne.loom import FuserLoom

loom = FuserLoom(
    dim=64,         # core latent dim
    d_enc=32,       # encoder output dim per port
    num_classes=5,
    core_vhdus=3,
    state_size=32,
)
```

Fused output = `head(backbone(fuser(fused_feature_stream)))`, where `fused_feature_stream` is the gate-weighted sum over active ports concatenated with the backbone's temporal read of the same stream.

### Port registry

Each modality is a `ModalityPort` (encoder → dimension sandbox). Built-in encoders:

| Name | Encoder | Input width (`d_in`) |
|---|---|---|
| `rf` | `RFEncoder` | 64 |
| `spectrogram` | `SpectrogramEncoder` | 32 |
| `radar` | `RadarEncoder` | 48 |
| `vision` | `VisionEncoder` | 48 |
| `gesture` | `GestureEncoder` | 24 |

```python
from mnemosyne.ports import build_port

port = build_port("radar", dim=64, d_enc=32)  # -> ModalityPort
```

### Adding a novel modality (QuickSwap)

```python
from mnemosyne.quickswap import add_novel_port, quicklearn, quickremove, quickadd_and_learn

# One-call version:
report = quickadd_and_learn(loom, "radar", steps=250, lr=5e-3, device="cpu")

# Step by step:
add_novel_port(loom, "radar")     # fresh random encoder + mixing gate
quicklearn(loom, "radar",
           steps=250,             # gradient steps (fuser-only fine-tune)
           lr=5e-3,
           drop_p=0.9,            # known-sensor dropout probability
           seed_offset=0)         # batch-stream offset for reproducibility

quickremove(loom, "radar")        # remove a port; other ports untouched
```

`quicklearn` trains only the target port's parameters plus its mixing gate; the backbone, fuser, and other ports stay frozen. Sensor dropout unplugs random subsets of known sensors per batch so the port must actually learn the new stream.

### Custom modality ports

To integrate your own sensor, provide an encoder whose forward emits `(batch, seq_len, d_enc)` frames and wrap it as a port:

```python
import torch.nn as nn
from mnemosyne.ports import ModalityPort
from mnemosyne.fuser import DimensionSandbox

class MySensorEncoder(nn.Module):
    def __init__(self, d_in, d_enc):
        super().__init__()
        self.net = nn.Sequential(
            nn.Conv1d(d_in, 32, 3, padding=1), nn.GELU(),
            nn.AdaptiveAvgPool1d(1),
        )
        self.proj = nn.Linear(32, d_enc)

    def forward(self, x):            # x: (batch, seq_len, d_in)
        b, t, d = x.shape
        h = self.net(x.reshape(b * t, d, 1)).squeeze(-1)
        return self.proj(h).reshape(b, t, -1)

encoder = MySensorEncoder(d_in=my_width, d_enc=loom.d_enc)
adapter = DimensionSandbox(loom.d_enc, loom.dim)
port = ModalityPort("mysensor", encoder, loom.d_enc, loom.dim)
loom.add_port(port, known=False)   # unknown provenance -> novel
```

Then run `quicklearn(loom, "mysensor", ...)` with windows from your sensor. Synthetic generators for the built-in modalities live in `mnemosyne.sensors` (`gen_rf_window`, `gen_spectrogram_window`, `gen_radar_window`, `gen_vision_window`, `gen_gesture_window`, `gen_batch`) if you need reference shapes.

## Persistence: GGUF bundle

```python
import torch
from mnemosyne.gguf import write_gguf, read_gguf

write_gguf(
    "checkpoints/fusion.gguf",
    loom,
    fused_embedding=torch.zeros(loom.dim),
    metadata={"name": "fusion", "seed": 1337},
)
data = read_gguf("checkpoints/fusion.gguf")   # dict of tensors + metadata KV
```

Bundle contents:

- `fused_embedding` `(dim,)` — current fused semantic vector
- `codebook` `(num_classes, dim)` — frozen retrieval targets
- per-port tensors `{name}.{param}` — int8 weights when available, fp16 otherwise
- metadata KV block: `name`, `seed`, `known[]`, `novel[]`, `class_names[]`, `port_info[]` (name, d_in, d_enc, params), `memory`

A dependency-free reader (`c_loader/mnemosyne_loader.c`, stdio-only) reads the same files; build with `make -C c_loader`. Bundle size ≈ 94 KB for the default configuration.

## Quantized inference

```python
from mnemosyne.eval import quantize_model

qloom = quantize_model(loom)       # int8 copy of the whole loom
# same forward pass through dequantized weights:
out = qloom(windows)
```

Measured cost: ~6–7 ms/sample, ~520 KB of weights, accuracy unchanged versus fp32 on the reference tasks.

## Enclave (licensing API)

Session budgeting and license enforcement live inside the model weights (`enclave_state` buffers persist with the model). Relevant entry points:

```bash
python -m mnemosyne.enclave_cli issue --kind dev --seats 4 --out seat.mnport   # issuer side
python -m mnemosyne.enclave_cli install seat.mnport                            # consumer side
python -m mnemosyne.enclave_cli status                                         # inspect state
```

```python
from mnemosyne.enclave import Enclave

enclave = Enclave(loom)
enclave.install_port("seat.mnport")   # validates form, HMAC tag, checksum, seat binding, lifespan
print(enclave.status())               # dict: kind, seats, lifespan, governor state
```

Programmatic issuance for site-side integrations: `mnemosyne.enclave.issue_port(kind="dev"|"prod", seats=N, seat_id=..., ...)` returns canonical single-line `.mnport` text; consumers may pass it via `mnemosyne.enclave.install_from_text(text, enclave)`. Port text is byte-canonical — re-wrapped or indented copies are rejected.

## Module map

| Module | Role |
|---|---|
| `mnemosyne.vhdu` | causal selective SSM block (VHDU) |
| `mnemosyne.nodeunits` | NodeUnit ensemble + Backbone (frozen core) |
| `mnemosyne.fuser` | FuserBridge sandbox ladder, DimensionSandbox |
| `mnemosyne.loom` | FuserLoom: port registry, mixing gates, OutputHead |
| `mnemosyne.ports` | ModalityPort + built-in encoders + `PORT_REGISTRY` |
| `mnemosyne.sensors` | deterministic synthetic window generators |
| `mnemosyne.train` | phase-1 warm-start training |
| `mnemosyne.quickswap` | `add_novel_port` / `quicklearn` / `quickremove` |
| `mnemosyne.eval` | reproducible eval harness, int8 quantization |
| `mnemosyne.gguf` | GGUF writer/reader |
| `mnemosyne.enclave` | session budget, governor, license ports |
| `mnemosyne.enclave_cli` | issue/install/status CLI |
| `c_loader/` | dependency-free C GGUF reader |