Collections

The Collections subsystem provides replicated, eventually consistent data structures that feel like local Rust collections but are automatically synchronized across all participating nodes using Raft consensus.

┌────────────────────────────────────────────────────────────────────────┐
│                   Collections                                          │
│                                                                        │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌────────┐ ┌──────────┐ ┌──────┐ │
│  │   Map    │ │   Vec    │ │   Set    │ │  DEPQ  │ │ Register │ │ Once │ │
│  │ <K, V>   │ │ <T>      │ │ <T>      │ │<P,K,V> │ │  <T>     │ │ <T>  │ │
│  └────┬─────┘ └────┬─────┘ └────┬─────┘ └───┬────┘ └────┬─────┘ └──┬───┘ │
│       └────────────┴────────────┴─┬─────────┴───────────┴──────────┘     │
│                                   │                                     │
│                                   │                                    │
│                                   │                                    │
│                  ┌────────────────▼───────────────────┐                │
│                  │    Groups (Raft Consensus)         │                │
│                  │    One group per collection        │                │
│                  └────────────────────────────────────┘                │
└────────────────────────────────────────────────────────────────────────┘

Each collection instance creates its own Raft consensus group. Different collections (or the same type with different StoreIds) run as independent groups and can span different subsets of the network.

Quick start

use mosaik::collections::{Map, StoreId};

// On every node — using the same StoreId joins the same group
let prices = Map::<String, f64>::writer(&network, StoreId::from("prices"));

// Wait until the collection is online
prices.when().online().await;

// Write (only available on writers)
let version = prices.insert("ETH".into(), 3812.50).await?;

// Wait for the write to be committed
prices.when().reaches(version).await;

// Read (available on both writers and readers)
assert_eq!(prices.get(&"ETH".into()), Some(3812.50));

Writer / Reader split

Every collection type offers two modes, distinguished at the type level using a const-generic boolean:

Both modes provide identical read access. Readers automatically use deprioritize_leadership() in their consensus configuration to reduce the chance of being elected leader, since leaders handle write forwarding.

// Writer — can read AND write
let w = Map::<String, u64>::writer(&network, store_id);

// Reader — can only read, lower chance of becoming leader
let r = Map::<String, u64>::reader(&network, store_id);

Available collections

Most collections use the im crate for their internal state, which provides O(1) structural sharing — cloning a snapshot of the entire collection is essentially free. This is critical for the snapshot-based state sync mechanism. Register and Once use a plain Option<T> since they hold at most one value.

Trait requirements

Collections use blanket-implemented marker traits to constrain their type parameters:

These traits are automatically implemented for any type that satisfies their bounds — you never need to implement them manually.

StoreId and group identity

Each collection derives its Raft group identity from:

1. A fixed prefix per collection type (e.g., "mosaik_collections_map", "mosaik_collections_once")
2. The StoreId — a UniqueId you provide at construction time
3. The Rust type names of the type parameters

This means Map::<String, u64>::writer(&net, id) and Map::<u32, u64>::writer(&net, id) will join different groups even with the same StoreId, because the key type differs. Likewise, a Once<String> and a Register<String> with the same StoreId form separate groups because the prefix differs.

Version tracking

All mutating operations return a Version, which wraps the Raft log Index at which the mutation will be committed:

let version: Version = map.insert("key".into(), 42).await?;

// Wait until the mutation is committed and visible
map.when().reaches(version).await;

Error handling

All write operations return Result<Version, Error<T>>:

SyncConfig

Collections use a snapshot-based state sync mechanism to bring lagging followers up to date. The SyncConfig controls this behavior:

Important: Different SyncConfig values produce different group signatures. Collections using different configs will not see each other.

use mosaik::collections::{Map, StoreId, SyncConfig};
use std::time::Duration;

let config = SyncConfig::default()
    .with_fetch_batch_size(5000)
    .with_snapshot_ttl(Duration::from_secs(30));

let map = Map::<String, u64>::writer_with_config(&network, store_id, config);

How snapshot sync works

1. A lagging follower sends a RequestSnapshot to the leader.
2. The leader wraps the request as a special command and replicates it.
3. When committed, all peers create a snapshot at the same log position.
4. The follower fetches snapshot data in batches from available peers.
5. Once complete, the follower installs the snapshot and replays any buffered commands received during sync.

Because im data structures support O(1) cloning, creating a snapshot is nearly instant regardless of collection size.

Deterministic hashing

Map and Set use BuildHasherDefault<DefaultHasher> (SipHash-1-3 with a fixed zero seed) for their internal im::HashMap / im::HashSet. This ensures that iteration order is identical across all nodes for the same logical state — a requirement for snapshot sync to produce consistent chunked transfers.