Thinking in Micro-Weights: Why We Built Mara, a 692k Parameter Atomic Function Model

1. The Autoregressive Tax vs. First Principles

When you want to calculate the trajectory of a basketball falling to the ground, you do not invoke general relativity and solve the Einstein field equations over curved spacetime. You use F = ma. You pick the simplest mathematical formulation that accurately captures the physical regime.

In machine learning, software engineers have accidentally conflated language understanding with token-by-token generative sampling.

Consider what tool calling actually is. Given a user utterance x and a set of candidate tools T = [t₁, t₂, ..., t_k]:

• It is a routing classification problem: select tool t* ∈ T ∪ {∅}.
• It is a slot filling problem: extract categorical choices (room ∈ {living room, kitchen, ...}) and bounded scalar values (temperature ∈ [15, 30]).

Neither of these operations requires autoregressive loop generation. Autoregression has \(O(N)\) sequential memory bandwidth bottlenecks—each token requires a full read of every parameter weight from RAM into the CPU cache. If you emit 100 tokens, you must stream the entire model through cache 100 consecutive times.

If, instead, you formulate tool calling as a single-pass discriminative representation, you pass the weights through cache exactly once.

2. What is an Atomic Function Model (AFM)?

We set out to build the absolute minimal architecture that could reliably execute smart home tool calling with zero hallucination. We call this family an Atomic Function Model (AFM).

Here are the exact design constraints we chose:

The architecture is surprisingly pure. It consists of:

1. Subword BPE Tokenizer: 512-entry compact vocabulary tuned for natural voice instructions.
2. 2-Layer Transformer Encoder: Hidden dimension d_model = 128, 4 attention heads (d_k = 32), feed-forward dimension d_ff = 512.
3. Classification Routing Head: Linear projection with calibrated temperature for tool confidence scoring P(tool | x).
4. Direct Slot Decoders: Point-wise softmax distributions over bounded categorical slots (room, device, door, action).

Because the slot heads project directly onto valid enumerated candidates, it is mathematically impossible for Mara to emit malformed JSON or fabricate a non-existent parameter name. Syntax errors are eliminated by construction.

3. The Honest Head-to-Head: Mara vs. Needle 3

In engineering, if you don't audit yourself ruthlessly, reality will audit you in production.

In our initial experiments, we pitted Mara against Cactus Needle 3, and Mara looked 100× superior in every chart. But a rigorous technical review called us out on three real flaws:

1. Needle was tested in a stateful loop where conversational history polluted subsequent turns.
2. The tool schemas had overlapping docstrings (`control_device` vs `set_lights`).
3. We were comparing a domain-specialized Mara against Needle Base without fine-tuning.

So we tore down the old benchmark and rebuilt it properly. We gave Needle fair, disjoint `typing.Literal` definitions, fine-tuned a Needle 3 Specialist using rank-16 LoRA on the exact same 1,650 domain training prompts, and evaluated both on a frozen 250-sample test suite with zero n-gram collisions.

Here are the verbatim results from our isolated whole-process benchmark:

The Real Conclusion

Notice something very important in those numbers: Mara is not a magic bullet on colloquial phrasing. On independently authored paraphrases featuring heavy slang (“nuke the salon spotlights”, “kill the chill in the master”), Mara’s accuracy drops to 44.0%. A general-purpose 70B LLM with extensive pretraining can parse slang easily because it has seen the entire internet.

However, on the operational axis, Mara operates in a completely different universe:

• It is 2,143× faster than the LoRA fine-tuned specialist.
• Its ONNX runtime footprint is 153 kilobytes—smaller than the favicon on most web pages.
• It executes deterministically in steady-state memory with zero allocation spikes.

4. Running 100% Client-Side in the Browser (WebAssembly)

One of the most liberating consequences of building micro-models is that you do not need a backend server.

In our live web playground, when you type an instruction or press an example chip:

1. The browser streams mara.onnx directly from our Hugging Face model repository (jaswanthsanjay88/mara) via the Fetch API and ReadableStream.
2. The binary is cached in the browser's persistent CacheStorage. Repeated page reloads load from disk in under 20 milliseconds with zero network requests.
3. Inference runs inside the user's browser tab using onnxruntime-web compiled to WebAssembly.
4. The complete inference forward pass executes in 2 to 4 milliseconds on a single thread.

No cloud compute bills. No GPUs spinning in a datacenter. No telemetry or privacy concerns, because the user's audio and text never leave their browser sandbox.

5. The PyTorch Implementation in 40 Lines

Here is the conceptual core of how the AFM forward pass is implemented in pure PyTorch:

import torch
import torch.nn as nn

class MaraAFM(nn.Module):
    def __init__(self, vocab_size=512, d_model=128, nhead=4, num_tools=4):
        super().__init__()
        self.embedding = nn.Embedding(vocab_size, d_model)
        self.pos_encoder = nn.Parameter(torch.zeros(1, 64, d_model))
        
        encoder_layer = nn.TransformerEncoderLayer(
            d_model=d_model, nhead=nhead, dim_feedforward=512,
            batch_first=True, norm_first=True
        )
        self.transformer = nn.TransformerEncoder(encoder_layer, num_layers=2)
        
        # Classification head for tool selection
        self.tool_head = nn.Linear(d_model, num_tools + 1)
        
        # Explicit slot decoding heads (bounded categorical spaces)
        self.room_head = nn.Linear(d_model, 5)        # living, kitchen, bed, bath, garage
        self.action_head = nn.Linear(d_model, 4)      # on, off, open, close
        self.temp_head = nn.Linear(d_model, 1)        # continuous regression scalar
        
    def forward(self, input_ids):
        # input_ids: [batch_size, seq_len]
        x = self.embedding(input_ids) + self.pos_encoder[:, :input_ids.size(1), :]
        features = self.transformer(x)
        
        # Mean pool over sequence length for global representation
        pooled = features.mean(dim=1)
        
        tool_logits = self.tool_head(pooled)
        room_logits = self.room_head(pooled)
        action_logits = self.action_head(pooled)
        temp_val = self.temp_head(pooled)
        
        return tool_logits, room_logits, action_logits, temp_val

6. What's Next & The Road Ahead

Small models are not dead. In fact, for edge devices, automobiles, IoT hubs, and microcontrollers, micro-transformers are the only path to deterministic, zero-latency software.

Our roadmap for Mara AFM includes:

• Learned Clause Segmentation: Replacing our rule-based planner with a 40k parameter sub-network to parse conjunctions like “before you lock up, kill the fan” in true execution order.
• INT8 & W4 Quantization: Compressing the 153 KB graph down to 42 KB to execute comfortably on ARM Cortex-M microcontrollers.
• Streaming Acoustic AFM: Connecting the encoder directly to raw audio filterbank features, bypassing text tokenization altogether for true end-to-end voice-to-device actuation.