We sent ScallopBot to school, and its 4B brain beat a giant

We think we are entering a weird new phase of AI, and it started with a weekend project.

ScallopBot, our self hosted assistant, was leaning on a 400B+ parameter hosted model for two jobs: deciding which tool to call, and deciding what to remember. That works, but it is slow, it is not local, and it felt like hiring a professor to sort the post.

So over a weekend we fine tuned Qwen3.5-4B with LoRA, twice, once per job. The training data came from ScallopBot's own production traces, anonymized and deduplicated, with labels written by a larger teacher model. The student never trained on its own generations. Rank 32, alpha 64, two epochs, nothing exotic.

We expected the small models to be cheaper but worse, and we would have settled for a 2x local speedup. That is not what happened. The chart above is the honest version, measured on held-out turns the models never saw.

For tool selection, scalloptools-1 hit 73.3 percent against 65.3 for the stock 4B, and Qwen3.6-35B managed only 46.5 percent on the same turns. The big hosted model it replaced lost too. Let that sink in: a model roughly 100x smaller made better decisions about which tools to use, about 4x faster, and it runs on 16 GB of RAM. It also parsed cleanly on 100 percent of turns and fabricated exactly zero tool calls.

Memory is a different story, and an honest chart shows it. scallopmemory-1 reads a conversation and writes down durable facts worth remembering, or nothing when a turn is just chatter. It scored 0.725 teacher agreement against 0.695 for stock, close to but not past its 0.748 teacher. That makes sense to us. Deciding what to remember, what to forget and when to update requires reasoning, and reasoning is where scale still earns its keep. The consolation prize: the 4B answers in 4.2 seconds where the teacher takes 31.

Two practitioner details that mattered. First, thinking is disabled at serving time, because chain of thought actively degrades performance at this scale. Second, capping empty examples per session during training proved critical, otherwise the memory model learns that silence is always safe.

Both specialists are on Hugging Face under Apache 2.0, 3.16 GB each as q5_k_m GGUF, and they run in llama.cpp, Ollama and LM Studio. Links and setup live on the ScallopBot homepage.

Maybe the future of personal agents is not bigger models. Maybe we send our agents to school, and they get better over time at the jobs that are actually theirs.