FOOTPRINT

Running the cohort on a laptop

CPU only, CUDA_VISIBLE_DEVICES empty, 8 pinned threads. Decoders via GGUF Q4_K_M under llama.cpp, encoders via dynamic int8 under transformers. 14 arms were converted or quantized and 13 have a published rows/min figure.

Memory

Nothing in the cohort reaches 24 GiB, and nothing reaches 8 GiB. The hungriest arm is Phi-4-mini at 3.137 GiB irreducible peak RSS, 5.048 GiB worst-observed under mmap, which leaves 4.863 GiB spare on an 8 GiB machine. Q4_K_M files run 0.871 GiB (OLMo-2-1B) to 2.323 GiB (Phi-4-mini).

The cohort's memory envelope against an 8 GiB and a 24 GiB machine

Nothing in the cohort reaches 8 GiB. The hungriest arm leaves 4.863 GiB spare on an 8 GiB machine.

Table view (every plotted value)

Quantity	Arm	GiB
Q4_K_M file, smallest	OLMo-2-1B	0.871
Q4_K_M file, largest	Phi-4-mini	2.323
Peak RSS, irreducible	Phi-4-mini	3.137
Peak RSS, worst observed under mmap	Phi-4-mini	5.048

Peak RSS came from the kernel's VmHWM high-water mark polled every 5 ms and cross-checked against getrusage. It already includes the weights, so it is not additive with the Q4_K_M file size. Every value plotted here is read at build time from pinned/laptop-feasibility.json. The table view lists them all.

Throughput

At 8 threads the fastest arm, OLMo-2-1B, does 30.38 rows/min and the slowest, granite-4.0-micro, does 9.61. A 3,000-row pass takes 1h39m at the top of that range and 5h12m at the bottom.

Rows per minute on 8 pinned CPU threads

Latency model row_latency = 1200/pp1200 + 1/tg32, from llama-bench -t 8 -p 1200 -n 32 -r 3. A 3,000-row pass takes 1h39m on OLMo-2-1B and 5h12m on granite-4.0-micro. DeBERTa-v3-base reaches its figure only at 512 tokens, its architectural maximum.

Decoder, GGUF Q4_K_M under llama.cppEncoder, dynamic int8

Table view (every plotted value)

Arm	Serving	Rows/min	Context tokens	3,000-row pass
OLMo-2-1B	decoder, Q4_K_M	30.38	1,200	1h39m
Falcon3-1B	decoder, Q4_K_M	27.67	1,200	1h48m
SmolLM2-1.7B	decoder, Q4_K_M	19.35	1,200	2h35m
granite-4.0-1b	decoder, Q4_K_M	18.83	1,200	2h39m
Falcon3-3B	decoder, Q4_K_M	13.68	1,200	3h39m
granite-guardian-3.1-2b	decoder, Q4_K_M	12.45	1,200	4h01m
granite-guardian-3.2-3b-a800m	decoder, Q4_K_M	11.98	1,200	4h10m
SmolLM3-3B	decoder, Q4_K_M	11.40	1,200	4h23m
Phi-4-mini	decoder, Q4_K_M	10.08	1,200	4h58m
Shieldstral (text tower)	decoder, Q4_K_M	9.91	1,200	5h03m
granite-4.0-micro	decoder, Q4_K_M	9.61	1,200	5h12m
DeBERTa-v3-base	encoder, dynamic int8	186.70	512	0h16m
ModernBERT-base	encoder, dynamic int8	44.60	1,200	1h07m

The host carried foreign load throughout and a real laptop also thermally throttles, so every figure here is an optimistic ceiling. Every value plotted here is read at build time from pinned/laptop-feasibility.json. The table view lists them all.

The encoders are roughly six times faster than the fastest decoder. DeBERTa-v3-base reaches 186.7 rows/min under int8 at 512 tokens, its architectural maximum, so a longer prompt is truncated; the extent on this corpus is on Results. ModernBERT-base does 44.6 rows/min at the full 1,200-token budget.

Caveats on the figures above

peak RSS came from the kernel's VmHWM high-water mark polled every 5 ms and cross-checked against getrusage, because the host carries no /usr/bin/time. The figure already includes the weights, so it is not additive with the Q4_K_M file size.
the host carried about 31 of its 96 cores of foreign load throughout, so there is memory-bandwidth contention that could not be removed. A real laptop also thermally throttles. Every throughput figure here is an optimistic ceiling.

The two multimodal arms

Two arms ship a vision tower that a text-only task never uses. Both splits were measured from the checkpoints and the converter output.

Where the parameters sit in the two multimodal arms

Shieldstral is Mistral3ForConditionalGeneration. llama.cpp's converter emits the text tower only for mistral3: 236 tensors, exactly 3,429,006,336 elements, zero vision tensors. The vision side exports separately as an mmproj. gemma-3-4b-it's full checkpoint is 8.0096 GiB, so the split is what makes it loadable on an 8 GiB machine.

Text towerVision towerProjector

Table view (every plotted value)

Arm	Architecture	Text tower	Vision	Projector	Total	Vision plus projector share
Shieldstral-1.0-3B	Mistral3ForConditionalGeneration	3,429,006,336	403,305,472	16,778,240	3,849,090,048	10.91%
gemma-3-4b-it	Gemma3ForConditionalGeneration	3,880,263,168	416,866,032	2,950,272	4,300,079,472	9.76%

Artifact	Bytes	GiB
Shieldstral, text-only Q4_K_M on disk	2,146,497,312	1.9991
Shieldstral, mmproj that is never built	840,284,704	0.7826
gemma-3-4b-it, full checkpoint	8,600,277,880	8.0096

Every value plotted here is read at build time from pinned/laptop-feasibility.json. The table view lists them all.

Shieldstral-1.0-3B

Shieldstral is Mistral3ForConditionalGeneration. Its vision tower is 403,305,472 parameters and its projector 16,778,240, so 420,083,712 of its parameters (10.91%) are dead weight for this task. llama.cpp's converter emits only the text tower for mistral3: 236 tensors, exactly 3,429,006,336 elements, matching language_model to the parameter, and zero vision tensors. The text-only Q4_K_M is 2,146,497,312 bytes on disk, 1.9991 GiB. The vision side exports separately as a 840,284,704-byte mmproj that is never built. Its laptop constraint is throughput, at 9.61 rows/min.

gemma-3-4b-it

Its vision tower is 416,866,032 parameters and its projector 2,950,272, so 419,816,304 parameters (9.76%) are dead weight, over a text tower of 3,880,263,168. Its weight bytes total 8,600,277,880, 8.0096 GiB, which is the only arm in the cohort past 8 GiB, so for this arm the split is what makes it loadable on an 8 GiB machine.

Serving settings that carried over

For a scoring-only workload that never generates a token:

logits_to_keep=1. Without it a batch materialises [batch, seq, vocab] logits and does not fit.
use_cache=False. A KV cache wastes about 31 GB on a 3B model here.
The token budget binds before the batch count does. max_batch 32 against 128 gave 291.8 against 271.6 rows/min, and batches average 28 against a maximum of 32.
Encoder classifiers run about eight times faster than generative arms on GPU, 3,059 against roughly 1,190 rows/min aggregate, because there is no generation step.

Reproducibility of the figures on this page

harness/build_llamacpp.sh builds the CPU benchmarking toolchain, and no script in the harness runs it or writes the CPU results. The measurements were ad-hoc invocations that were never saved to a file. Their logs survive on the studio and the measurement step does not, so the figures on this page are archived and are not reproducible from vendored code. A search of the studio for any script referencing rss_results, max_rss, ru_maxrss, llama-bench or convert_hf_to_gguf found nothing outside llama.cpp itself and the virtual environments.

harness/score_arm.py measures cuda_alloc_gib, elapsed_seconds, rows_per_min and batch duration_ms. It measures no RSS and no CPU latency. Everything on this page came from outside it.

The primitives are staged in pinned/laptop-feasibility.json. Every derived figure on this page is recomputed from them at build time: both 3,000-row pass times, both multimodal parameter shares and every GiB conversion.

Source: benchmarks/EXPERIMENTS.md and benchmarks/slm_toolcall/harness/README.md, staged into pinned/laptop-feasibility.json. Linked from Reproduce.