--- title: "Local coding on one NVIDIA L4, 24 GB" description: "Three measured coding configurations on a single NVIDIA L4: llama.cpp, SGLang, and vLLM, with their runs, memory, and pins." canonical: https://oloop.run/local/nvidia-l4 updated: 2026-10-05 evidence: /setups/evidence/linux-x64-l4@1.2.2.json --- # Local coding on one NVIDIA L4, 24 GB llamacpp/qwen3-8b on llamacpp is the setup the catalog recommends for this class, one of 3 measured configurations on a single L4. The probe took 136.4 s cold, 35.6 s warm, and 9.21 GB peak VRAM. The GPU was an L4; AWS g6 was not tested. ## Run it Install the CLI. Start this exact setup. Open the harness. The first pull creates a free handle. An explicit setup runs on the OS its catalog entry declares, after that entry's RAM, VRAM, and driver checks. The no-arg bootstrap is macOS on Apple silicon only. ```sh curl -fsSL https://oloop.run/install.sh | sh oloop host start --setup linux-x64-l4@1.2.2 oloop host open ``` - [Install and requirements](/docs#install) - [Setup compatibility](/docs#compatibility) - [The exact plan and lock](/docs#exact-plan) - [This setup in the catalog](/setups#linux-x64-l4-1.2.2) ## The tested setup The bundle pins llamacpp@b11292, opencode@1.18.21, and a 5 GB model download. The featured list names this setup for the class. That recommendation comes from the list, not from this run, and it makes no speed claim about the host. The exact pins and hardware. | Fact | Value | | --- | --- | | OS / arch | linux / x64 | | Memory kind | discrete | | Host | llamacpp@b11292 | | Harness | opencode@1.18.21 | | Model | llamacpp/qwen3-8b | | Model revision | unsloth/Qwen3-8B-GGUF@a6adef130ffb23ddaf1a62fec9dced968c9bc482 | | Quantization | q4_k_m | | Context window | 32,768 tokens | | Download | 5 GB | Source: /setups/evidence/linux-x64-l4@1.2.2.json (qualified 2026-10-05, engine 0.1.0). ## Measured results The timer measures one probe. The prompt asks for the word OK in answer.txt, and the run is capped at 3 turns. Cold is the first run, which starts the host and loads the model; warm repeats the same probe on the same machine. It is not coding throughput and not a tokens-per-second figure. Warm was 100.8 s faster than cold here. One probe pair gives a direction, not a rate. The qualification run on the tested machine. | Run | Cold | Warm | Peak RAM | Peak VRAM | | --- | --- | --- | --- | --- | | Probe | 136.4 s | 35.6 s | 2.49 GB | 9.21 GB | Measured on NVIDIA L4, 15 GB system RAM (Linux/x64) on 2026-10-05. Source: /setups/evidence/linux-x64-l4@1.2.2.json. ## Tested on, listed for Tested on: NVIDIA L4, 15 GB system RAM (Linux/x64). Listed for: Linux x64, 16–64 GB RAM, NVIDIA ≥ 24 GB VRAM. The catalog lists a range, and the record covers one machine in it. The times and memory numbers belong to that machine. ## The alternatives on this class llamacpp leads both columns here: 9.21 GB peak VRAM and 35.6 s warm. Each host is a separate pinned bundle with its own evidence document. Read the row for the host you plan to run. 3 measured configurations on one L4: 2 models across 3 hosts, not a controlled host benchmark. | Host | Model | Quantization | Cold | Warm | Peak VRAM | Peak RAM | | --- | --- | --- | --- | --- | --- | --- | | llamacpp | qwen3-8b | q4_k_m | 136.4 s | 35.6 s | 9.21 GB | 2.49 GB | | sglang | qwen2.5-7b-awq | awq | 290.6 s | 85.8 s | 19.49 GB | 7.19 GB | | vllm | qwen2.5-7b-awq | awq | 90.3 s | 65.9 s | 18.12 GB | 4.49 GB | Source: /setups/evidence/linux-x64-l4@1.2.2.json, /setups/evidence/linux-x64-l4-sglang@1.1.2.json, /setups/evidence/linux-x64-l4-vllm@1.1.2.json. Measured on one L4 with 15 GB of system RAM. ## What the run proves One pinned setup passed one probe on the stated hardware and date, using the checksummed bundle from the catalog. - The run is not a security review. It checks the named file; it does not audit the model, the harness, or their permissions. - A pass is not a quality score. It means the configured check passed, not that every answer is correct. - The model license belongs to the model publisher. Read it before you ship the output. - The bundle is pinned to one version. A newer catalog version is a different record. - We did not test another vendor's 24 GB card. The record is one card, not a family of them. - [What the checks mean](/learn#checks) - [All machine setups](/setups) ## Questions ### Was this tested on AWS? No. The runs were on one NVIDIA L4 with 15 GB of system RAM. AWS g6 uses an L4, and we have not run the qualification there. ### Is this a benchmark of llama.cpp, SGLang, and vLLM? No. The table is 3 measured configurations with 2 models on one GPU. A host benchmark holds the model fixed; this record does not. ### Why do the rows use different models? Each setup pins a host and a model together. SGLang and vLLM were measured with qwen2.5-7b in AWQ, and llama.cpp with qwen3-8b in Q4_K_M. The catalog has no row where only the host changes. ### Which configuration should I start with? Start with linux-x64-l4. The other rows show what a different host and model did on the same GPU. ### Does this apply to another 24 GB GPU? We measured an L4. We do not carry the times or the peak memory to another card, even one with the same VRAM size. ## Related - [Local coding on Apple silicon, 64–128 GB](/local/apple-silicon-64-128) - [Local coding on Linux with 8–32 GB, no GPU](/local/linux-cpu-8-32) - [Local coding model by memory size](/guides/local-coding-model-by-memory) - [Self-hosted coding agent for a team](/guides/self-hosted-coding-agent) ## Evidence Tested on NVIDIA L4, 15 GB system RAM (Linux/x64). Listed for Linux x64, 16–64 GB RAM, NVIDIA ≥ 24 GB VRAM. - [The qualification record](/setups/evidence/linux-x64-l4@1.2.2.json) - [linux-x64-l4@1.2.2](/setups#linux-x64-l4-1.2.2) Updated 2026-10-05 · Outerloop