RTX 5090 AI Box on a GMKtec Proxmox Host: LXC, 160K Context, and 150 t/s Local Qwen

TL;DR: I wanted a 32 GB RTX 5090 for local models without turning my GMKtec mini PC into a different machine. The host already runs Proxmox and keeps an RTX 5060 Ti on OCuLink for VM passthrough. The answer was not Docker on the hypervisor and not another full VM: keep the 5090 driver on Proxmox, expose it to one trusted Ubuntu LXC, and build llama.cpp for Blackwell there. The result is Qwen 3.8 27B UD-Q6_K_XL at 150.5 tok/s on a short coding workload, 84.1 tok/s at a real 128K active context, and a completed interactive car configurator at 131.3 tok/s.

Update (August 21, 2026): Two follow-up experiments changed the tuning picture. First, the MTP depth curve had not peaked at 4: a controlled sweep on the same binary found depth 8 at 155.2 tok/s (+4.9%), with depth 10 declining and depth 12 collapsing to 139 tok/s as acceptance fall-off beats draft savings. Depth 8 also held at a ~21K active context (115.6 vs 113.7 tok/s), so the gain survives real workloads. Second, llama.cpp’s chained drafting changes the shape of the speed curve entirely: --spec-type ngram-mod,draft-mtp hits 290–380 tok/s on repetition-heavy code tasks (90% draft acceptance, 22-token mean accepted runs — up to 2.4x plain MTP once the n-gram pool is warm) but costs ~25% on novel content, where failed n-gram drafts waste verification batches. The practical result is two profiles instead of one: depth 8 everywhere as the default, and the chained profile switched in for refactor-and-boilerplate sessions. Both configs are in the server profiles snippet.

The hardware is a GMKtec mini PC running Proxmox. Its OCuLink-connected RTX 5060 Ti remains available for a conventional VM. The new GPU is a Gigabyte AORUS RTX 5090 AI Box connected over USB4/Thunderbolt. That combination is useful precisely because it is awkward: I wanted to retain the hypervisor, retain the 5060 Ti passthrough, and still make the 5090 a normal local inference endpoint.

This is the record of what actually mattered.

Tested stack, not a universal recipe: GMKtec K12 Pro, Proxmox VE 9.2, kernel 7.0.14-6-pve, AMD USB4 host, AORUS RTX 5090 AI Box, and NVIDIA open driver 610.57.04.

The Constraint Was Not CUDA. It Was Ownership.

An eGPU on a normal desktop is usually a driver-install problem. On this host it was an ownership problem with three competing requirements:

• Proxmox must continue to be the host OS.
• The 5060 Ti must stay bound to vfio-pci for its existing VM.
• The 5090 must be host-owned so a lightweight inference guest can use CUDA without a second virtual PCIe stack.

That rules out the tempting “pass every GPU to a VM” answer. It also rules out adopting a broad eGPU installer whose defaults may change IOMMU policy or claim every NVIDIA device.

The working split is deliberately boring:

Device	Owner	Job
RTX 5060 Ti on OCuLink	vfio-pci / existing Ubuntu VM	Existing GPU experiments
RTX 5090 AI Box on USB4	Proxmox host NVIDIA driver	CUDA provider for one LXC
Ubuntu LXC	Host /dev/nvidia* devices + matching libraries	llama.cpp and the model API

I had already tried the obvious alternative: assigning the 5090 directly to a VM through VFIO. Starting the guest froze the Proxmox host. The working design avoids that reset path: the host initializes and retains the kernel driver, then the LXC receives the CUDA device files. That made the setup smaller and more reliable than a second GPU VM on this hardware.

The USB4 eGPU Quirks

The AI Box is not a PCIe slot. It is a GPU behind a USB4/Thunderbolt tunnel, bridge chips, power management, and hot-plug discovery. The most useful lesson was to treat the connection as infrastructure, not as a cable that happens to carry a GPU.

The host boot configuration keeps IOMMU enabled for the 5060 Ti passthrough, disables the Thunderbolt host reset behaviour that can interfere with resource assignment, and reserves sufficient PCIe resources for the tunnelled GPU. The 5090 boot service then does four small, intentional things:

1. authorizes the Thunderbolt device;
2. waits up to 30 seconds for the 5090 PCI function to enumerate, because the first boot exposed a timing race;
3. pins the PCIe/USB4 path out of aggressive runtime power saving and caps the bridge at the stable link policy for this enclosure;
4. loads NVIDIA and nvidia_uvm, then starts persistence mode.

The stable link policy matters more than an impressive-looking PCIe generation number. USB4 is the bottleneck here; forcing an unstable higher bridge mode does not create a desktop PCIe x16 link. For LLM inference, the weights stay resident in VRAM, so a reliable link wins over a theoretical bus-speed number.

One confusing detail: after boot, some bridge status registers can report Gen1 even after a successful Gen3 retrain. On this machine, the boot-time retrain log is the useful signal; the later register readout is not a reason to retune a stable link.

I also treat the AI Box as non-hot-pluggable in day-to-day operation. It is connected and powered before boot; if I want to switch it off, I first stop the LXC, shut down Proxmox, and only then power down or disconnect the enclosure. With Qwen resident but idle, NVIDIA reports about 22 W of GPU-board power; enclosure and PSU overhead are additional.

I used NVIDIA’s open kernel-module package, version 610.57.04, on the Proxmox host. The important part was not the package manager command; it was ensuring that the module only binds the 5090 while the 5060 Ti’s PCI IDs remain reserved for VFIO. After a clean boot, the host could create a CUDA context and allocate GPU memory while the old VM configuration stayed intact.

This is hardware-specific work. Treat BDFs, bridge IDs, and boot arguments as examples to re-derive for your own host, not copy-paste magic.

Why an LXC Won Here

I considered three shapes:

• Run the model directly on Proxmox. Simple, but it puts model toolchains and experimental dependencies directly on the hypervisor.
• Create another GPU VM. Stronger isolation, but another full guest driver, more passthrough configuration, and more moving pieces for a single API service.
• Use a dedicated LXC. Host owns the fragile eGPU driver; the guest gets CUDA, the model cache, systemd, SSH, and a clean API endpoint.

For this homelab, the LXC was the cleanest boundary. It appears in the Proxmox UI like every other container, starts with the host, and I can SSH into it as a normal Ubuntu server.

There is an important security footnote: this is a privileged, trusted, single-purpose LXC. Mapping /dev/nvidia* into a container is not equivalent to VM isolation. I would not use this design to run arbitrary third-party workloads. For my own llama.cpp service, it is the right trade-off.

The container mounts the host’s NVIDIA device files and the exact matching CUDA/NVML userspace libraries. Matching matters: an Ubuntu nvidia-utils package from a slightly different driver build can make nvidia-smi fail even though the kernel module is healthy. Binding the host’s matching libraries read-only made the LXC see the same driver ABI as the host.

Inside the container, nvidia-smi and a direct CUDA allocation were the first two acceptance tests. Only then did I install the model server.

The Prebuilt Binary Was Good. Building for Blackwell Was Better.

The convenient llama installer got Qwen 3.8 running quickly. It was the right first smoke test. But this is an RTX 5090: a generic binary is not where I wanted to stop.

I built current llama.cpp in the LXC with CUDA 13.1 and an explicit Blackwell target. The useful ingredients were:

cmake -S llama.cpp -B build \
  -DCMAKE_BUILD_TYPE=Release \
  -DBUILD_SHARED_LIBS=OFF \
  -DGGML_CUDA=ON \
  -DCMAKE_CUDA_ARCHITECTURES=120 \
  -DGGML_CUDA_F16=ON \
  -DGGML_CUDA_FA_ALL_QUANTS=ON \
  -DLLAMA_OPENSSL=ON \
  -DGGML_NATIVE=ON

cmake --build build -j"$(nproc)" --target llama-server

The build system resolves 120 to Blackwell’s 120a target. The quantized Flash Attention option is deliberate: long-context Qwen sessions use the KV cache constantly, so a build that silently lacks a suitable fast path is not a long-context build.

The first coding benchmark used five warm runs on a short Python task, with one slot and thinking disabled. It is not a universal model score; it is a matched local comparison.

Server configuration	Median decode
Prebuilt binary, MTP n=3	85.2 tok/s
Source build for Blackwell, MTP n=3	125.5 tok/s
Source build for Blackwell, MTP n=4	150.5 tok/s

Depth 4 was 19.9% faster than depth 3 on this card and workload, so it became the default. I did not add a speculative confidence gate: the useful community finding for desktop Blackwell is that a higher acceptance rate is not automatically a higher token rate. Measure throughput, not the prettiest acceptance statistic.

Context: 160K Capacity Is Not 160K Usable

My earlier 5060 Ti work established a rule I did not want to forget: a server that starts at a given context length has proved almost nothing. The test that matters is a non-cached prompt at that length followed by generation.

The Q6 model is higher quality than the small IQ3 quant I used on the 16 GB card, so Q8 K/V cache left too little room for long sessions. Switching both caches to q4_0 was the context lever. The final long-context server has one parallel slot, MTP depth 4, batch 2048, micro-batch 1024, Flash Attention, and Q4 K/V cache.

Capacity setting	Active prompt validated	Prefill	Decode	VRAM after request
131K	96K	1,760 tok/s	92.7 tok/s	29.8 / 32.6 GiB
160K	128K	1,471 tok/s	84.1 tok/s	30.7 / 32.6 GiB
192K extreme mode	160K	1,265 tok/s	76.7 tok/s	31.7 / 32.6 GiB

The 192K server did start and the 160K request completed. That is a valid maximum experiment, not a daily recommendation: it leaves less than 1 GiB of VRAM free and triggered llama.cpp’s warning that its all-GPU layer request could not be automatically fitted. I restored the 160K capacity service afterwards.

My practical policy is therefore simple:

• 32K: fastest interactive coding; this is where the 150 tok/s result belongs.
• 128K: the real high-context OpenCode setting; measured, responsive, and still has operating margin.
• 160K active: possible for a deliberate one-off session through the 192K extreme profile, not the default service.

This is a better result than a capacity screenshot. It says exactly how much context I can actually put in front of the model and what it costs in speed.

The Car Test: Fast Is Only Helpful if the File Finishes

I repeated the spirit of the 5060 Ti car-configurator test: one self-contained HTML document, no external assets, a side-view SVG sports car, paint swatches, wheel options, working headlights, a pausable animated showroom, responsive styling, and click-to-spin wheels.

The first 4,096-token run was a useful failure: it generated at 138.1 tok/s but hit the output cap. More GPU did not repeal the earlier lesson about completion budgets.

I then asked for a compact implementation and allowed a 6,144-token output budget. The model returned a complete 12.6 KB HTML document in 39.4 seconds, stopped normally after 5,173 tokens, and decoded at 131.3 tok/s.

That is not a strict cross-card A/B: the 5060 Ti used a smaller quant, a different MTP depth, and a separately worded prompt. It is still the result I care about operationally. The 5090 is fast enough that a full one-shot interactive artifact takes well under a minute, provided the prompt gives the model a completion budget instead of an invitation to decorate forever.

The Server Configuration

This is the relevant inference portion of the systemd service. The API key is stored separately in a root-readable environment file; do not put it into the unit itself.

/opt/llama-sm120/bin/llama-server \
  -hf unsloth/Qwen3.8-27B-GGUF:UD-Q6_K_XL \
  -ngl 999 \
  --ctx-size 163840 \
  --cache-type-k q4_0 --cache-type-v q4_0 \
  --flash-attn on \
  --parallel 1 \
  --spec-type draft-mtp --spec-draft-n-max 4 \
  --batch-size 2048 --ubatch-size 1024 \
  --main-gpu 0 --jinja \
  --host 0.0.0.0 --port 8080

For ordinary coding, I send a request-level profile like this:

{
  "temperature": 0.7,
  "top_p": 0.8,
  "top_k": 20,
  "chat_template_kwargs": { "enable_thinking": false }
}

Thinking remains an explicit choice. Long reasoning can be valuable, but it consumes the same finite context that makes this machine useful for repository-sized work.

The Setup Order That Worked

The order mattered more than any single command: stabilize the host-owned eGPU first, prove CUDA allocation on Proxmox, then create the trusted LXC, and only after that install and tune llama.cpp. Keeping those checkpoints separate made it clear whether a failure belonged to USB4, the host driver, the container boundary, or the inference runtime.

Outlook: TurboQuant Is a Separate Experiment, Not a Free Upgrade

TurboQuant is the obvious next idea because it targets the same constrained resource: KV-cache memory. My Qwen 3.6 tests on the 5060 Ti showed that it could push capacity beyond the model’s usual configuration. They also showed why I will not casually replace this Q4-cache server with it: the turbo3 variants had more room but materially slower decode at heavy context in that earlier workload.

For this 5090, the order should be:

1. keep the measured Q4 K/V profile as the stable baseline;
2. build the TurboQuant branch separately, never over the working binary;
3. run the same 96K, 128K, and 160K active-context tests;
4. compare time-to-answer, VRAM margin, and quality—not just the server’s maximum context flag.

There may be a useful 192K-or-beyond profile there. Until it survives that matrix, “more context” is an experiment, not a feature.

The broader lesson is pleasantly unglamorous: the 5090 AI Box did not become useful because of one magical driver flag. It became useful after I gave each layer one job—Proxmox owns the hardware, the LXC owns the inference service, llama.cpp owns the model—and then measured the point at which context becomes real.