7 min read AI

One DGX Spark, a Mac Mini, and a Local Jev in Front of the GPU

One DGX Spark, a Mac Mini, and a Local Jev in Front of the GPU

I did not want another chatbot tab.

I wanted a small private stack: a real GPU for tokens, an always-on box for agents, and a laptop that can reach both from the couch without opening the LAN to the world.

That is three machines, not one. People keep trying to collapse it. Put the agents on the GPU box. Put the model on the laptop. Punch a hole in the router so the phone can hit vLLM. It works until it does not, and then you have a science project instead of a house.

So I split it the boring way.

The GPU writes tokens. The Mini runs the work. A local System One hop sits in front of the GPU so AGNT — and the MacBook over Tailscale — get a judge and a classified chat path without pretending that hop is official @typesafeai Jev.

Spark writes. It does not manage.

The GPU is a single @nvidia DGX Spark. GB10, 128 GB unified memory, the small gold slab that is supposed to make “a 200B model in the study” a normal sentence.

I did not invent a serving stack for it. I followed a @MiaAI_lab recipe for one box — DeepSeek-v4-Flash-One-DGX-Spark — so the OpenAI-compatible endpoint comes up without me arguing with GB10 flags. That recipe is EXL3 + a vLLM / sparkinfer path, tuned for one Spark. I switch the live slot between Qwen, DeepSeek, and Ling. Today DeepSeek is on :8001.

Spark does inference only. One live model at a time.

That is the whole job description. It does not run agents. It does not hold Telegram. It does not keep my calendar. It does not get a personality. It is the writer in the other room.

If you put the control plane on the GPU box you will eventually want a second model, a second agent, a restart, and a reason the whole house went quiet. Keep the expensive box dumb and hot. Let something cheaper stay awake.

The Mini is the desk

The house computer is an @Apple Mac Mini. It stays on.

That is where @agnt_gg AGNT lives — the control plane. Web UI. Telegram. Agents. Workflows. Coding CLIs sit there too. The Mini is the CEO’s desk. Spark is hired help with a very good pen.

AGNT already knew how to talk to Spark (/v1 on the live slot). Cloud rows were already there: Grok, GPT, whatever I still pay for. I did not want a fourth religion. I wanted one extra hop that could classify a turn before it burned GPU tokens, and a local POST /v1/systemone that did not leave the house.

That hop is localjev-spark. I wrote it because the official thing and the local thing are not the same object, and I was tired of explaining that to myself every time I opened Settings.

Official Jev stays official

Hosted Jev from @typesafeai is the decision model. You send state plus typed questions — Choice, Score, Noul — to POST /v1/systemone and you get probabilities back. Usually through jev-mcp. I keep that. I already wrote the other half of this: Jev is not a chatbot, and that is the point.

Jev is not a chat model. I did not replace it.

What I still wanted was a local System One judge next to the Spark hop. Same request shape. Different weights. Different trust story.

The local judge is Laya (convaiinnovations/laya). Small encoder-style decision model. One forward pass. No essay. It is a clone of the wire, not a clone of TypeSafe’s training run, not a clone of their calibration, not a drop-in for jev-mcp.

Use hosted Jev when the answer has to be the one you would defend. Use the local hop when you want an on-box classify-and-forward and you can live with “this is Laya, not Jev.”

If you collapse those two sentences you will ship a confident wrong fork and call it architecture.

What localjev-spark actually is

Two ports on the Mini. One process.

Port Job
:8090 Local System One (Laya). GET /health, POST /v1/systemone. Not a chat provider.
:8091 OpenAI chat proxy. Classifies the last user message, logs it, forwards the same request to whichever Spark port is already up.

Spark still writes every token.

If Laya cannot classify, chat still goes through. The proxy marks that with x-localjev-status: bypass. A dead judge is not an outage. A dead process is: if localjev-spark is down, both ports are dead, and you point AGNT at Spark directly.

The proxy does not pick a “best” model. Spark is one live slot. It probes :8000 / :8001 / :8002 on spark-local, then spark, then localhost, and uses the first /v1/models that answers. House convention only:

Spark port What I usually park there
:8000 Qwen
:8001 DeepSeek
:8002 Ling

Pin it if you hate surprises: LOCALJEV_SPARK_UPSTREAM=http://spark-local:8001/v1.

Bind is 0.0.0.0. There is no extra auth. Tailscale is the door. If that sentence makes you itchy, set LOCALJEV_BIND=127.0.0.1 and stop pretending this is a product for the open internet.

One extra row in AGNT

On the Mini, AGNT gets one extra provider. I do not rewrite the rest of the picker. Existing Spark / Grok / GPT rows stay.

Base URL   http://127.0.0.1:8091/v1
API key    not-needed
Model id   spark-via-localjev
Protocol   Chat Completions

Path for a normal chat turn:

AGNT → Mini :8091 → localjev-spark → Spark (today DeepSeek on :8001).

Path for a local judgment:

something you own → Mini :8090 → Laya.

:8090 is not a chat base URL. If you paste it into a Chat Completions field you will have a bad afternoon and it will be your fault.

The installer does not write AGNT, Grok, or Codex config. You add the row in the UI. That is deliberate. A script that edits three agent apps is how you get a weekend of “why did my default model change.”

The laptop is a remote, not a second GPU

The MacBook is @Tailscale, not another accelerator. MagicDNS name ai-stack is the Mini. Same two ports, from the sofa:

  • Judge: http://ai-stack:8090
  • Chat via localjev: http://ai-stack:8091/v1

The Mini binds 0.0.0.0. The proxy reaches Spark on the Mini’s LAN as spark-local. The laptop does not need that hostname. If I want to skip the judge I can still hit Spark itself as http://spark:8001/v1 over the tailnet.

No port-forward on the router. No “just for a minute” nginx. No second copy of the weights on a machine that goes through airport security.

This is the same split I already argued for agents: the lid closes, the work should not. The Mini stays. The Spark stays. The laptop is a glass window.

What this is not

A short list, because this genre of post usually lies here.

  • It is not official Jev. No TypeSafe weights. No claim that Laya’s 0.91 means what Jev’s 0.91 means. Measure it on your tickets or do not put a threshold on it.
  • It is not agents on the Spark. If your AGNT workflow can rm, it can rm on the Mini. That is still a choice. It is a better choice than giving the GPU box a Telegram bot and a filesystem.
  • It is not multi-model serving. One live Spark slot. Switching Qwen / DeepSeek / Ling means switching what is loaded, not a tiny load balancer with feelings.
  • It is not a Mia Lab fork. The recipe brings the tokens up. localjev-spark only sits in front of whatever /v1 is already healthy.
  • It is not security theater that survived contact with a real attacker. Tailscale identity is the gate. The process itself will happily talk to anyone who can reach :8091.

The honest version of “local Jev” on the public internet right now is a family of adapters. GitHub Next has a LocalJev that prompts a chat model for JSON probabilities. Other people put DiffusionGemma on the Spark and read a decision head. Mine is Laya on the Mini plus a chat hop onto Spark. Different honesty levels. Same temptation to stamp the word Jev on the folder and walk away.

I stamped the folder localjev-spark on purpose. The first five letters are a warning.

Install is one script. Wiring is still a human.

Clone and run, or paste a prompt into Grok / Codex / Cursor / Claude and let it follow the skill:

git clone https://github.com/rimusz/localjev-spark.git
cd localjev-spark
./scripts/install.sh

or:

curl -fsSL https://raw.githubusercontent.com/rimusz/localjev-spark/main/scripts/install.sh | bash

That puts a venv in ~/.localjev-spark and a user service (launchd on macOS, systemd --user on Linux). First boot pulls Laya and waits until :8090 /health is HTTP 200 and :8091 is up. It warns if Spark is not found yet. That warning is correct. The proxy can exist before the writer shows up.

~/.localjev-spark/.venv/bin/localjev-spark status

Then add the one row in AGNT. Restart the coding apps so the copied skills load. Do not point the whole of Codex at :8091 if you already have a gateway that speaks Responses — add one custom provider, leave the rest alone.

Config lives in ~/.config/localjev-spark/localjev.env. Logs in ~/logs/localjev-spark.log. Uninstall does not touch TypeSafe jev-mcp.

Why bother

Because the default 2026 move is to buy a loud box and then run your life on it.

Spark is good at one thing: sitting on the desk and emitting tokens from a model that does not fit on a laptop. The Mini is good at a different thing: staying on, talking to humans, holding workflows, not melting when you ask it to keep a Telegram session and a cron. Jev — the real one — is good at a third thing: returning a typed fork instead of a paragraph.

localjev-spark is glue. Glue is allowed to be unglamorous. The point of glue is that you can replace one side without setting fire to the other two.

If Laya is wrong on your workload, you still have hosted Jev and you still have Spark. If Spark is down, AGNT still has Grok. If I take the laptop to a train, the Mini is still the house.

Three machines. Two ports. One live GPU model. A judge that is allowed to fail open.

That is the whole story.

Repos: Mia one-box Spark · AGNT · localjev-spark · TypeSafe

Rimantas Mocevicius

Rimantas Mocevicius