There is a specific kind of friction that has followed local model experimentation for the past two years: the model runs fine, inference is fast enough, but the tooling around it lags. You can pull a model and get a chat interface, but the moment you want something more agentic, something that edits files and runs commands and maintains context across a session, you end up back at a cloud API. That friction is mostly gone now.
George Liu’s walkthrough of running Gemma 4 locally with LM Studio’s headless CLI and Claude Code is a good illustration of where the local AI stack has arrived. Each of the three components in that setup has matured independently, and the combination works because they’ve each solved a different layer of the problem.
LM Studio Gets a Real CLI
LM Studio has been the most polished GUI for local model management on desktop, but GUI-only tools have a hard ceiling in developer workflows. You cannot script them, cannot run them on a remote machine over SSH, and cannot automate model loading as part of a larger pipeline.
The lms CLI changes that. Starting a local inference server is now a two-command operation:
lms server start
lms load google/gemma-4-27b-itFrom there, LM Studio exposes an OpenAI-compatible REST API on http://localhost:1234/v1, including the /v1/chat/completions endpoint that virtually every AI client library knows how to talk to. You can verify the server is running and see what is loaded:
lms ps
lms statusThis is not a new idea. Ollama has offered headless operation and an OpenAI-compatible API since its early releases, and llama.cpp has had a server mode for longer than either. What LM Studio brings is the model management layer: downloading models from Hugging Face, handling quantization selection, managing GPU memory across multiple loaded models, and the GGUF format support that makes large models tractable on consumer hardware. The lms CLI exposes all of that without requiring the GUI to be open.
For anyone running these setups on a workstation accessed remotely, or trying to integrate model serving into a development environment that starts and stops automatically, this matters more than it might seem.
Gemma 4 as a Coding Model
Google’s Gemma 4 follows the same design philosophy as its predecessors: open weights, multiple sizes targeting different hardware profiles, and a training process that prioritizes instruction-following alongside raw capability. The model family has refined significantly since Gemma 2, with Gemma 3 having introduced proper multimodal support and substantially improved reasoning on structured tasks.
For coding work specifically, the 27B parameter variant is the practical target on current consumer hardware. At 4-bit quantization, it fits comfortably in 16-20GB of VRAM, covering most current midrange to high-end GPUs. Smaller variants trade capability for speed and memory headroom: a 12B model at Q4 will complete tokens noticeably faster and still perform adequately on well-scoped coding tasks.
The meaningful question for using any local model with a coding agent is not raw benchmark performance but whether the model reliably follows the structured output formats the agent expects. Coding agents like Claude Code issue tool calls formatted as JSON, parse the model’s responses for specific fields, and behave poorly when the model deviates from the expected schema. Larger models with stronger instruction-following tend to stay in the lane; smaller ones break format more frequently, which manifests as agent failures that look confusing until you look at the raw completions.
Wiring Claude Code to a Local Endpoint
Claude Code normally speaks to Anthropic’s API, but it respects a handful of environment variables that redirect that traffic. The critical one is ANTHROPIC_BASE_URL:
export ANTHROPIC_BASE_URL=http://localhost:1234
export ANTHROPIC_API_KEY=lm-studio
claude --model gemma-4-27b-itThe API key value is arbitrary when pointing at a local server that does not enforce authentication. LM Studio accepts whatever string you send in the Authorization header and proceeds. Setting it to something recognizable like lm-studio just makes logs easier to read.
The --model flag tells Claude Code which model identifier to include in the request payload. LM Studio will use whatever model is currently loaded, so the identifier in the request does not need to exactly match the loaded model name, but keeping them consistent avoids confusion when you are switching between models.
What you get from this configuration is Claude Code’s full interface: the file editing tools, the bash execution tool, the context window management, the multi-step planning behavior. What you give up is the tuned behavior that comes from a model specifically trained to work with those tools. Claude (the model) was trained to use Claude Code’s tool schema correctly and to produce the right output formats reliably. Gemma 4 will do this reasonably well on most tasks, but you will encounter cases where it hallucinates tool call syntax or produces partially-valid JSON that breaks the agent loop.
The practical effect is that local model setups work better for contained, well-specified tasks: “fix this bug,” “add error handling to this function,” “write a test for this module.” Long multi-file refactors that require the agent to maintain a plan across many steps are where the gap with a purpose-trained model becomes more visible.
The OpenAI Compatibility Layer as Infrastructure
One thing worth noting is that the OpenAI-compatible API format has effectively become the USB standard for local AI inference. Every major local server implementation supports it, and every client library targets it. This is what makes setups like this possible without any glue code: Claude Code speaks the format natively through its base URL override, and LM Studio serves it natively through its built-in API server.
This is a meaningful shift from two years ago, when connecting a local model to an agentic tool required writing a shim layer, handling prompt format conversion by hand, and debugging format mismatches that were rarely well-documented. The ecosystem has standardized in practice even without any formal standardization process.
Ollama is worth comparing here. For users who do not need LM Studio’s model management GUI, Ollama offers the same headless operation, the same OpenAI-compatible API, and a similarly simple model loading interface (ollama run gemma4:27b). The functional difference for this use case is narrow. LM Studio tends to have broader hardware support and more flexible quantization options; Ollama has a simpler installation story and better Linux support. Both will work with the Claude Code configuration above.
What This Actually Changes
The practical value of this setup depends on what you are optimizing for. Privacy is the obvious one: code, prompts, and any sensitive context in the conversation never leave the machine. For work on proprietary codebases where sending source to a third-party API is a policy concern, local inference removes that constraint entirely.
Cost is secondary but real. Running coding agent sessions against a cloud API is not cheap, especially when you are iterating quickly and accumulating large context windows. Local inference has a hardware cost but no per-token cost, and the economics shift in favor of local setups once you are using the model heavily enough.
The capability gap remains. Gemma 4 at 27B is a genuinely capable model, but it is not Claude 3.7 Sonnet or GPT-4o in terms of coding performance. For most day-to-day coding assistance tasks this difference is not critical. For the tasks where you need the agent to navigate a large unfamiliar codebase, make judgment calls about architecture, or handle complex multi-step reasoning, the frontier cloud models still hold a meaningful advantage.
What is interesting about the current moment is that the infrastructure gap has closed faster than the capability gap. Running a coding agent locally is no longer an engineering project in itself. It is a configuration step. Whether the model is good enough for your specific use case is a separate question, and one worth actually testing rather than assuming the answer.
The HN thread on this article has some useful discussion of quantization choices and comparisons with Ollama-based setups for anyone working through this themselves.