A clean chat UI backed by a small HTTP server that proxies to an Ollama instance on a remote GPU (over a private VPN). The browser never talks to the GPU directly: the gateway is the only thing that knows where the model lives. A single entry point a whole team shares, the GPU off the public network, and nothing going to any cloud provider. All in Python stdlib — no framework, no build, no pip install.
Clients hit the gateway; the gateway holds OLLAMA_URL. The model host stays on a private network — one shared, auditable entry point instead of N browsers each configured with the GPU's address.
/api/chat relays Ollama's NDJSON as chunked HTTP transfer, and the browser reads the incremental stream — tokens show up as they're generated. Streaming is preserved even against the demo's mock.
/api/models enriches Ollama's /api/tags with plain-language notes about quantization (Q4_K_M…) and capabilities (vision, tools, thinking), so a non-expert can pick well. The context window is exposed and configurable per conversation.
Static file serving is hardened against path-traversal; if Ollama is unreachable, it returns a clear 502 with the target URL instead of hanging or leaking a stack trace; mid-stream disconnects are swallowed cleanly.
Runs on any box with a Python 3 interpreter and nothing else: python gateway/server.py. A trivially auditable codebase, no dependency supply chain, no version drift, and a Docker image that's just python:3.12-slim + a few files.
docker compose up brings up a mock Ollama that streams a canned but coherent response, so the whole path (browser → gateway → Ollama) works without a GPU or a real model. Point OLLAMA_URL at your host and you're talking to the real model.
The test suite is stdlib-only, the same constraint as the gateway itself. Each run boots the real gateway and a mock Ollama as separate subprocesses on ephemeral ports, waits for both to come up, then drives them over HTTP like a browser would. Nothing is stubbed in-process: if the wiring is broken, the test fails.
test_health/api/health answers. Local liveness, independent of the model host.
test_models_proxiedThe mock catalog arrives through the gateway — the proxy path is real, not a fixture.
test_chat_streamsA reply streams end to end: client → gateway → mock. This is the one that matters; streaming is where a proxy usually breaks.
test_index_servedThe UI is served from static/.
There is no coverage measurement, no load benchmark and no security suite in this repository — so this page will not quote you one. These four tests prove the path is wired end to end; they do not prove the gateway is hard against a hostile client. For 216 lines of standard library whose deployment assumption is a private network, that is a deliberate ceiling rather than an oversight — but it is a ceiling, and a number nobody measured would be worth less than saying so.
| Method | Path | Purpose |
|---|---|---|
GET | / | Serves the chat UI |
GET | /api/health | Liveness + the configured OLLAMA_URL |
GET | /api/models | Model catalog enriched with readable metadata |
POST | /api/chat | Proxies chat to Ollama; streams NDJSON back |
The browser only ever sends POST /api/chat. The gateway is the single component that knows where the GPU host lives — and it has to handle a bad request, a dead upstream, and a user who closes the tab mid-answer without ever hanging.