Run open-source models inside your Go application with the Kronk SDK. Develop locally, then deploy that application to Google Cloud Run using NVIDIA GPUs. Try three live demos with real inference—no separate model server or hosted LLM API.
Try the Live Demos Explore Kronk →
Build Go services that perform their own inference. With the Kronk SDK, your service generates text and embeddings through llama.cpp in the same Go process—no separate model server or hosted inference API. These demos run that same integration on a development machine and in Google Cloud Run using NVIDIA GPUs.
The public demo runs on an NVIDIA L4 GPU in Cloud Run, with both text generation and embeddings running inside the Go service.
Explore Go, Rust, and software design through a conversation powered by your own content and open-source models running inside Go.
Kronk generates answers and Qwen embeddings inside the Go application.
Retrieve relevant passages from Ardan Labs learning content using MongoDB vector search.
Choose Go Notebook, Ultimate Rust, or Ultimate Software Design and stream the response.
Play against an AI that combines a language model with thousands of recorded game boards. No account required.
Ask about the weather in plain English and see a language model turn your question into a database query.
These are not wrappers around a hosted model API. Kronk loads open-source model files and executes inference inside the application. Locally, that means your machine. In production, it means the Google Cloud Run instance serving your request.
Model weights and native inference libraries ship in the container image. Google Cloud Storage holds the source model releases and recorded boards. MongoDB stores embeddings and saved games, and Cloud SQL for PostgreSQL supplies the weather records. The data services are remote; GPU inference stays inside the Go backend.
Kronk is Ardan Labs' open-source Go SDK for running models on your own hardware. These demos use the SDK directly—not Kronk's optional model server—so model loading, text generation, and embeddings live alongside the application logic.
Call models from Go. No Python service or separate model-serving process is required.
Run model files you control instead of sending inference requests to a hosted LLM provider.
Generate answers and vectors through the same SDK. Build retrieval into real applications.
Use the same Go integration in local development and a deployed container, with models selected for each environment.
Run inference on CPUs or accelerate it on GPUs with Metal on Apple Silicon and CUDA on NVIDIA GPUs. This deployment uses an NVIDIA L4 GPU in Cloud Run.
Explore the SDK and the demo source to see how the models, retrieval, and application code fit together.
The Kronk SDK loads GGUF model files through llama.cpp. Our application calls the SDK directly for text generation and embeddings; there is no HTTP hop to a separate inference server.
Our GPU deployment uses Ornith 1.5 9B Q4_K_M for text generation, the same 9B model used in local development. Both use Qwen3 Embedding 0.6B for the vectors that drive retrieval.
Choose models for the memory and compute available, without replacing the application integration. Inference is local to the runtime—not necessarily local to the visitor's browser—and model output can still be imperfect.
Watch William Kennedy's Kronk series to learn about running open source models and building Go services that perform their own inference.
Then explore the Kronk SDK and this demo's source to see model loading, embeddings, and inference inside Go—and how we package the application for Google Cloud Run.
Read the Kronk manual to get started with open source models.
Watch the Kronk SeriesSelect open-source text and embedding models that fit your task, hardware, and license requirements.
Load the models in Go, build your retrieval and application logic, and test on your development machine.
Package the model weights, native inference libraries, and Go application in a container. Reuse model and runtime layers for application updates.
Configure the NVIDIA GPU, CPU, memory, startup probes, and scaling. Run GPU inference inside the deployed Go application.
