# Experiment with local AI on a student budget Start with a small, measurable task and synthetic data before downloading a local AI model. Compare a simple baseline with a model your laptop can run, and measure quality and speed. Local inference still costs memory, storage and electricity; Docker does not remove those limits. ## Start with the task, not the biggest model You can learn how AI applications work without an expensive laptop or a recurring API bill. Local inference removes a per-request provider charge, but still uses memory, storage, electricity and download bandwidth. Docker packages and runs software; it does not remove model hardware requirements. Choose a small task with an answer you can check. For a first experiment, classify a synthetic campus notice as ‘event’, ‘deadline’ or ‘other’. Keep a keyword-based baseline and ten manually labeled examples. Do not begin by uploading private messages or expecting a small model to operate your computer. ## Check the machine before downloading Read Docker’s current Model Runner requirements for your exact operating system, CPU architecture and acceleration support. Installation support and GPU support are separate questions. Leave memory for your operating system, editor and browser; a model file’s download size is not its total runtime memory requirement. On a memory-constrained machine, begin with a small quantized model and a short context. There is no reliable RAM-to-model-size guarantee: architecture, quantization, context and runtime overhead all matter. If your machine starts swapping heavily, stop and reduce the experiment. A slow CPU-only run may be enough to learn an API, but unsuitable for an interactive demo. Check the model license and download size before pulling it. Use a supported model from the current documentation, and record the exact identifier and version in your notes. Do not promise that the same model or acceleration path will work on every student laptop. Sources: [Docker Model Runner: requirements and supported platforms](https://docs.docker.com/ai/model-runner/) ## Run one controlled experiment On a supported Docker Desktop setup, enable Model Runner in the AI settings. Docker Engine uses a separate installation path; follow the current official guide for your distribution. Confirm the CLI is available before downloading a model. These commands follow Docker’s introductory example; the model tag can change, so record the resolved model information when you run it. After the model opens, ask for one classification, then exit the interactive session before continuing. Use synthetic text such as ‘Robotics club demo this Friday’. Record the prompt and output. Do not expose a model endpoint to the public internet for this exercise; review host-side TCP and network settings before connecting another application. Terminal · after installing and enabling Model Runner ``` docker model version docker model run ai/smollm2 ``` Sources: [Docker: get started with Model Runner](https://docs.docker.com/ai/model-runner/get-started/) ## Measure mistakes and latency Create a fixed evaluation file with ten notices, expected categories and a short reason for each label. Include ambiguity: a notice that mentions both an event and a registration deadline, an empty input and text unrelated to campus activity. Decide how your app should handle uncertainty before running the model. For each example, record the predicted category, whether the format is valid and elapsed time. Keep the first run separate because loading can be slower. Compare against your keyword baseline. Ten examples are a small learning exercise, not a benchmark proving the model is reliable. Change one thing at a time: the prompt, model or context limit. Do not silently remove hard examples. If the model returns extra prose, your application should reject or safely handle it instead of assuming every response is valid JSON. Keep inference behind an interface so tests can use deterministic fixtures. ### Diagram: Run the same ten notices through a keyword baseline and a local model. 1. **Fix the evaluation set** — Use ten synthetic notices with expected event, deadline or other labels. 2. **Run both approaches** — Keep the inputs identical for the keyword baseline and small local model. 3. **Inspect the result** — Record mistakes, valid output, memory and latency; ten examples are not a reliability benchmark. ## When local inference is too slow Keep building the application with recorded synthetic responses or a mock implementation. You can still implement input validation, timeouts, retry limits, an accessible UI and an evaluation report. Make the demo label explicit: ‘mock responses’ is honest; implying a model is running when it is not is not. If you choose a hosted API later, check the current pricing and retention terms, set a small spending limit and limit request size. Free tiers can change or expire. Never commit an API key or put a secret in frontend JavaScript. A local experiment should not accidentally become an unbounded public proxy. ## Make the experiment worth sharing Publish the synthetic dataset, evaluation command, hardware notes, model identifier and the cases that failed. Include a short explanation of why you kept or rejected the model compared with the baseline. This shows engineering judgment even if the model is not impressive. OpenAI’s harness-engineering article describes a team’s experience building the environment and feedback loops around coding agents. Treat it as a case study, not proof that a beginner can reproduce its results. Borrow a small principle: define observable behavior, supply a test and inspect the result. Your next project can apply that principle without reproducing the team’s tools or scale. A useful finish line is simple: another student can run the experiment, understand its limits and reproduce the result. Put that evidence in your README and explain your own contribution when sharing the project. Sources: [OpenAI: harness engineering](https://openai.com/index/harness-engineering/) · [Opened: make your GitHub project inspectable](/blog/github-projects-that-show-your-work) ## References - [Docker Model Runner: requirements and supported platforms](https://docs.docker.com/ai/model-runner/) - [Docker: get started with Model Runner](https://docs.docker.com/ai/model-runner/get-started/) - [OpenAI: harness engineering](https://openai.com/index/harness-engineering/) Updated 2026-11-02 by Opened editorial.