A private LLM on your own server: open models for sensitive business data

What a private LLM gives you

• Your data stays in-house: documents, customer data and emails are processed on a server in the EU that is yours. Nothing goes to an external AI provider.
• Predictable costs: you pay for the server, not per question or per word. More use does not mean a higher bill from an AI provider.
• Not dependent on one provider: many open models have a permissive licence, such as Apache 2.0 or MIT. Price or terms changes by a cloud provider do not affect you.
• An honest picture of quality: a local model is not equally good at every task as the largest cloud model. That is why I test on your own examples first, and we choose per task.

What you use a self-hosted LLM for

• Summarise and classify sensitive documents
• Pre-sort customer questions or tickets
• A knowledge base AI on your own documents
• Create embeddings for searching by meaning
• A fallback route when a cloud model is not available

How the integration works

Ollama runs the model on a server in the EU that is yours. In n8n, the Ollama node calls that model, just like a cloud model. If many people or processes need a model at the same time, a model server such as vLLM can take over; it offers an OpenAI-compatible API. For heavy tasks, a cloud model can step in through the same gateway, so you choose per task what stays local.

Which models run locally

A selection from the official Ollama library. The sizes are the variants available there; which one fits depends on the task and the hardware.

Model	From	Sizes in Ollama	Licence
Qwen3	Alibaba (Qwen)	0.6B to 235B	Apache 2.0 (including Qwen3-8B)
Llama 3.3	Meta	70B	Llama 3.3 Community License
Llama 3.1 / 3.2	Meta	1B to 405B	See Meta's licence per version
Gemma 3	Google	270M to 27B	Gemma Terms of Use
Phi-4	Microsoft	14B	MIT
gpt-oss	OpenAI	20B and 120B	Apache 2.0
DeepSeek-R1	DeepSeek	1.5B to 671B	MIT (derived versions follow their base model)
Mistral	Mistral AI	7B	Apache 2.0
Granite	IBM	2B to 30B	Apache 2.0

Facts about the local AI API

Checked on 2 November 2026 against the official documentation. Limits, prices and versions change; the source is listed per row.

Fact	What the documentation says	Source
Authentication	None locally. Ollama listens on 127.0.0.1:11434 only by default; external access goes through a proxy with a token or password.	Ollama: FAQ
Rate limits	None. You set capacity with the number of parallel requests (1 by default) and a queue of 512; when the queue is full you get a 503.	Ollama: FAQ
Version	The API is not strictly versioned but deliberately stable. Deprecations are rare and announced; the old /api/embeddings has been replaced by /api/embed.	Ollama: API
Compatibility	OpenAI-compatible /v1 endpoints and an Anthropic-compatible /v1/messages, so existing code and gateways work.	Ollama: API
Cost	Free locally (open source); you pay for the server. Ollama's cloud variant has its own pricing.	Ollama: Cloud
n8n	Built-in sub-nodes Ollama Chat Model, Ollama Model and Embeddings Ollama.	n8n documentation

What you need on your side

• A server in the EU with enough memory for the chosen model: a small model runs on an ordinary server, a large model or heavy use needs a GPU
• Examples of real tasks to test the quality beforehand
• A decision on who may use the model and through which access
• With much concurrent use: room for a model server such as vLLM

Where it often goes wrong, and how I prevent it

• Hardware: the size of the model determines how much memory and computing power is needed. I work that out for you in advance.
• Quality: a small local model is not good enough for every task. I test it on your examples before it goes live.
• Maintenance: a local model needs updating and monitoring, like any other service.
• Speed: without suitable hardware, a local model is slow. Tasks that are not urgent then run overnight.

In practice

Holos33, my own app, uses the Qwen family through its own model-independent gateway. Connecting a different model there is a configuration choice, not a rebuild.

What it costs

I build a process with local AI as one scoped process for a fixed price, from €750. Maintenance and monitoring are a monthly fee: n8n management from €250 per month, server management €145 per server. You rent or buy the server itself; I work out beforehand what is needed. All prices and what is included →

What I do not do

• I do not expose a local model on the internet without access control.
• I do not promise cloud quality from a small local model; I test on your examples first.
• You buy or rent the hardware yourself; I work out beforehand what is needed.

Frequently asked questions about a private LLM

What is the difference with ChatGPT?

ChatGPT runs at OpenAI. A local model runs on your own server; your data stays there. For many business tasks, a well-chosen open model is more than enough.

Which models can run locally?

Among others the Qwen family, Meta's Llama and Mistral, via Ollama. Which one fits depends on the task and the hardware.

Do I need my own GPU?

Not always. Small models run on an ordinary server; for larger models or heavy use, a GPU server is sensible.

How many questions can a local model handle at once?

By default Ollama processes one request at a time and queues the rest, up to 512. For more concurrency I raise that setting or add a model server such as vLLM, depending on the hardware.

Can I switch to a different model later?

Yes. Ollama offers OpenAI-compatible endpoints and n8n talks through a gateway; another model is then a setting, not a rebuild. That is how it works in Holos33 as well.