Local AI
What it is
Local AI is a model on a graphics card on the company's server. A runner is installed on the server, the admin connects it in Organization AI → Runners → Add AI → Own server with local AI (see Organizations and the organization's AI) and picks in Routing which features run on the card.
Local AI is in the left menu under Settings → the organization's name → Local AI. Everyone in the organization sees its state, only an organization admin can switch it and change its settings. The Settings and Logs tabs are for admins only. When the organization has more runners with a card, the runner is picked in the top right corner.
Overview: card modes
The card always does one mode. The modes that come with it are One person, Team, Agent, Transcription and Images and video.
- Clicking a tile switches the card. A switch takes tens of seconds and the local AI does not answer meanwhile. What the card cannot do in the new mode waits, or goes to the fallback of the routing; the dialog says so beforehand.
- The pencil on a tile changes the mode's name and icon.
- Add mode adds a mode from Tuning and measuring to the overview, or creates a new one.
- Release the card stops the engine, for a measurement or another tool, say. The calendar does not switch a released card on by itself, picking a mode does.
Calendar
The Calendar of modes says when the card has which mode, say Team during working hours and Agent at other times.
- Click an empty spot in the week and Add a block: days, from, to and the card mode. A Locked block keeps its mode, the card can be switched by hand only with Unlock and switch.
- Exceptions are another mode or free time once, a holiday say. They win over the blocks.
- Switch by the queue: outside the blocks, a card that idles switches to the mode in which jobs wait.
- Save the calendar.
Meetings waiting for a transcript get their turn even on a busy card, within half an hour at the latest.
Models
Models shows the state of each model: Downloaded (with its variants), Not downloaded, or Downloading with a percentage. The link at a model opens its page on Hugging Face.
Your own model:
- Add a model from Hugging Face. The local AI loads only a text model in the GGUF format, a quantized (shrunk) version of the model. Its repository usually has the same name ending with
-GGUF, for exampleunsloth/Qwen3.8-27B-GGUF. A repository with the original weights cannot be added. - Type a part of the name (for example
qwen 27b) and Search: the list shows models in the GGUF format, the most downloaded first. Or paste the name of a repository or the address of the model's page and Load variants. The link Open GGUF models on Hugging Face opens the list with the GGUF filter on. - Choose a variant. Each shows its size, what is downloaded and the disk space. The variant that fits the card best is marked recommended.
- Download it. The option Measure and tune right after the download tries the model at once.
Licenses. The models of the catalog are under Apache 2.0 or MIT. Your own model can be added under another license too: the dialog shows its name with a link and the download has to be confirmed, the confirmation is recorded in the audit. A license that forbids commercial use comes with a loud warning: such a model is not for the company's regular work.
A model with terms. Some models are released only after their terms are accepted. Steps: sign in on the model's page on Hugging Face and accept the terms, create a read token in the account settings and save it on the server with the card as HF_TOKEN.
Models for images, video and audio cannot be added, onlybrain manages them: they run in another engine with prepared and tested workflows and the model file alone would not be enough.
Disk space. Below the catalog are the Downloaded files: the free space of the server's disk, how much the models take, the size of each file and what uses it. An unused model can be deleted.
Tuning and measuring
For advanced users, change only with a good reason.
- The Card mode choice shows every mode of the runner. Add a card mode creates a new one, Duplicate copies the model and the parameters.
- A mode gets a model from the downloaded ones, a context (up to 1 million tokens) and the number of people at once (up to 256), with a slider or a number. The calculator shows at once what fits the card's memory. What does not fit cannot be saved.
- Show in the overview: only modes in the overview can be switched to and put in the calendar.
- Measure and tune tries setting variants and several people at once and saves the best setting. The local AI does not answer while it measures, which takes a few minutes.
- Discard changes drops unsaved edits, Back to the shipped values returns a shipped mode to its original setting.
Jobs
Jobs shows the Queues by mode: what waits for the card and in which order. A member sees the queues and their own jobs. An admin can move, pause, cancel or move jobs to another mode, and also has the History (jobs and chats of the local AI for 30 days, only facts about the work, no content, with a filter by user and mode).
Settings
- Behavior on a switch: Finish the job (the card switches once the running job ends), or Back to the queue and switch (the job goes back to its place in the queue and starts again when the card is back in its mode).
- Fallback while the card is in another mode: off means features routed to the card wait and the data stays in the company. On: they go to the fallback of the routing.
- Access for developer tools (/v1): an address with the OpenAI interface and a key for your own tools. The chat and the jobs in onlybrain do not need it.
Local AI at work
- Chat: where the organization routes the chat to the card, the chat offers Local AI right after a brain opens, and each answer says which AI answered. See The chat in the app.
- Meeting transcription: see How to record a call.
- Processes: the AI chat of a process can answer from the local AI, for big folders too.
- Images and video: in the brain under More → Images and video. Pick what to make (a picture, an edit of a brain picture, a short video), write the prompt and click Prepare from the brain. The AI reads from the brain what the product and the brand are, shows the files it draws on and prepares a prompt (for a video, shot by shot) that can be edited before it starts. The result is saved into a folder of the brain, for a role that may only propose as a proposal.