< | Same as above, scoped to the sub-task's own conversation |
| Gem agent | When a conversation uses a gem's agent, or a gem's workflow (SOP) reaches an agent step | The model chosen for this agent under "Agent"; if none, the `chat` role | Same as Chat, with the system prompt replaced by this gem's own |
| Gem Studio assistant | When you talk to the assistant in Gem Studio | The model chosen under "Agent › Gem Studio"; if none, the `chat` role | Same as Chat, plus the draft files it reads |
Which model a conversation uses is fixed when it is created. Changing settings afterwards affects only new conversations.
### Done automatically by the app in the background
| Feature | When it is called | Which model | What is sent to the model |
| --- | --- | --- | --- |
| Generate a conversation title | The first message of a new conversation. A message of 6 characters or fewer is used directly as the title, with no model call | The model chosen under "Agent › Background tasks"; if none, the `chat` role | Only the text of that first message |
| Summarize a long conversation | When the conversation's size exceeds "Settings › System › Auto-compact conversations", before the assistant's next reply. Default 100,000 tokens; set to 0 to turn off | The model chosen under "Agent › Background tasks"; if none, the `chat` role | The earlier part of this conversation: your messages and the assistant's (overly long ones keep only the beginning and end); for tool calls, only the tool name, shortened arguments, and the result. The most recent 10 messages are not included |
| One-line description of a knowledge-base collection | After the subfolders or file names of a top-level collection in the knowledge base change. At most once every 30 minutes per collection | The model chosen under "Agent › Background tasks"; if none, the `chat` role | The directory structure and file names of this collection. Document content is not included |
| Reading text in scans and images | When you import a scanned PDF or images into the knowledge base; you are not asked first. Each PDF page with no text is one call, and at most the first 500 pages of each file are read (pages beyond that are not read, and the import result says so); each image you select or drag in on its own is one call. When you import a whole folder, images inside it are not read | The `vision` role (called "Vision" in the interface); if not set, the `chat` role | The pages or images themselves (converted to grayscale and shrunk). The text read out is stored in your knowledge base |
| Calls a gem starts itself | When a gem you installed calls a model in its tool or workflow. Only a gem that was granted the `llm` permission at install can do this | The `chat` role | Decided by the gem. What it can read depends on the other permissions you have granted it |
Calls of this kind are billed at your model provider's prices like any other, and the usage counts toward the "Stats" page. The tokens they use do not count toward the "Token limit per run". Reading scans is one call per page, so a scan of dozens of pages costs noticeably more than anything else in the table above.
Titling, summarizing long conversations, and describing the knowledge base read your messages and file names, and that content is sent to the provider of the model used for "Background tasks". Choose a small model that runs on this computer for it under "Agent › Background tasks" (for example the "Local" role), and this content never leaves your computer.
To read text in images, the model must be able to see images. If the model in use cannot (for example a text-only model, or a local GGUF model), the import of that file fails and the import result says why. Under "Settings › AI Settings › Roles", choose a model that can see images for "Vision", then import again. Once a file has been read with a given model, importing it again does not read it again; the exception is the first import after the app updates to a new version.
### How long-conversation summarizing works
When a conversation gets very long, its full content exceeds what a model can process at once. After it passes the size you set, the app asks the "Background tasks" model (the `chat` role if none is chosen) to write the earlier part up as a summary. From then on, what is sent to the model is "this summary + the recent messages verbatim", no longer the whole history.
- The chat record is not affected. All messages are still there; a divider "Earlier messages summarized" appears in the record, and clicking it shows the summary's content.
- After summarizing, the assistant knows the details of the earlier content only as far as the summary wrote them down. If it needs some earlier detail, just tell it again.
- If the conversation keeps growing, it is summarized again. The new summary is written on top of the previous one.
- This size must be set smaller than the context length of the model you use. The app currently does not know each model's context length and will not adjust it for you.
- If the summary could not be written (for example the model service is temporarily unavailable), this reply proceeds as usual, using the full history.
### What does not call your models
- Knowledge-base search and the vector index use the local embedding model installed with the app. It runs on your computer, and content is not sent to any service.
- Importing web pages: the page is opened in the app's own browser in the background and its main text extracted, without going through a model. Images in the page are not read.
- "Deep thinking", "Local", and the roles you added yourself are used only when you point an agent at them, or when an agent hands a sub-task to them; "Vision", besides that, is also used to read text in scans and images (see the table above).
## Choosing a model for a gem
After a gem is installed, its page opens. Under "Settings › AI Model" you choose a model for each of its agents: the dropdown lists your roles first (choosing a role follows whatever model you configured for that role under "Roles"), then all the models you have added. If none is chosen, the `chat` role is used. What you change under "Settings › AI Settings › Agent" is the same setting. This item is not shown while no model has been added.
Some gems require the model, besides text, to also see images, hear audio, or watch video, or to generate images, audio, or video. When you choose a model for its agent on the gem's page or under "Agent", Notochord checks the chosen model against a list of the capabilities of common models:
- If it matches, nothing is shown.
- If the list says this model cannot, or the model is not in the list (for example a name you gave the model yourself), or the chosen role has no model configured yet, a line of hint appears under the dropdown. Check for yourself: if the model can in fact do it, ignore the hint; if not, pick another.
- The hint does not block installing, and does not block use.
]]>