2026-09-04  ·  llm · ollama · ai · claudecode

What to Delegate and What Not to Delegate to Local LLMs (with Benchmarks)

When using cloud-based LLMs, token costs become a concern. It is natural to think, "Can I offload light tasks to a local LLM to save money?"

I ran Ollama on my local machine and tested task delegation in practice. The conclusion is that there is a clear divide between tasks that benefit from this approach and those that actually cost more. I will outline these boundaries with measured data.

Environment Used

• Ollama on a local GPU machine (called via HTTP API)
• qwen3.6:35b (MoE) — Bulk processing of long texts
• qwen3.6:27b (dense) — Short text and templates
• bge-m3 — Embeddings

The caller side consisted of a thin Node wrapper script.

const HOST = process.env.OLLAMA_HOST ?? "http://192.168.x.x:11434";

export async function chat(prompt: string, opts: ChatOptions = {}) {
  const res = await fetch(`${HOST}/api/chat`, {
    method: "POST",
    headers: { "content-type": "application/json" },
    body: JSON.stringify({
      model: resolveModel(opts.model),
      messages: [...(opts.system ? [{ role: "system", content: opts.system }] : []),
                 { role: "user", content: prompt }],
      stream: false,
      think: opts.think ?? false,          // Reasoning tokens are generally unnecessary
      options: { temperature: opts.temperature ?? 0.7 },
    }),
  });
  const data = await res.json();
  return { text: data.message.content.trim(), tokens: data.eval_count };
}

Do not forget to set think: false. Models that output reasoning by default will endlessly emit unnecessary reasoning tokens.

---

Tasks That Worked Well

1. Bulk Translation of Long Texts — The Main Use Case

I translated 10 chapters (882 lines) of a technical document into English.

For a single article (217 lines, 2030 output tokens), it took 32.6 seconds.

The key benefit is the complete preservation of structure. I constrained the prompt as follows:

- Preserve the markdown structure exactly: headings, tables, code blocks, lists.
- Do NOT translate content inside code blocks. Keep code identical.
- Keep relative markdown links exactly as they are (e.g. [text](03-architect.md)).
- Keep the tone factual and plain. No hype words
  (revolutionary, seamless, supercharge, unlock, effortless).
- Output ONLY the translated markdown. No preamble.

Explicitly listing prohibited words was effective. Without this, words like "revolutionary" or "seamless" tend to appear.

Verification is done mechanically by comparing the count of structural elements in the source and target texts.

def stats(t):
    return (len(re.findall(r'^#{1,3} ', t, re.M)),   # Headings
            t.count("```") // 2,                      # Code blocks
            len(re.findall(r'^\|', t, re.M)),         # Table rows
            len(re.findall(r'\[[^\]]+\]\([^)]+\)', t)))  # Links

If the counts do not match, it indicates missing translations or unauthorized additions.

2. Article Duplicate Detection (Embeddings)

As you write more articles, you tend to repeat previous content. I use bge-m3 for embeddings and mechanically filter duplicates using cosine similarity.

0.945  a.md <> b.md   ← Paraphrased repetition. Detected.
0.805  Article 1 <> Article 3  ← Similar topics. Acceptable range.
0.591  Article 2 <> Article 3  ← Distinct content.

I have a rule not to publish if the similarity exceeds a threshold of 0.85. In practice, one article was caught at 0.855, prompting me to restructure it. To the author, it may seem like they are writing about something different, so machine judgment is necessary.

3. Generating Headlines and Title Options

It is sufficient for generating 10 options and selecting one.

---

Tasks That Cost More When Delegated

Writing New Sales Copy or Article Body

This was the biggest lesson. Rewriting based on a draft costs more than writing from scratch.

Here is an actual output example:

* Five AI Agent Roles with Claude Code: How to Complete 10 Indie Apps in 3 Months
* Accelerating Development Efficiency with Parallel Processing: Releasing 10 Apps in 3 Months Using Claude Code and Role Division
* The Speed Revolution of Indie Dev: Why the 5-Person Workflow Built with Claude Code Produced 10 Practical Apps in 3 Months

"Speed revolution," "leveraged," "completed." Japanese hype expressions are mixed in. While a Japanese prohibited word list would reduce this, fundamentally this task requires judgment on "what not to say," which is difficult to convey via instructions.

Economically, it also does not make sense. You must read the entire draft (input tokens) and rewrite it (output tokens). If you write from scratch, you only pay for output tokens. Instead of reducing costs, they increase.

Short Texts

With the 27b model, generating 147 tokens took 12.3 seconds. For short texts, latency dominates, making the wait time the primary cost factor. For dozens of tokens, it is faster to write them yourself.

---

The Boundary Line

Criteria derived from benchmarks.

In one line: Can you pipe the output directly into a file without reading it?

If you don't need to read it, delegation is effective. If you need to read and correct the full text, writing it yourself from the start is cheaper. Translation falls into the former category. Since there is a mechanical pass/fail criterion for structural consistency, there is no need to visually scan the entire text.

Implementation Notes to Avoid Failure

Do not fire and forget. Once, ten chapters of translation were not running and I did not notice for several minutes. The log sat there empty, so the failure was invisible.

I nearly wrote that nohup was the cause. That would have been wrong. nohup exists precisely to keep a child alive when the parent or terminal goes away, so it is not a reason for anything to die. The real cause was elsewhere: a missing &, a kill sent to the whole process group, cleanup by the execution environment.

The lesson is simpler than the diagnosis. Do not use a form of invocation whose completion you cannot confirm. Leave job management to the caller, which can wait for the result.

Set timeouts generously. Bulk processing of long texts takes dozens of seconds. If you abort after the default few seconds, the model is still running, but the result is discarded.

Use aliases for models. Resolving aliases like fast, big, coder, or embed on the wrapper side allows you to swap models without touching the calling code.

Summary

• Bulk translation of long texts is effective. The ability to verify structure mechanically is key.
• Duplicate detection via embeddings finds issues invisible to the author.
• Do not delegate new text writing. It costs more to read and rewrite than to write from scratch.
• Short texts lose to latency.
• The decision criterion is "Can you pipe the output into a file without reading it?"

---

I publish the configuration for splitting Claude Code into separate personas — Architect, Coder, Reviewer, Conflict Resolver — under MIT. Copy it, run ./setup.sh, and it works. It does not depend on your tech stack.

https://github.com/quintetkit/quartet

I built one real tool using nothing but this workflow. Every Issue, PR, review and merge is still there. The parts that went wrong were not deleted.

https://github.com/quintetkit/mdlinkcheck

The version that adds a UI Designer persona, review criteria, a per-Issue parallel execution script and a 10-chapter guide is on the product page.