Pipeline

This page follows the cells of notebook 04 in order. Each code cell is quoted with the output it produced when the notebook was executed on 15 October 2026 with nishad/apertus:8b (the 16 GB tier). Where the 8 GB tier differed, the page says so and quotes the 8 GB copy in tier-8gb/. The notebook’s HTML answers are shown here as plain text. Long outputs are cut and marked with “…”.

The notebook works with two documents in data/docs/: the IFLA Library Reference Model and the DCMI Metadata Terms. Both are licensed under CC BY 4.0 and stored as text extracts.

04.01 Configuration

import os
os.environ.setdefault("WORKSHOP_TIER", "16gb")
from workshop.config import load_settings, context_limit
settings = load_settings(); MODEL = settings.model; EMBED_MODEL = settings.embed_model
LIMIT = context_limit(MODEL)
K = 3 if LIMIT <= 4096 else 4         # fewer chunks on the 4B's 4k window
print("Model:", MODEL, "| context:", LIMIT, "| chunks per question:", K)

Model: nishad/apertus:8b | context: 8192 | chunks per question: 4

From the executed notebook 04, 15 October 2026. K is the number of chunks per question. On the 8 GB tier the same cell printed Model: nishad/apertus:4b | context: 4096 | chunks per question: 3, from the executed notebook 04 on the 8 GB tier, 15 October 2026. The 4B has a context window of 4,096 tokens, so it gets one chunk fewer.

04.02 Load and chunk

from pathlib import Path
from workshop.rag import load_document, chunk_text
chunks = []
for path in sorted(Path("data/docs").glob("*.md")):
    text = load_document(str(path))
    chunks += chunk_text(text, source=path.stem, size=300, overlap=50)
by_id = {c.id: c for c in chunks}
print(len(chunks), "chunks from", len(set(c.source for c in chunks)), "documents")
print(chunks[5].id, "→", chunks[5].text[:200], "...")

173 chunks from 2 documents
dcmi-terms#5 → The distinction between titles and alternative titles is application-specific. Has Range http://www.w3.org/2000/01/rdf-schema#Literal Subproperty of Title (http://purl.org/dc/elements/1.1/title) Title ...

From the executed notebook 04, 15 October 2026. Two documents became 173 chunks of about 300 words with 50 words of overlap. The id dcmi-terms#5 names the source file and the chunk’s position in it, and that is what a citation points back to. Chunk 5 starts in the middle of a term’s description, because the chunking code cuts by word count and ignores the document’s structure.

04.03 Embed once

import time
from workshop.search import build_index, search
t = time.time()
index = build_index([c.id for c in chunks], [c.text for c in chunks], model=EMBED_MODEL)
print(f"indexed {len(chunks)} chunks in {time.time()-t:.0f} s")

indexed 173 chunks in 19 s

From the executed notebook 04, 15 October 2026. The embedding model is qwen3-embedding:0.6b. The index is kept in memory only for the session, so closing the notebook discards it. The 19 seconds were measured on the machine that ran the notebook; expect a longer wait on a slower or CPU-only machine.

04.04 Look at what was retrieved first

question = "Which DCMI term expresses the date on which a resource was formally issued?"
for cid, text, score in search(question, index, k=K):
    print(f"{score:.3f}  {cid}\n      {text[:160]}...\n")

0.721  dcmi-terms#7
      Definition A point or period of time associated with an event in the lifecycle of the resource. Comment Date may be used to express temporal information at any ...

0.630  dcmi-terms#8
      property Date, of which this is a subproperty. Has Range http://www.w3.org/2000/01/rdf-schema#Literal Subproperty of Date (http://purl.org/dc/elements/1.1/date)...

0.612  dcmi-terms#6
      resource, spatial applicability of the resource, or jurisdiction under which the resource is relevant. Comment Spatial topic and spatial applicability may be a ...

0.601  dcmi-terms#24
      resource. ### FileFormat Class More details → URI http://purl.org/dc/terms/FileFormat Label File Format Definition A digital resource format. Subclass of http:/...

From the executed notebook 04, 15 October 2026. This is a retrieval failure. The expected answer is dcterms:issued, and none of the four chunks is the one that defines it. The first result is the general definition of Date. The other three are nearby chunks about date subproperties, spatial coverage and file formats.

04.05 Ask with the documents in the prompt

from workshop.rag import answer, build_prompt
from workshop.display import show_markdown
text, used = answer(question, index, by_id, k=K, model=MODEL)
show_markdown(text)
print("\nCited material:")
for n, c in enumerate(used, 1):
    print(f"[{n}] {c.source}#{c.position}: {c.text[:100]}...")

The DCMI term that expresses the date on which a resource was formally issued is "dateAccepted". This term is defined as "Date of acceptance of the resource. Comment Recommended practice is to describe the date, date/time, or period of time as recommended for the property Date, of which this is a subproperty. Examples of resources to which a date of acceptance may be relevant are a thesis (accepted by a university department) or an article (accepted by a journal)."

[1] and [2] do not provide information about the date on which a resource was formally issued. [3] and [4] do not provide information about the date on which a resource was formally issued.

Cited material:
[1] dcmi-terms#7: Definition A point or period of time associated with an event in the lifecycle of the resource. Comm...
[2] dcmi-terms#8: property Date, of which this is a subproperty. Has Range http://www.w3.org/2000/01/rdf-schema#Litera...
[3] dcmi-terms#6: resource, spatial applicability of the resource, or jurisdiction under which the resource is relevan...
[4] dcmi-terms#24: resource. ### FileFormat Class More details → URI http://purl.org/dc/terms/FileFormat Label File For...

From the executed notebook 04, 15 October 2026. The answer is wrong: dateAccepted is a different term. The top chunk, dcmi-terms#7, holds the generic definition of Date and the neighboring term dateAccepted. The passage that defines dcterms:issued was not in the top four. The model then answered confidently from the wrong passage it was given: the definition it quotes is in chunk [1]. Its last sentences claim that none of the four chunks provides the information, which contradicts its own answer.

The retrieval step decided this outcome: the passage that defines dcterms:issued was never in the prompt. A wrong answer with a real citation is also harder to notice than an invented one. Read the cited passage and check that it answers the question you asked, not a closely related one.

On the 8 GB tier the 4B, with three chunks, answered with the single line Date (http://purl.org/dc/terms/date), which is also wrong (from the executed notebook 04 on the 8 GB tier, 15 October 2026).

04.06 When the documents are silent

text, used = answer("Who won the 2026 Nobel Prize in Literature?", index, by_id, k=K, model=MODEL)
show_markdown(text)

The documents do not answer the question. The question asks who won the 2026 Nobel Prize in Literature, but the documents are about bibliographic identities, context of use, and the recognition of distinct bibliographic identities. They do not contain information about the Nobel Prize in Literature or its winners.

From the executed notebook 04, 15 October 2026. This is the refusal the prompt asks for: the model says the documents do not answer, with a reason.

On the 8 GB tier the 4B did not refuse. Its whole answer was a bare citation, a chunk number and nothing else:

[2]

From the executed notebook 04 on the 8 GB tier, 15 October 2026. Chunk 2 is a passage from the library standards that says nothing about the prize. A smaller model follows the refusal instruction less reliably. When no cited passage supports an answer, do not trust it.

04.07 The context budget

The system prompt, the retrieved chunks, the question and the answer must all fit in the model’s context window: 8,192 tokens for the 8B tag and 4,096 for the 4B tag. The helper estimates the size of the prompt and refuses to build one that is too large. Ollama has its own check. The cell asks for 40 chunks, which is far too many, and shows what each of the two checks does.

from workshop.rag import PromptTooLong
import ollama
many = [by_id[cid] for cid, _, _ in search(question, index, k=40)]
try:
    build_prompt(question, many, max_tokens=LIMIT)
except PromptTooLong as err:
    print("helper refused:", err)
try:
    big = "\n\n".join(c.text for c in many)[: int(LIMIT * 1.5 * 3.5)]   # about one and a half windows of text
    ollama.Client(host=settings.host).chat(model=MODEL, messages=[{"role": "user", "content": big + "\n\nSummarize."}],
                                          options={"num_ctx": LIMIT, "temperature": 0, "num_predict": 20})
    print("Ollama accepted the long prompt (it may have truncated silently; check num_ctx).")
except ollama.ResponseError as err:
    print("Ollama said:", err)

helper refused: About 25842 tokens of prompt for a 8192-token window. Use fewer or shorter chunks (lower k or size).
Ollama said: request (10162 tokens) exceeds the available context size (8192 tokens), try increasing it (status code: 400)

From the executed notebook 04, 15 October 2026. The helper estimates the whole prompt at one token per 3.5 characters, and that estimate is deliberately high. It also keeps 25 percent of the window free for the system prompt and the answer. Ollama’s number, 10,162 tokens, is a real count of the shorter text the cell sent. Both refuse.

The 8 GB tier behaved differently. The helper refused as before (About 25842 tokens of prompt for a 4096-token window). But then the cell printed Ollama accepted the long prompt (it may have truncated silently; check num_ctx). (from the executed notebook 04 on the 8 GB tier, 15 October 2026). Ollama’s server log for that run recorded that the 5,076-token prompt was cut to 2,051 tokens. The part that was cut off is gone, and neither you nor the model is told. On that tier, the helper’s refusal is the only protection you can rely on.

04.08 Five questions, scored by you

The questions come from data/docs/questions.json. You score each answer yourself, by reading it against the expected answer. The notebook does not compare the two texts automatically, and that is deliberate. The notebook cut each answer at 400 characters.

import json
questions = json.loads(Path("data/docs/questions.json").read_text(encoding="utf-8"))
for q in questions:
    text, used = answer(q["question"], index, by_id, k=K, model=MODEL)
    print("Q:", q["question"]); print("A:", text.strip()[:400]); print("expected:", q["expected"]); print("-" * 80)

Reading the 8B column: question 1 starts correctly, and the visible part matches the expected entities. Question 2 is wrong, as in 04.05. Question 3 gets the item right and the manifestation roughly right. Question 4 names LinguisticSystem, though its opening sentence is awkward. Question 5 refuses correctly. Four out of five are usable, and you only know which four by reading them. On the 4B every answer is a bare citation or a wrong term, so none of them can be trusted. The 4B column is from the executed notebook 04 on the 8 GB tier, 15 October 2026.

04.09 Bring your own document

Put a PDF, Markdown or text file into data/my/ and run the cell. It cuts the files into chunks of the default sizes, builds an index, and asks one question. When the notebook runs unattended, as it did for the committed copy, it asks the fixed question “What is this document about?” In a live session it waits for your question. On the executed run data/my/ was empty, so the cell printed Put a .pdf, .md or .txt file into data/my/ and run this cell again.

Three tiers

GUI

LM Studio’s Chat with Documents does this retrieval without code. Attach dcmi-terms.md from data/docs/ (save a copy as .txt if the app does not accept .md) and ask the dcterms:issued question from 04.04. Compare the answer with 04.05 above. Does the app find the right term? Which chunks did it show you, if any?

Run and change

Set K = 1 and re-run 04.08. Which answers go wrong? Then try size=150 in 04.02, which makes the chunks smaller, and see whether the dcterms:issued question retrieves a better passage. Finally, ask the same question in German: does retrieval over English documents still find the passage?

Stretch

Rerank the chunks, that is, let the chat model check the retrieved chunks before they are used. Use format= with a tiny Pydantic model {"relevant": bool, "reason": str}, and ask it whether each of the top 10 chunks answers the question. Keep only the relevant ones and answer again. Compare with 04.05. A second stretch: index this workshop’s website pages (download them as Markdown) and ask the index about itself.

On the 8 GB tier

The same notebook runs with WORKSHOP_TIER=8gb: three chunks per question and a 4,096-token window. Expect bare citations and no refusals, as above, and treat every answer as unchecked until you have read the cited chunk.

Check yourself

• I can turn documents into chunks and embed them once.
• I read the retrieved chunks before I read the answer.
• I checked a citation against the passage it points to.
• I know what happens when a prompt does not fit the window.