what is rag?

rag stands for retrieval augmented generation, and it means the system looks something up before it answers.

your question goes to a search step first. that step finds relevant passages from a corpus, those passages get placed into the prompt alongside your question, and the model answers from what is in front of it rather than from training alone. that is the entire idea, and everything else is engineering.

it matters because it is how a model gets access to anything private, recent or specific to you. here is how the pipeline actually works as of september 2026, and where it breaks.

the pipeline, in five steps

two phases: one you run ahead of time, one that runs per question.

ahead of time, once per document:

1. ingest. collect the documents.
2. chunk. split them into passages.
3. embed and index. turn passages into vectors, store them, usually alongside a keyword index.

per question:

1. retrieve. find the passages closest to the question.
2. generate. paste them into the prompt and answer.

the whole thing is a search engine with a writer bolted to the end. that framing is useful, because it tells you where to look when quality is poor: it is almost always the search, not the writer.

rag became the default pattern because the alternatives are worse. anthropic published the model context protocol in late 2024 to standardise the tool side of the same problem, and by 2026 it was the default integration layer across labs and editors. the research literature has moved on to how agents should discover and select among tools at scale rather than whether to retrieve at all.

chunking, which decides your ceiling

chunk boundaries set the quality ceiling before ranking ever gets a vote.

this is the least glamorous step and the one that determines whether the system works. a passage has to be small enough to be specific and large enough to be self contained. split an argument in half and retrieval returns something adjacent to the answer. leave chunks too big and the relevant sentence gets diluted by 800 words of context around it.

there is no universal right answer, which is why this stays hand tuned. the practical move is to read 20 retrieved chunks for real questions and ask whether each one would answer the question on its own. if not, fix the splitting before touching anything else.

embeddings and the vector database question

you probably need less infrastructure than you think.

an embedding turns a passage into a list of numbers positioned so that similar meanings land near each other. a vector database stores those and finds the nearest ones fast. that is genuinely useful for questions phrased differently from the source text.

but embeddings are bad at exactly what keyword search is good at: error codes, product names, dates, identifiers, rare terms. searching for ERR_CONN_REFUSED semantically is a worse idea than searching for it literally, and hybrid retrieval, running both and merging, beats either alone in most published evaluations.

so start with the search you already have. add vectors when you can demonstrate they improve results on your own questions, not because the architecture diagram looks incomplete without one.

rag vs fine tuning

rag changes what the model knows. fine tuning changes how it behaves.

that distinction resolves most of the confusion, and getting it backwards is expensive.

most teams reach for fine tuning when they needed rag, because the failure mode looked like the model not knowing something. if the answer is "the model has never seen our data," that is a retrieval problem. if the answer is "it knows, but formats it wrong," that is fine tuning.

they compose. a fine tuned model with good retrieval is a reasonable place to end up. it is a bad place to start.

where rag actually breaks

five failure modes, and only one of them is the model's fault.

1. the chunk was wrong. covered above, and the most common by far.
2. retrieval found nothing relevant and the model answered anyway, fluently.
3. the corpus contradicts itself. two documents disagree, both got retrieved, nothing resolves it.
4. the passage is stale. it was true in march. nothing marks it as out of date.
5. the fact was never written down. no retrieval strategy reaches it.

number 2 is worth guarding explicitly. a system that says "i could not find anything about that" is more valuable than one that improvises, and it is a prompt and threshold decision rather than a model capability.

number 3 has a security dimension too. if anything can write into your corpus, it can plant instructions rather than facts, which is rag poisoning. the cloud security alliance reported that the share of crawled pages carrying malicious injection grew 32% in relative terms between november 2025 and february 2026, covered in what is prompt injection.

number 5 is the interesting one, and it is not a rag problem at all.

it is also the one every vendor tried to paper over in 2026. openai shipped a background memory system on june 4, 2026, anthropic reworked claude's into editable entries on july 10, 2026, google rebranded gemini's context layer as personal intelligence in january 2026, and apple shipped siri ai with onscreen awareness and personal context on june 8, 2026. all of them retrieve from what their own products already hold, compared in ai memory is everywhere in 2026.

what retrieval cannot reach

a corpus only contains what somebody wrote down.

per the anthropic economic index for may 2026, the number one work task in sampled ai conversations is searching electronic sources for information at 4.95%, with reference searching second at 3.74%. retrieval is the dominant activity. now look at what stays small.

searching your own organisation's knowledge is 3.61% and anything about a meeting is 0.26%. those numbers describe supply, not demand. the decision made on a call, the constraint someone explained once, the page you read in july and closed: none of it is in any corpus, so no chunking strategy reaches it.

this is the absence failure from context engineering, and it is why we argue the recording layer matters more than the retrieval layer for personal context.

rag over your own screen

if the corpus is what you actually saw, the retrieval problem gets much easier.

remynd builds that corpus on a mac. it captures the focused window, runs ocr locally so what was on screen becomes text, and indexes it so you can search it. since august 2026 it exposes that archive to claude code and codex over a local connector, which is the practical setup in how to give an ai agent context.

the honest scope: capture, ocr, call transcription and storage stay on your mac by default, you can exclude specific apps or sites from capture entirely, and your history is read only everywhere in the app. asking a question sends the retrieved slices to a cloud model, and there is no local model option today. the fuller architecture is in private ai on your mac.

how to build one that works

in order, and the order is the advice.

1. fix the corpus first. what is missing matters more than how you rank.
2. read your chunks. 20 of them, for real questions, before any tuning.
3. use hybrid retrieval. keyword plus vector, merged.
4. let it say it does not know. set a threshold and honour it.
5. evaluate on your own questions. 50 real ones with known answers beats any public benchmark.
6. only then reach for a bigger model. it is rarely the bottleneck.

that list is deliberately unglamorous, and it is the difference between a demo and something people keep using. more on the wider picture in what is an ai agent.

download remynd for mac and give retrieval something of yours to find.