F3. Knowledge Bases & RAG

Track: Path 0: AI Foundations · Module: F3 Last updated: 2026-07-31 Level: Intermediate Time: 2 hours Prerequisites: F1 The Evolution of AI, F2 Prompt Engineering

flowchart LR
F1["F1 The Evolution of AI"]
F1 --> F2
F2["F2 Prompt Engineering"]
F2 --> F3
F3[" F3 Knowledge & RAG<br/>(you are here)"]:::current
F3 --> F4
F4["F4 Automation & Agents"]
classDef current fill:#ff9900,stroke:#333,color:#fff,font-weight:bold

Chapter Navigation

1. Why AI Doesn’t Know Your Product Information · 2. Embeddings · 3. Vector Databases · 4. RAG Architecture · 5. Hands-On Overview · 6. RAG Optimization Techniques · 7. FAQ · 8. Learning Resources · 9. Common Traps · 10. Completion Checklist

What You’ll Understand

Why doesn’t ChatGPT know your product details? How do you get AI to answer from your private data? RAG is the core technology that solves this.

After this module you’ll be able to:

• Explain why AI doesn’t know your products, policies, or internal data
• Explain embeddings in plain language
• Know what a vector database is and why you need one
• Understand the full RAG architecture and workflow
• Judge when a scenario needs RAG and when it doesn’t
• Follow the basic steps of building a product knowledge base (for code, see B3 RAG Knowledge Base)

This module’s scope: conceptual understanding — no code required. To actually build a RAG system, continue to Path B: B3 RAG Knowledge Base afterward.

1. Why AI Doesn’t Know Your Product Information

1.1 Where AI’s knowledge comes from

Recall F1: an LLM’s knowledge comes entirely from training data — public text from the internet: Wikipedia, news, forums, code repositories.

What AI knows:

• Amazon’s general rules and policies (public information)
• General traits of common categories (public discussion)
• Generic e-commerce operations knowledge (blogs, tutorials)

What AI doesn’t know:

• Your product’s actual specs and selling points
• Your internal pricing strategy and profit data
• Your suppliers and sourcing costs
• Your sales history and trends
• Your customer-service SOPs and internal policies
• The latest platform policy changes (training data has a cutoff)

1.2 Three ways to make AI “know” your data

1.3 The e-commerce analogy

Pasting = printing all your documents and spreading them on the desk for your assistant to consult.

• Fine when there’s little material
• The desk runs out of space when there’s a lot (context window limit)

Fine-tuning = making your assistant spend a month memorizing everything.

• Fast answers once memorized
• But updated material means re-memorizing (retraining)
• And they may mix things up (hallucination)

RAG = giving your assistant a filing cabinet and a retrieval system. For every question, they first pull the relevant folder, then answer from it.

• The cabinet updates any time
• Answers are traceable (back to specific documents)
• The cabinet can grow without limit

Bottom line: for cross-border e-commerce teams, RAG is the practical choice. Large volumes, frequent updates, and traceability requirements — exactly RAG’s strengths.

2. Embeddings: Teaching AI to “Understand” Meaning

2.1 What is an embedding

An embedding converts text into a list of numbers (a vector) that captures the text’s meaning.

The intuition:

Imagine placing products on a map. The traditional way is keyword matching — “Bluetooth earbuds” only matches documents containing exactly those words.

The embedding way places each product in a “semantic space”:

• “Bluetooth earbuds” and “wireless earbuds” sit close together (similar meaning)
• “Bluetooth earbuds” and “Bluetooth speaker” are a medium distance apart (related, different)
• “Bluetooth earbuds” and “kitchen knives” are far apart (unrelated)

Semantic space sketch (simplified to 2D):

audio devices
↑
Bluetooth speaker    wireless earbuds
Bluetooth earbuds
smartwatch    wired earbuds

← wearables    accessories →

phone case

kitchen knives (far away, not in this region)

Real embeddings aren’t 2D but 768D or 1536D (hundreds to thousands of dimensions), but the principle is the same: semantically similar text → nearby vectors.

2.2 How embedding works

Input text → embedding model → vector (a list of numbers)

Example:
"This Bluetooth headset has great noise cancellation"
→ [0.12, -0.34, 0.56, 0.78, -0.23, ..., 0.45] (1536 numbers)

"The active noise cancelling on these wireless earbuds is excellent"
→ [0.11, -0.32, 0.55, 0.79, -0.21, ..., 0.44] (1536 numbers)

The two vectors are very close → similar meaning!

2.3 Common embedding models

Recommendations for cross-border e-commerce:

• Tight budget: OpenAI text-embedding-3-small (cheap and good)
• Multilingual needs: BGE-M3 (free, open source; supports Chinese/English/Japanese/German/French)
• Data privacy: BGE-M3 self-hosted (data never leaves your servers)

2.4 Keyword search vs semantic search

In practice the best answer is hybrid search: keyword search to narrow the field, then semantic search to match precisely. That’s the mainstream RAG design in 2026.

3. Vector Databases: Storing and Retrieving Meaning

The numbers in this section are constructed to illustrate the point, not measured.

3.1 Why you need one

Ordinary databases (MySQL, PostgreSQL) excel at exact queries: “find products where price = $25.99.”

They’re poor at semantic queries: “find reviews semantically similar to ‘noise cancellation is weak’.”

Vector databases are purpose-built to store and retrieve vectors — finding the most similar few among millions in milliseconds.

3.2 The main vector databases

Recommendations:

• Just starting: Chroma (simplest — 10 lines of code)
• Production: Pinecone (no ops) or Qdrant (self-hosted)
• Already on PostgreSQL: pgvector (no extra infrastructure)

Sources: Vector Databases 2026 Guide, Embeddings and Vector Databases Guide

3.3 How a vector database is used

Write phase (one-time):
documents → chunking → embedding model → vectors → stored in the vector DB

Query phase (every question):
user question → embedding model → query vector → vector DB search → most similar chunks returned

Chunking is the crux:

You can’t store a 50-page manual as one vector — too big; the meaning gets diluted. Split documents into chunks:

The golden rule of chunking: each chunk should hold one complete unit of information. Too small loses context; too big adds noise. 500–1,000 words is usually a good starting point.

4. RAG Architecture: the Full Workflow

4.1 RAG’s three stages

Stage 1: Indexing — one-time preparation

collect documents → chunk them → generate embeddings
↓ ↓ ↓
product manuals 500 words/chunk vectorize
FAQ documents
review data → store in the vector database
policy files


Stage 2: Retrieval — every question

user question → query vector → vector DB search
↓ ↓
"Is this product waterproof?" top 5 relevant chunks returned


Stage 3: Generation — every question

system prompt + retrieved chunks + user question
↓
sent to the LLM
↓
the LLM answers based on the retrieved content
"According to the product manual, this product is IPX5 water-resistant..."

4.2 RAG vs asking the AI directly

Scenario: a customer asks “which Bluetooth version do your earbuds support?”

Direct question (no RAG):

AI: "Generally, earbuds from 2024–2025 support Bluetooth 5.0 or 5.3..."
→ generic — not your product's actual answer

With RAG:

Retrieved chunk:
"Model XB-500, Bluetooth 5.3, supports AAC/SBC/LDAC codecs,
15 m range, connects to 2 devices simultaneously."

AI answers from the retrieved content:
"Our XB-500 earbuds support Bluetooth 5.3 with AAC, SBC, and LDAC
codecs, a 15-meter range, and simultaneous connection to 2 devices."
→ precise, specific, grounded in your product data

4.3 RAG applications in cross-border e-commerce

Related: B3 RAG Knowledge Base for the build; A4 Customer Service for RAG-driven FAQ answering.

4.4 Prompt design for RAG

A RAG prompt typically has three parts:

System prompt (fixed):
"You are a product support assistant. Answer using the reference
material below. If the material doesn't cover it, tell the user you
are not sure — do not invent an answer. Cite your sources."

Retrieved chunks (dynamic):
---reference start---
[Chunk 1]: Model XB-500, Bluetooth 5.3...
[Chunk 2]: Water resistance IPX5, usable in rain...
[Chunk 3]: Battery: 30 h with ANC on, 50 h off...
---reference end---

User question (dynamic):
"Can I swim with these earbuds?"

Key design principles:

4.5 Evaluating a RAG system

How do you know your RAG system is good? Evaluate two dimensions:

Retrieval quality:

Generation quality:

A simple evaluation method:

Prepare 20–30 test questions with reference answers, run them regularly, and track quality over time.

Test set example:
| Question | Expected answer | Expected source |
|----------|-----------------|-----------------|
| "Which Bluetooth version?" | "5.3" | product_spec.md |
| "Can it go in water?" | "IPX5 — splash-proof, not submersible" | product_spec.md |
| "How long is the warranty?" | "12 months" | warranty_policy.md |

4.6 RAG cost analysis

Example estimate (small product knowledge base):

Volume: 50 product manuals + 500 FAQs ≈ 200 pages
Embedding cost: $0.02 (one-time)
Vector DB: $0 (local Chroma)
Monthly queries: 1,000
LLM cost: $5–10/mo (T3 fast tier)
Total monthly cost: $5–10

Against labor:
Support answers 30 product questions/day × 5 min each = 2.5 h/day
Monthly labor: 2.5 h × 22 days × $15/h = $825

ROI: ($825 − $10) / $10 = 8,150%

5. Hands-On Overview: Building a Product Knowledge Base

This section is conceptual. For the full code walkthrough, see B3 RAG Knowledge Base.

5.1 The minimal RAG system (10 lines of code)

With LlamaIndex + Chroma, 10 lines of Python get you a working RAG system:

# Conceptual code (full version in module B3)
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader

# 1. Load documents (manuals, FAQs, ...)
documents = SimpleDirectoryReader("product_docs/").load_data()

# 2. Build the index (auto chunking + embedding + storage)
index = VectorStoreIndex.from_documents(documents)

# 3. Create a query engine
query_engine = index.as_query_engine()

# 4. Ask
response = query_engine.query("Which Bluetooth version does this product support?")
print(response)
# → "According to the manual, the XB-500 supports Bluetooth 5.3..."

5.2 Build steps at a glance

Step 1: collect documents (1–2 h)
product manuals (PDF/Word)
FAQ documents
common support questions and answers
spec sheets
internal policy documents

Step 2: preprocess (30 min)
convert to one format (text/Markdown)
clean formatting issues (mojibake, stray blank lines)
verify completeness

Step 3: build the RAG system (1–2 h)
install dependencies (pip install llama-index chromadb)
configure the embedding model and LLM
load documents and build the index
test queries

Step 4: optimize and operate (ongoing)
tune the chunking strategy
tune retrieval parameters
add new documents
monitor answer quality

5.3 No-code RAG options

If you don’t want to write code, these tools ship RAG out of the box:

6. RAG Optimization Techniques

6.1 What drives RAG quality

6.2 Advanced RAG patterns (2026)

Naive RAG:
question → retrieve → generate
Simple and effective — fine for most scenarios

Advanced RAG:
question → query rewriting → hybrid retrieval → reranking → generate
Query rewriting: optimize the user's question with an LLM
Hybrid retrieval: keyword + semantic together
Reranking: reorder results with a cross-encoder
For quality-critical scenarios

Modular RAG:
question → routing → best retrieval strategy → multi-source retrieval → fusion → generate
Routing: classify the question, choose the strategy
Multi-source: query several knowledge bases at once
Fusion: merge multi-source results
For complex enterprise applications

Sources: RAG Architecture Guide 2026, RAG Systems Production Guide 2026

7. FAQ

7.1 RAG FAQ

7.2 When you don’t need RAG

8. Learning Resources

8.1 Getting started

8.2 Going deeper

9. Common Traps

9.1 Assuming RAG eliminates hallucination

RAG reduces invention from nothing. It does not eliminate “retrieved it but misread it” or “found nothing and answered anyway.” The real defense is requiring source attribution in the prompt and permitting the model to answer “not in the material.”

9.2 Copying default chunking settings

Fixed-length chunking cuts a spec table or a compliance clause in half, and the retrieved fragment naturally can’t answer the question. For e-commerce, chunking by semantic unit (one SKU, one policy, one FAQ) usually beats chunking by character count.

9.3 Building the knowledge base and never maintaining it

Product specs, platform policies, and shipping rules all change. The most common RAG failure isn’t technical — it’s that nobody updated the material for six months, which makes it more dangerous than not having it.

9.4 Using RAG for what belongs in a database

“Which SKU sold best last month” is a SQL query, not a semantic retrieval problem. Forcing structured queries through RAG is both slower and less accurate.

When this doesn’t work

• You have fewer than a few dozen documents. RAG earns its keep by finding the relevant few passages in a large body of text. With a dozen product manuals, putting all of them in the context window is simpler and more accurate — today’s context windows hold hundreds of thousands of characters, and the retrieval layer only adds a new place to fail.
• The answer needs an aggregate, not a location. “Which three of our products have the highest return rate” is not a question retrieval can answer — it returns a few similar passages, not a total. Query a database for that. Retrieval-based QA is good at “where did we say X”, not at “how much X is there in total”.
• The documents themselves are wrong or stale. RAG faithfully surfaces whatever you gave it. Leave a two-year-old fee schedule in the knowledge base and it will confidently quote two-year-old fees to a customer. Auditing the documents before you launch matters far more than tuning chunk_size.
• You are liable when the answer is wrong. A support bot replying to customers directly, compliance answers feeding a declaration — in those settings RAG’s habit of inventing when retrieval comes up empty is a real risk. Either add human review, or instruct the prompt to say it does not know and then actually test that it does (the support prompt in §7 exists for this).

10. Completion Checklist

• Can explain why AI doesn’t know your product information
• Understand embeddings (text → vector → semantic similarity)
• Know what vector databases do and the main options
• Can sketch RAG’s three-stage architecture (index → retrieve → generate)
• Can judge whether a scenario needs RAG
• Know at least one no-code RAG option (ChatGPT file upload / Claude Projects)

Complete all of the above and you understand the core technology for grounding AI in private data. Next: F4 Automation & Agents — making AI execute tasks, not just answer questions.