AWS Bedrock Knowledge Bases - RAG Without the Infrastructure

Every RAG tutorial makes standing up a vector database look trivial, and then you're on-call for an OpenSearch cluster at 2am because someone reindexed the wrong collection. Bedrock Knowledge Bases exists precisely to take that operational burden off your plate, at the cost of some flexibility. Here's when that trade is worth it.

What Bedrock Knowledge Bases Actually Manages For You

A Knowledge Base wraps the full RAG pipeline — ingestion, chunking, embedding, vector storage, and retrieval — behind a managed API. You point it at a data source, pick a vector store, and it handles the sync, embedding calls, and index maintenance.

aws bedrock-agent create-knowledge-base \
  --name "support-docs-kb" \
  --role-arn arn:aws:iam::123456789012:role/BedrockKBExecutionRole \
  --knowledge-base-configuration '{
    "type": "VECTOR",
    "vectorKnowledgeBaseConfiguration": {
      "embeddingModelArn": "arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v3"
    }
  }' \
  --storage-configuration '{
    "type": "OPENSEARCH_SERVERLESS",
    "opensearchServerlessConfiguration": {
      "collectionArn": "arn:aws:aoss:us-east-1:123456789012:collection/support-docs",
      "vectorIndexName": "support-docs-index",
      "fieldMapping": {
        "vectorField": "embedding",
        "textField": "text",
        "metadataField": "metadata"
      }
    }
  }'

Everything after that — re-embedding on document change, incremental sync, index compaction — is Amazon's problem, not yours.

Data Source Ingestion From S3

The most common data source is an S3 bucket, and ingestion is a matter of pointing the Knowledge Base at a prefix and triggering a sync job:

aws bedrock-agent create-data-source \
  --knowledge-base-id KB1A2B3C4D \
  --name "help-center-articles" \
  --data-source-configuration '{
    "type": "S3",
    "s3Configuration": {
      "bucketArn": "arn:aws:s3:::acme-help-center-content",
      "inclusionPrefixes": ["articles/", "faqs/"]
    }
  }'

aws bedrock-agent start-ingestion-job \
  --knowledge-base-id KB1A2B3C4D \
  --data-source-id DS9Z8Y7X6W

Ingestion jobs are incremental by default — a re-sync only re-embeds documents whose content hash changed, which matters once your help center has thousands of articles and you don't want a content edit to trigger a full re-embed of everything.

Chunking Configuration

Chunking strategy is the setting that has the biggest effect on retrieval quality, and Bedrock gives you three tiers of control:

• Fixed-size chunking — token count with overlap, the simplest option, fine for uniform prose
• Semantic chunking — splits on embedding-detected topic shifts, better for long-form docs with multiple subtopics per page
• Hierarchical chunking — parent/child chunks, where retrieval matches on small child chunks but returns the larger parent chunk for context — this is usually the best default for technical documentation

{
  "chunkingConfiguration": {
    "chunkingStrategy": "HIERARCHICAL",
    "hierarchicalChunkingConfiguration": {
      "levelConfigurations": [
        { "maxTokens": 300 },
        { "maxTokens": 1500 }
      ],
      "overlapTokens": 60
    }
  }
}

Hierarchical chunking is the one setting worth spending real time tuning — it directly determines whether your retrieved context has enough surrounding detail to actually answer the question, or just a fragment that reads correctly but is missing the caveat two sentences later.

Managed Vector Store Options

Bedrock Knowledge Bases supports several backing stores, and the right one depends on scale and what else is already in your stack:

For most teams starting fresh, OpenSearch Serverless is the pragmatic default — it's the tightest integration and needs the least manual capacity planning.

Retrieval API Usage

Once ingested, querying is a single API call — no separate embedding step, no manual similarity search:

import boto3

bedrock_agent_runtime = boto3.client("bedrock-agent-runtime")

response = bedrock_agent_runtime.retrieve_and_generate(
    input={"text": "What's our policy on prorated refunds for annual plans?"},
    retrieveAndGenerateConfiguration={
        "type": "KNOWLEDGE_BASE",
        "knowledgeBaseConfiguration": {
            "knowledgeBaseId": "KB1A2B3C4D",
            "modelArn": "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-opus-4-6",
            "retrievalConfiguration": {
                "vectorSearchConfiguration": {"numberOfResults": 5}
            },
        },
    },
)

print(response["output"]["text"])
for citation in response["citations"]:
    print(citation["retrievedReferences"])

The retrieve_and_generate call does retrieval and generation in one round trip and returns citations back to the source chunks, which you want anyway for any support or compliance-adjacent use case. If you need retrieval only — say, to feed a custom prompt template — the plain retrieve API returns just the ranked chunks.

Cost and Latency Tradeoffs vs Self-Hosted RAG

The honest comparison against running your own stack (a self-managed Qdrant or Weaviate cluster, custom chunking pipeline, embedding batch jobs):

• Latency: comparable for query-time retrieval; Bedrock adds a small amount of overhead versus a co-located self-hosted store, usually 30-80ms, which rarely matters against overall LLM generation time
• Cost at low-to-mid scale: Bedrock is cheaper because you're not paying for idle infrastructure — OpenSearch Serverless bills by OCU-hour with a low floor
• Cost at very high scale: a self-hosted cluster can become cheaper per-query once you're running near-constant load, because you're not paying AWS's managed-service margin
• Flexibility: self-hosted wins if you need a custom reranking step, hybrid sparse+dense retrieval tuning, or a chunking strategy Bedrock doesn't expose
• Operational cost: Bedrock wins decisively here — no cluster upgrades, no reindex-on-schema-change incidents, no on-call rotation for vector infra

Rough monthly cost, 500K documents, moderate query volume
────────────────────────────────────────────────────────
Bedrock Knowledge Base (OpenSearch Serverless)   ~$380/mo
Self-hosted (EC2 + managed OpenSearch cluster)   ~$610/mo
                                                  + on-call burden

When to Reach for Bedrock Knowledge Bases

• You want RAG live in days, not sprints
• Your team doesn't have dedicated infra capacity for vector database operations
• Your retrieval needs are standard (semantic search over documents), not exotic (custom hybrid scoring, multi-modal retrieval)
• You're already inside the AWS ecosystem and want IAM-based access control rather than managing a separate auth layer for a vector store

Wrapping Up

Bedrock Knowledge Bases trades some retrieval-tuning flexibility for a fully managed ingestion-to-generation pipeline, and for the majority of RAG use cases that trade is a clear win — you get a production-grade retrieval system without owning a vector database's operational lifecycle. Reach for a self-hosted stack only once you've hit a concrete limitation Bedrock can't configure around, not preemptively.