Knowledge layer

Give your AI a memory of your writing

Every document in MarkNest is split into heading-aware chunks, embedded with OpenAI text-embedding-3-small, and indexed in pgvector. Your AI assistant retrieves the exact passage instead of guessing.

Start writing free View API docs

marknest_search_chunks

{
  "query": "pgvector HNSW index",
  "limit": 5
}

200 OK · hit[0] Retrieve

{
  "id": "01J..chunk",
  "document_id": "01J..doc",
  "document_title": "Roadmap 11",
  "document_slug": "roadmap-11",
  "heading_path": "Roadmap 11 > pgvector chunking",
  "text": "HNSW index for cosine similarity over 1536-dim
           embeddings, tuned for recall at low cardinality...",
  "score": 0.91,
  "source": "vector"
}

Chunk, embed, retrieve

A three-stage pipeline turns Markdown into a queryable knowledge base. Each stage is observable, encrypted at rest, and idempotent.

Chunk

A heading-aware Markdown chunker walks the CommonMark AST, keeps code blocks intact, splits long paragraphs with overlap, and merges tiny fragments. Each chunk keeps its heading path so retrieval stays navigable.

Embed

Each chunk is vectorized with OpenAI text-embedding-3-small (1536 dimensions, no dim reduction) and written to pgvector with an HNSW index. The chunk text stays AES-256 encrypted at rest; only the vector is plain.

Retrieve

Hybrid search runs a vector leg (pgvector cosine, top-40) and a keyword leg (Meilisearch, top-20), then fuses them with Reciprocal Rank Fusion (k=60). The result is chunk-level hits with scores and source attribution.

Built for retrieval, not chat

MarkNest is the knowledge layer — your AI app does the talking. We expose retrieved context via REST and MCP, never a hosted LLM. Your token spend stays between you and your model provider.

Hybrid search

Vector similarity catches semantic matches; keyword search catches exact terms. RRF fusion balances both, so a search for "HNSW index" finds the chunk that actually says "HNSW index for cosine similarity".

Chunk-level retrieval

POST /api/v1/search/semantic returns hits with excerpt, heading path, document, score, and source (vector / keyword / both). GET /api/v1/documents/{doc}/chunks/{chunk} fetches a single chunk by id.

Two MCP tools

marknest_search_chunks and marknest_get_chunk, gated by the rag:read ability on your API key. Drop them into any MCP-ready assistant.

Quota aware

Daily query and monthly embedding-token limits keep usage predictable. Check /api/me/rag-usage anytime. Partial-fit: when embedding quota runs low, the pipeline embeds what it can and leaves the rest pending.

Wire your AI to your archive

Semantic search and the rag:read ability are available on Pro and Max plans.

Start writing free