Building an Internal AI Assistant for Your Company Wiki

To build a secure internal AI assistant that answers employee questions from a company wiki without leaking proprietary data, use a Retrieval-Augmented Generation (RAG) architecture. This approach indexes wiki content into vector embeddings, retrieves relevant chunks matching an employee's prompt, and passes those chunks to a Large Language Model (LLM) to generate grounded answers.

1. Architecture Overview

A functional enterprise RAG setup consists of four modular layers:

• Document Loader: Pulls raw text or Markdown from your wiki platform (e.g., Confluence, Notion, or GitHub Wiki).
• Chunking & Vectorization: Splits documents into optimal text windows and generates vector embeddings using an embedding model.
• Vector Store: Index database (such as ChromaDB, Pinecone, or Qdrant) that stores vector representations for fast semantic lookup.
• LLM Pipeline: Constructs a strict prompt with retrieved context and passes it to an LLM (e.g., GPT-4o-mini or a hosted open-source model).

2. Ingesting and Chunking Wiki Data

Install the required Python packages first:

pip install langchain langchain-openai langchain-community chromadb

The code below loads Markdown files exported from a wiki, splits them into 1,000-character chunks with an overlap to maintain context continuity, and indexes them into ChromaDB using OpenAI embeddings.

from langchain_community.document_loaders import DirectoryLoader, UnstructuredMarkdownLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import Chroma

# 1. Load wiki documents from export directory
loader = DirectoryLoader("./wiki_data", glob="**/*.md", loader_cls=UnstructuredMarkdownLoader)
raw_docs = loader.load()

# 2. Chunk text to fit within model context windows
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=150)
doc_chunks = text_splitter.split_documents(raw_docs)

# 3. Vectorize and persist locally
vector_db = Chroma.from_documents(
    documents=doc_chunks,
    embedding=OpenAIEmbeddings(model="text-embedding-3-small"),
    persist_directory="./chroma_db"
)

3. Implementing Retrieval and Answer Generation

Query the vector store for the top matching document chunks and instruct the model to answer only using the provided context. Set the temperature to zero to minimize hallucinated responses.

from langchain_openai import ChatOpenAI
from langchain.chains import create_retrieval_chain
from langchain.chains.combine_documents import create_stuff_documents_chain
from langchain_core.prompts import ChatPromptTemplate

# Load existing vector database
vector_db = Chroma(
    persist_directory="./chroma_db", 
    embedding_function=OpenAIEmbeddings(model="text-embedding-3-small")
)
retriever = vector_db.as_retriever(search_kwargs={"k": 4})

# Enforce grounded responses via strict system prompt
system_prompt = (
    "You are an internal wiki assistant. Answer employee questions based strictly "
    "on the provided context below. If the answer is not present in the context, "
    "reply: 'I cannot find that information in the company wiki.'\n\n"
    "Context:\n{context}"
)

prompt = ChatPromptTemplate.from_messages([
    ("system", system_prompt),
    ("human", "{input}"),
])

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
document_chain = create_stuff_documents_chain(llm, prompt)
rag_chain = create_retrieval_chain(retriever, document_chain)

# Execute query
result = rag_chain.invoke({"input": "What is our remote work technology stipend policy?"})
print(result["answer"])

4. Production Deployment Considerations

• Automated Data Sync: Rather than full database rebuilds, configure webhook listeners (e.g., Confluence REST API webhooks) to upsert single documents into the vector store whenever wiki pages are created or edited.
• Access Controls: Vector searches bypass wiki layer permissions by default. Store access control lists (ACLs) as document metadata in the vector database and apply metadata filters to ensure employees only retrieve context they are authorized to view.
• Privacy Controls: Ensure zero-data-retention agreements are active with commercial API providers, or host open-source LLMs (e.g., Llama 3) on private cloud infrastructure to prevent proprietary leaks.

Need this done fast? order a RAG assistant on FreelanceHunt.