Build Your Own AI Coding Assistant With Access to Your Entire Codebase (2026)

Let me tell you what GitHub Copilot actually does. You type a function name, it auto-completes the body. You paste a stack trace, it suggests a fix. It looks at the current file, maybe the open tabs, and synthesizes from that. And it costs $19/month and uploads your code to Microsoft's servers on every keystroke.

Now let me tell you what it doesn't do. It doesn't know that your authentication middleware uses a custom JWT format you wrote in 2023. It doesn't know that the /api/users route calls through three service layers before touching the database. It doesn't know why that one file in lib/legacy/ is never to be touched. It has no persistent memory of your codebase. It sees one file at a time, at best a few open tabs.

That's the gap. And in 2026 you can close it, entirely locally, with open-source tools, for free.

This guide builds a real coding assistant — one that indexes your entire codebase, understands how components connect, retrieves relevant context on demand, and integrates directly into your editor. No API keys. No monthly fees. No code leaving your machine.

What We're Actually Building

Before touching a terminal, understand the architecture. This isn't "install Copilot but locally." It's a different design.

Cloud AI coding tools work like this: editor plugin → your current file → remote API → LLM → code suggestion. The LLM only sees what's in the current request. Context is bounded by what fits in one API call.

What we're building works like this:

Your editor (VS Code / Cursor / Neovim)
    ↓
Continue.dev extension (MCP client)
    ↓
Local RAG engine (Tree-sitter + vector index)
    ↓
Ollama (local LLM inference server)
    ↓
Coding model (Qwen2.5-Coder, DeepSeek-Coder-V2, or similar)

The RAG layer is the difference. When you ask "how does auth work in this project?", the RAG engine doesn't guess. It queries a vector index of your entire codebase, finds the actual relevant files and functions, and injects that context into the prompt before the LLM ever sees your question. The LLM answers with real knowledge of your code because the retrieval layer found it first.

We covered the basics of running a local LLM on your PC — that article gives you the Ollama foundation. This one builds the coding assistant on top of it.

Hardware Reality Check

Let me be blunt about this because tutorials that gloss over hardware requirements waste your afternoon.

The quantization suffix matters. q4_K_M is 4-bit quantization with medium quality preservation — cuts model size roughly in half with minimal quality loss. q2_K is 2-bit, which lets you run massive models on consumer hardware at the cost of some coherence. For code specifically, q4_K_M is the right default. Code generation degrades more noticeably at q2_K than prose generation does.

For the embedding model (the RAG layer), you need a separate smaller model running in parallel. nomic-embed-text runs fine on CPU and barely touches your resources. Don't put it on your GPU — save that VRAM for the main model.

Install Ollama and Pull Your Models

If you already have Ollama from our local LLM guide, skip to step 2. If not:

# Linux / macOS
curl -fsSL https://ollama.com/install.sh | sh
 
# Windows: download installer from ollama.com

Pull your models. Pick one coding model based on your VRAM, plus the embedding model for everyone:

# Embedding model — runs on CPU, required for RAG
ollama pull nomic-embed-text
 
# 8GB VRAM — solid baseline
ollama pull qwen2.5-coder:7b-instruct-q4_K_M
 
# 16GB VRAM — this is where it gets genuinely good
ollama pull qwen2.5-coder:32b-instruct-q4_K_M
 
# 24GB+ — Flagship. Matches cloud quality on most code tasks
ollama pull qwen2.5-coder:72b-instruct-q2_K
 
# Alternative: DeepSeek-Coder-V2, excellent for reasoning-heavy refactors
ollama pull deepseek-coder-v2:16b-instruct-q4_K_M

Verify everything runs:

ollama list
# Should show your models
 
curl http://localhost:11434/api/chat -d '{
  "model": "qwen2.5-coder:7b-instruct-q4_K_M",
  "messages": [{"role": "user", "content": "Write a TypeScript function that debounces async calls"}]
}' | jq '.message.content'

If that returns coherent TypeScript, you're running. Now let's wire it into your editor.

Install Continue.dev in VS Code

Continue.dev is the open-source VS Code extension that bridges your editor to local LLM backends. Everything runs locally — no telemetry, no cloud, no code leaving your machine.

# From VS Code command palette: Extensions → search "Continue"
# Or install via CLI:
code --install-extension Continue.continue

Once installed, open the Continue sidebar (look for the Continue icon, or press Ctrl+L). You'll see a chat panel. Now configure it to talk to Ollama:

Open ~/.continue/config.json (or use the gear icon in Continue's sidebar):

{
  "models": [
    {
      "title": "Qwen2.5 Coder 7B",
      "provider": "ollama",
      "model": "qwen2.5-coder:7b-instruct-q4_K_M",
      "apiBase": "http://localhost:11434"
    }
  ],
  "tabAutocompleteModel": {
    "title": "Qwen2.5 Coder 7B (Autocomplete)",
    "provider": "ollama",
    "model": "qwen2.5-coder:7b-instruct-q4_K_M",
    "apiBase": "http://localhost:11434"
  },
  "embeddingsProvider": {
    "provider": "ollama",
    "model": "nomic-embed-text",
    "apiBase": "http://localhost:11434"
  },
  "contextProviders": [
    {
      "name": "codebase",
      "params": {
        "nRetrievalResults": 10,
        "rerankThreshold": 0.3
      }
    },
    {
      "name": "code",
      "params": {}
    },
    {
      "name": "currentFile",
      "params": {}
    }
  ]
}

The contextProviders section is what makes this more than a local chatbot. The codebase provider is the RAG layer — it automatically retrieves relevant code from your indexed project. The code provider lets you @ mention specific files or functions. currentFile always includes the file you're editing.

Index Your Codebase

This is the part that makes everything different from Copilot. Open a project in VS Code, open the Continue sidebar, and run:

@codebase

Continue will start indexing your project — walking the file tree, parsing code with Tree-sitter, generating embeddings with nomic-embed-text, and building a local vector index in .continue/index/. On a 50,000 file codebase, expect 5-15 minutes on first run. Subsequent runs only re-index changed files.

What's Tree-sitter doing here? It's not just treating your files as blobs of text. Tree-sitter parses code into an AST — an Abstract Syntax Tree. It knows the difference between a function definition and a comment, between a class declaration and an instantiation. When it chunks your code for embedding, it splits on semantic boundaries — function edges, class edges — not arbitrary character counts. The result is that your vector index stores meaningful units of code, not arbitrary 512-character slices that might cut through the middle of a function.

This matters for retrieval quality. When you ask "how does the payment processing work?", the retrieval returns complete, coherent function bodies and class methods — not half-functions sliced at an arbitrary character count.

Once indexed, try it:

@codebase How does authentication work in this project?

Watch what happens. Continue runs a semantic search against your codebase index, finds the relevant files — your auth middleware, your JWT utilities, your session handling — and injects those as context before the LLM sees your question. The model answers with actual knowledge of your code.

The MCP Layer

Model Context Protocol is the open standard Anthropic published in 2024 that defines how AI assistants connect to external tools and data sources. It's the protocol that lets your local coding assistant talk to your Git history, your database schemas, your documentation, your issue tracker — anything you write an MCP server for.

Continue.dev implements MCP as a client. You can run MCP servers that expose your project-specific tools. Here's a minimal example — an MCP server that gives your coding assistant read access to your project's database schema:

// schema-mcp-server.ts
import { Server } from "@modelcontextprotocol/sdk/server/index.js";
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
import { Client } from "pg";
 
const server = new Server(
  { name: "schema-inspector", version: "1.0.0" },
  { capabilities: { tools: {} } }
);
 
const db = new Client({
  connectionString: process.env.DATABASE_URL
});
 
server.setRequestHandler("tools/list", async () => ({
  tools: [{
    name: "get_schema",
    description: "Get the database schema for a table",
    inputSchema: {
      type: "object",
      properties: {
        table: { type: "string", description: "Table name" }
      },
      required: ["table"]
    }
  }]
}));
 
server.setRequestHandler("tools/call", async (request) => {
  if (request.params.name === "get_schema") {
    const { table } = request.params.arguments as { table: string };
    const result = await db.query(
      `SELECT column_name, data_type, is_nullable
       FROM information_schema.columns
       WHERE table_name = $1`,
      [table]
    );
    return {
      content: [{
        type: "text",
        text: JSON.stringify(result.rows, null, 2)
      }]
    };
  }
  throw new Error(`Unknown tool: ${request.params.name}`);
});
 
await db.connect();
const transport = new StdioServerTransport();
await server.connect(transport);

Register it in your Continue config:

{
  "experimental": {
    "modelContextProtocolServers": [
      {
        "transport": {
          "type": "stdio",
          "command": "npx",
          "args": ["ts-node", "/path/to/schema-mcp-server.ts"]
        }
      }
    ]
  }
}

Now your local assistant can answer: "What columns does the users table have and which are nullable?" — and it will query your actual database schema to find out. That's not possible with any cloud coding assistant without significant custom integration work.

Building a Codebase RAG Pipeline from Scratch

If you want to understand what's happening under the hood — or build your own pipeline instead of using Continue's built-in indexer — here's the full implementation in Python. This is worth understanding even if you use Continue, because knowing the mechanics tells you how to tune retrieval quality.

# codebase_rag.py
import os
import json
from pathlib import Path
from typing import Generator
import tree_sitter_python as tspython
import tree_sitter_typescript as tstypescript
import tree_sitter_javascript as tsjavascript
from tree_sitter import Language, Parser
import chromadb
import ollama
 
# --- Tree-sitter Setup ---
 
PY_LANGUAGE = Language(tspython.language())
TS_LANGUAGE = Language(tstypescript.language())
JS_LANGUAGE = Language(tsjavascript.language())
 
LANGUAGE_MAP = {
    ".py": PY_LANGUAGE,
    ".ts": TS_LANGUAGE,
    ".tsx": TS_LANGUAGE,
    ".js": JS_LANGUAGE,
    ".jsx": JS_LANGUAGE,
}
 
# Function and class node types per language
CHUNK_NODE_TYPES = {
    PY_LANGUAGE: {"function_definition", "class_definition", "decorated_definition"},
    TS_LANGUAGE: {
        "function_declaration",
        "method_definition",
        "class_declaration",
        "arrow_function",
        "export_statement",
    },
    JS_LANGUAGE: {
        "function_declaration",
        "method_definition",
        "class_declaration",
        "arrow_function",
    },
}
 
def get_semantic_chunks(file_path: Path) -> Generator[dict, None, None]:
    """
    Parse a file with Tree-sitter and yield semantic chunks
    (functions, classes) rather than arbitrary text slices.
    """
    ext = file_path.suffix.lower()
    language = LANGUAGE_MAP.get(ext)
    
    if not language:
        # Fallback: plain text chunking for unsupported types
        text = file_path.read_text(errors="replace")
        chunk_size = 1500
        for i in range(0, len(text), chunk_size):
            yield {
                "content": text[i:i + chunk_size],
                "file": str(file_path),
                "type": "text_chunk",
                "start_byte": i
            }
        return
 
    source = file_path.read_bytes()
    parser = Parser(language)
    tree = parser.parse(source)
    
    target_types = CHUNK_NODE_TYPES.get(language, set())
    
    def walk(node):
        if node.type in target_types:
            chunk_text = source[node.start_byte:node.end_byte].decode("utf-8", errors="replace")
            # Only yield chunks that are substantive
            if len(chunk_text.strip()) > 50:
                yield {
                    "content": chunk_text,
                    "file": str(file_path),
                    "type": node.type,
                    "start_byte": node.start_byte,
                    "end_byte": node.end_byte
                }
        # Always recurse to find nested functions/classes
        for child in node.children:
            yield from walk(child)
    
    yield from walk(tree.root_node)
 
 
def embed_text(text: str) -> list[float]:
    """Generate embedding using local nomic-embed-text via Ollama."""
    response = ollama.embeddings(
        model="nomic-embed-text",
        prompt=text
    )
    return response["embedding"]
 
 
def index_codebase(root_dir: str, collection_name: str = "codebase") -> chromadb.Collection:
    """
    Walk a codebase, chunk with Tree-sitter, embed with Ollama,
    and store in a local ChromaDB vector store.
    """
    root = Path(root_dir)
    client = chromadb.PersistentClient(path="./.codeindex")
    
    # Drop and recreate for a clean index
    try:
        client.delete_collection(collection_name)
    except Exception:
        pass
    
    collection = client.create_collection(
        name=collection_name,
        metadata={"hnsw:space": "cosine"}
    )
    
    IGNORE_DIRS = {
        ".git", "node_modules", "__pycache__", ".venv", 
        "dist", "build", ".next", "coverage"
    }
    SUPPORTED_EXTS = {".py", ".ts", ".tsx", ".js", ".jsx", ".go", ".rs", ".md"}
    
    batch_docs = []
    batch_embeddings = []
    batch_ids = []
    batch_metadatas = []
    
    BATCH_SIZE = 50  # Upsert in batches for performance
    chunk_count = 0
    
    for file_path in root.rglob("*"):
        # Skip ignored directories
        if any(part in IGNORE_DIRS for part in file_path.parts):
            continue
        if not file_path.is_file():
            continue
        if file_path.suffix not in SUPPORTED_EXTS:
            continue
        if file_path.stat().st_size > 500_000:  # Skip files > 500KB
            continue
        
        print(f"Indexing: {file_path.relative_to(root)}")
        
        for chunk in get_semantic_chunks(file_path):
            chunk_id = f"{file_path}::{chunk['start_byte']}"
            embedding = embed_text(chunk["content"])
            
            batch_ids.append(chunk_id)
            batch_docs.append(chunk["content"])
            batch_embeddings.append(embedding)
            batch_metadatas.append({
                "file": chunk["file"],
                "type": chunk["type"],
                "relative_path": str(file_path.relative_to(root))
            })
            
            chunk_count += 1
            
            if len(batch_ids) >= BATCH_SIZE:
                collection.add(
                    ids=batch_ids,
                    documents=batch_docs,
                    embeddings=batch_embeddings,
                    metadatas=batch_metadatas
                )
                batch_ids, batch_docs, batch_embeddings, batch_metadatas = [], [], [], []
    
    # Flush remaining
    if batch_ids:
        collection.add(
            ids=batch_ids,
            documents=batch_docs,
            embeddings=batch_embeddings,
            metadatas=batch_metadatas
        )
    
    print(f"Indexed {chunk_count} chunks from {root_dir}")
    return collection
 
 
def query_codebase(
    collection: chromadb.Collection,
    question: str,
    model: str = "qwen2.5-coder:7b-instruct-q4_K_M",
    n_results: int = 8
) -> str:
    """
    Query the indexed codebase and generate an answer using local LLM.
    """
    # Embed the question
    question_embedding = embed_text(question)
    
    # Retrieve relevant code chunks
    results = collection.query(
        query_embeddings=[question_embedding],
        n_results=n_results,
        include=["documents", "metadatas", "distances"]
    )
    
    # Build context string with file references
    context_parts = []
    for doc, meta, distance in zip(
        results["documents"][0],
        results["metadatas"][0],
        results["distances"][0]
    ):
        relevance = 1 - distance  # cosine distance → similarity
        if relevance > 0.25:  # Filter low-relevance results
            context_parts.append(
                f"// File: {meta['relative_path']} (relevance: {relevance:.2f})\n{doc}"
            )
    
    if not context_parts:
        return "No relevant code found for this query."
    
    context = "\n\n---\n\n".join(context_parts)
    
    prompt = f"""You are a coding assistant with access to the following relevant code from this project.
Answer the question using only the provided code context. Be specific and reference actual function names, file paths, and line patterns from the code.
 
RELEVANT CODE:
{context}
 
QUESTION: {question}
 
ANSWER:"""
    
    response = ollama.chat(
        model=model,
        messages=[{"role": "user", "content": prompt}]
    )
    
    return response["message"]["content"]
 
 
# --- Main usage ---
if __name__ == "__main__":
    import sys
    
    if len(sys.argv) < 2:
        print("Usage: python codebase_rag.py <project_root> [question]")
        sys.exit(1)
    
    project_root = sys.argv[1]
    
    # Index (or re-use existing index)
    collection = index_codebase(project_root)
    
    if len(sys.argv) > 2:
        question = " ".join(sys.argv[2:])
        answer = query_codebase(collection, question)
        print(answer)
    else:
        # Interactive mode
        print("Codebase indexed. Ask questions (Ctrl+C to exit):")
        while True:
            try:
                question = input("\n> ")
                if question.strip():
                    print(query_codebase(collection, question))
            except KeyboardInterrupt:
                print("\nExiting.")
                break

Run it:

pip install tree-sitter tree-sitter-python tree-sitter-typescript tree-sitter-javascript chromadb ollama
 
# Index your project and start querying
python codebase_rag.py /path/to/your/project "How does the payment flow work?"

The key design decisions here: ChromaDB stores your vector index locally on disk, so you don't re-embed everything on every run. The Tree-sitter chunking means you're embedding complete, coherent code units. The relevance threshold (0.25) filters out garbage results that would confuse the model. And the local Ollama inference means your code never touches an external server.

Tuning Retrieval Quality

The biggest lever for RAG quality isn't the LLM — it's the retrieval. A great model with poor retrieval gives poor answers. Here's what to tune:

Chunk size and overlap. The default Tree-sitter chunking gives you function-sized chunks, which is right for most queries. For very large classes or files, you may want to add a secondary pass that splits at method boundaries within a class. A function that's 2,000 lines is one chunk that may exceed your context window.

Number of retrieved results (n_results). Start at 8. For complex architectural questions you want more context — try 12-15. For specific function lookups, 4-6 is enough and avoids polluting the context with loosely relevant code.

Re-ranking. After retrieval, you can apply a cross-encoder re-ranking step that scores each retrieved chunk against your query more precisely. The initial retrieval uses embedding similarity (fast, approximate). Re-ranking uses a model that reads the actual query and chunk text together (slower, more accurate). For production setups, add cross-encoder/ms-marco-MiniLM-L-6-v2 from Hugging Face as a re-ranking step after your vector retrieval.

Hybrid search. Combine keyword search (BM25) with vector search and merge results. Keyword search is better at finding exact identifiers — error codes, function names, specific strings. Vector search is better at concept matching. The project we mentioned earlier with ai agents crushing GitHub's infrastructure is a perfect example of why this matters at scale — the same hybrid approach is what makes production search systems handle both "show me the exact function called handlePayment" and "show me all the payment-related code."

Context Window Strategy

Your 7B model has a 32K token context window. Your 32B model has 128K. These numbers sound large until you're indexing a 200,000-line codebase and trying to fit relevant context.

The math: at roughly 3.5 tokens per word, and average code line being about 10 tokens, a 128K context window holds approximately 12,800 lines. Your entire project probably has more than that. Retrieval is how you fit relevant context into finite windows.

A useful mental model: the context window is RAM, your codebase is disk. Retrieval is the L1 cache — the fast-path to the most relevant data. Design your RAG system to maximize the relevance-to-noise ratio of what lands in the LLM's context window, not the total volume of context retrieved.

One trick that works well: include file path metadata in every chunk. When the LLM knows that the authentication code is in src/middleware/auth.ts and the user model is in src/models/user.ts, it can reason about the relationship between the two and ask follow-up retrieval queries. Continue.dev does this automatically. In your custom pipeline, it's the relative_path field in the metadata.

Running on a Remote GPU

If you're on a team and one machine has the beefy GPU, you can run Ollama as a network inference server and have everyone's editor hit it:

# On the GPU machine: configure Ollama to listen on all interfaces
sudo systemctl edit ollama
# Add:
# [Service]
# Environment="OLLAMA_HOST=0.0.0.0"
 
sudo systemctl restart ollama

Then in each developer's Continue config:

{
  "models": [{
    "title": "Team Coding Assistant",
    "provider": "ollama",
    "model": "qwen2.5-coder:32b-instruct-q4_K_M",
    "apiBase": "http://YOUR-GPU-SERVER-IP:11434"
  }]
}

One RTX 4090 serving a 32B model can handle 4-6 concurrent developers with acceptable latency. The cost of one GPU beats $19/month/developer * 6 developers ($114/month) in about 3 months. After that it's pure savings with full privacy.

For the TypeScript developers on the team who just upgraded to TS 6.0, this setup also means your local assistant can index the new .d.ts files generated by the Go-rewritten TypeScript compiler and answer questions about module resolution in your specific project. Cloud Copilot doesn't have any of that context.

The Models That Actually Work

Not all models are equal for local code assistance. Here's what I've found works in practice:

Qwen2.5-Coder series (Alibaba) — currently the best open coding models at their respective sizes. The 7B model is genuinely impressive for its VRAM footprint. The 32B model matches or beats GPT-4-level performance on most coding benchmarks. The 72B model is the best non-cloud model available in 2026 for pure coding tasks.

DeepSeek-Coder-V2 — stronger at multi-step reasoning and refactoring. Better when you need the model to understand why code is structured a certain way, not just what it does. If your workflow involves explaining architectural decisions or generating code that respects existing patterns, lean toward DeepSeek.

Codestral (Mistral) — the 22B parameter model is excellent for fill-in-the-middle autocomplete. If you want a model that's good at completing the middle of a function given the surrounding context, Codestral is the specialist. Drawback: the license is non-commercial, so check your use case before production deployment.

For the embedding model, don't overthink it. nomic-embed-text runs CPU-only, generates 768-dimensional embeddings, and the retrieval quality for code is solid. If you want higher quality embeddings at the cost of running on your GPU, mxbai-embed-large generates 1024-dimensional embeddings with better semantic precision. Switch by updating embeddingsProvider.model in Continue's config.

What You Actually Get

I'll tell you what changes when your coding assistant knows your whole codebase.

You stop writing boilerplate. Not because the model generates boilerplate — you were getting that from Copilot. You stop writing boilerplate because the model knows your boilerplate. It knows how you wire repositories to services in your project. It knows your error handling conventions. It knows the patterns you established in lib/core/ and generates code that fits them without you explaining the pattern every time.

You ask questions you couldn't ask before. "What calls the processOrder function and what arguments does each call site pass?" is a retrieval query against your codebase, not an LLM generation task. The RAG layer finds the call sites. The LLM synthesizes the answer. Without whole-codebase indexing, this question returns a hallucinated guess.

You get onboarding for free. New developer joins the team. They clone the repo, set up the local assistant, and can ask "what does this service do and where is the documentation?" and get an actual answer from the codebase, not from the model's training data.

The catch? You maintain this yourself. The indexer needs to stay current as your codebase changes. Continue handles incremental re-indexing automatically — it watches for file changes and re-embeds modified files. If you build the custom pipeline, you need to wire that up yourself. Set a file watcher or a pre-commit hook that triggers re-indexing on changes to source files.

Given the r/programming community's ongoing debate about whether AI tools actually make developers better or just faster at writing bad code, I'll say this: a coding assistant that knows your codebase is categorically different from one that doesn't. The failure mode of cloud Copilot is confident hallucination — it generates code that looks like your codebase but isn't. The failure mode of a well-indexed local assistant is retrieval misses — it tells you it doesn't know, or retrieves adjacent context. That failure mode is better.

Related reading from CoderOasis: Running a local LLM on your PC is the prerequisite foundation for everything in this guide. The AI agents breaking GitHub's infrastructure shows where this tooling is heading at scale. Amazon's Kiro AI and the security risks of shipping AI-generated code is the counterweight — understanding why local, auditable tooling matters for production code.