Overview

Andrej Karpathy shared a setup that caught my attention: Claude connected to a local Obsidian vault, acting as a personal knowledge agent. You talk to it, it stores notes, links ideas, builds a graph you can navigate. Not a SaaS product. Not a subscription. Just your own files with an intelligence layer on top.

I tried it. It worked well.

Then I tried to use it outside of Claude Desktop, built something to solve that, used it for two weeks, and realised I’d created more work than I’d saved.

This is that story.

---

The Setup

Karpathy’s approach is straightforward. Claude Desktop connects to a local folder via MCP. The folder is your Obsidian vault. Claude can read files, write files, search content, create linked notes. You get a conversational interface to your own knowledge graph.

It works exactly as described — inside Claude Desktop, on your local machine.

The moment you want to use it from another device, you’re on your own. Claude Desktop is a local application. Its MCP connection is local. There’s no built-in way to reach it from outside.

---

The First Solution

My first instinct was to build around the constraint.

My laptop was already running the knowledge system. If I turned it into an HTTP server, requests from my phone would hit the server, it would process them using the Anthropic SDK and the same Obsidian files, and responses would stream back through a chat interface I’d build on top.

For private remote access I used Tailscale — a mesh network that creates secure tunnels between your own devices without exposing anything publicly.

I built the whole stack. HTTP server, streaming chat interface, Tailscale configuration, SDK integration. Tested it from my phone. It worked.

Then I started using it daily.

Every bug that appeared was infrastructure debt, not product debt. The streaming interface had edge cases I hadn’t handled. The server needed to stay awake. The SDK dependency meant any pricing or access change upstream immediately affected whether I could use my own notes. Each fix revealed something else that wasn’t mature enough.

I wasn’t improving my knowledge system. I was maintaining the scaffolding around it.

Two weeks in, I stopped and asked a different question.

---

The Right Question

Instead of asking “how do I fix this server?”, I asked: “does this problem already have a solved form?”

I didn’t want to reinvent the wheel. I’d already done that once and it hadn’t gone well.

That question led me back to MCP — Model Context Protocol. I’d been using it as a Claude Desktop feature. It’s actually Claude’s native protocol for connecting to local tools and files. Any MCP server that exposes the right tools can connect to Claude directly — including from the web interface, from any device, without a custom server sitting in between.

I had been building a custom bridge when the bridge was already part of the protocol.

---

The Replacement

Once I understood what I actually needed to build, it took a day.

That’s the part worth paying attention to. The build wasn’t slow — understanding what to build was the hard part. Once that was clear, AI did most of the coding. An MCP server is not a complex piece of software when you know exactly what it needs to do.

The server has one job: expose my Obsidian vault to Claude. Read files, write files, search content, create notes, update frontmatter. Claude handles everything else — the conversation, the intelligence, the interface.

What this replaced:

• The custom HTTP server and all its maintenance surface
• The Tailscale tunnel and its configuration
• The direct Anthropic SDK dependency
• The streaming chat interface I’d built from scratch

What didn’t change: my laptop needs to be on. That was always true. The difference is that when it is on, there’s nothing between me and Claude except one small local process doing exactly what it’s supposed to do.

I host mine on Cloudflare. Set up the connection once in Claude’s MCP settings. Works from any device I use Claude on.

---

What This Actually Taught Me

Two things.

First: bugs in infrastructure you built yourself are a signal, not just a todo list item. Each bug in the custom server was telling me the abstraction was wrong. I was patching symptoms instead of reading the signal.

Second: when AI does most of the coding, the bottleneck shifts entirely to understanding. Knowing what to build — the right abstraction, the right protocol, the right scope — is where the time goes now. The implementation follows quickly once that’s clear.

The custom server took weeks to build and two weeks to outgrow. The MCP server took a day and does more with less.

---

The System Today

The MCP server is open source on GitHub: github.com/DhyeyShah45/knowledge-os-mcp

Point it at any local Obsidian vault, connect it to Claude via the MCP settings, and you have a conversational interface to your own knowledge graph. No Claude Desktop required. No custom server. No tunnel.

My current setup:

• Obsidian vault on my laptop with Karpathy’s folder structure (wiki, concepts, entities, sources, queries, raw)
• MCP server running locally, connected to Claude via the web interface
• The vault grows with every conversation — notes linked, context accumulated

When the machine is on, it works. That’s the whole thing.

The build took a day. Knowing what to build took two weeks to figure out.

That’s usually how it goes.