Follow my series on building fIOx, a custom, privacy-first AI orchestration system running locally on my Mac Mini. No commercial tools, just full control.
It was about 1 AM on a Thursday. I was sitting at my home computer, the only light coming from my monitor, nursing a lukewarm coffee and staring at a fragmented mess of browser tabs, half-finished automation workflows, and several different online subscriptions. I had a specific task in mind—something involving a quick transcription and a summary of a few notes—but instead of a seamless flow, I found myself jumping between three different paid tools, logging in and out, and wondering where exactly my data was being stored and who was training their models on my private thoughts.
I felt like a digital nomad living in a rented apartment where the landlord kept a spare key to every room. I wanted a home of my own. That was the moment fIOx was born.
One Hub, Three Doors
fIOx (my working title, a clever fox that can handle input and output) isn’t a commercial product you can download from an app store or find as an open source on Github. It’s a personal, self-built orchestration system that lives on a Mac Mini at my home. It runs 24/7, acting as a central brain for all my digital tasks.
The architecture was the most important part of the design. I didn’t want to build a tool that only worked in one place. Instead, I designed fIOx around a “three doors” philosophy. Every time I add a new capability to the system, it automatically becomes available through three different interfaces:
- The API: A core communication layer that allows different software programs to talk to fIOx—essentially a digital doorway for other apps to send requests and get answers.
- MCP (Model Context Protocol): This is a way to connect my system directly to other (local) AI assistants—think of it as giving the AI a set of hands to actually do things rather than just talk about them.
- The Web UI: A clean, browser-based interface—a visual dashboard I can access from any device – desktop or mobile – to interact with the system like a human would.
The Privacy Imperative
As much as I love the convenience of modern cloud tools, I grew tired of the “convenience tax” on privacy. I didn’t want my personal contacts, my financial summaries, or my private notes living on someone else’s server.
By building fIOx locally, I have total sovereignty. Because I use Tailscale (a secure private network), I can reach my Mac Mini from my phone or laptop anywhere in the world, but the data never leaves my control. It’s the best of both worlds: the power of the cloud with the privacy of a local hard drive.
A Glimpse Under the Hood
Over the last six months, fIOx has grown from a few simple automations into a sprawling ecosystem of specialized tools. I’m not a professional developer, so I’ve been building this piece by piece, often late at night after my day job.
The sheer scale of what it can do has surprised even me. Because of my “three doors” architecture, every single one of these capabilities is instantly usable via the Web UI, through an API, or as a tool for my AI assistants. All based on many Python scripts with FastAPI, React/TS and FastMCP.
To give you an idea of the depth we’re talking about, here is a glimpse into the many modules currently living inside fIOx:
Creative & Media Engine
- LLM Chat: A central hub for talking to various AI models.
- Image Generation: Creating visuals from text prompts.
- Image Analysis: Image descriptions, image-to-text workflow.
- Text-to-Speech (TTS): High-quality voice generation, including voice cloning and a voice and character database.
- Music Generation: Creating original musical compositions.
- Audio Editing: Tools for trimming, mixing, and processing sound files.
- Stem Separation: Breaking a song down into its individual parts, like vocals and drums.
- Music Library Management: Full control over my local music collection and Jellyfin integration.
- Sounds: A vast, searchable database of sound effects and ambient textures.
Personal Life & Knowledge Management
- Reader: A personal news and RSS reader that can even turn articles into a custom podcast. Read more!
- Photos: A central hub for managing and organizing my personal images. Read more!
- Diary: Personal notes
- Work: Time tracking and payroll assistance for my full-time job
- Obsidian Integration: Directly reading from and writing to my personal knowledge vault (Look up what Obsidian is right away—otherwise you’ll miss out on something!).
- Contacts: A dedicated system for managing personal contact information.
- Tasks: A sophisticated manager for daily to-dos, projects and long-term goals – for work and private life.
- Finances: A deep, privacy-first personal finance tracker with automated categorization.
- Body: Visualization and analysis of my health metrics recorded via my smartwatch. Read more!
- Places: A Map and database of locations, ranging from hiking trails to coffeehouses and accommodations
- Cookbook: A recipe manager that connects to my grocery lists and even suggests meals based on what I have.
- Shopping: Managing grocery lists and food master data.
- Self-Image: A unique module that builds a structured “self-image” by analyzing my own notes and conversations.
Intelligence & Data Processing
- Transcription: Converting spoken audio into written text.
- OCR: Extracting text from images and scanned documents.
- PDF Intelligence: Reading, rendering, and extracting data from PDF files.
- Web Crawling: Deeply scanning websites to extract meaningful content.
- Web Downloading: Automatically pulling content or files from the web.
- Defuddler: A tool to clean up messy web data into a usable format.
- Geocoding: Turning addresses into precise map coordinates and vice versa.
Home & Environment
- Local Radio: A custom AI-DJ system that mixes music and news through my home speakers.
- Smart Home: Integrating sensor data and managing home maintenance logs – accompanying Home Assistant.
- Weather: Real-time weather data and long-term atmospheric tracking.
- BusyBar: Controlling a physical LED display in my office to show notifications.
Utility & System Tools
- Files: A small file browser for locally generated or downloaded files
- Snippets: A lightning-fast system for managing text templates, keyboard shortcuts and AI prompts.
- Job Queue: A serial task planner that handles heavy background jobs while I sleep.
- Notification Hub: A central station that collects alerts from all other modules and sends them where I need them.
- News: News aggregator for use in personal podcasts and knowledge management systems
- Photo Editor: A suite for quick fixes, retouching, and AI-powered background removal.
- Colors: Color Palettes and Design Tools
- Document Printing: Controlling print jobs directly from the hub.
It’s becoming a true digital shadow—a system that knows my preferences, manages my data, and performs tasks on command. And everything can be interlinked!
How a Hobbyist Built This
I want to be very clear about one thing: I am not a professional coder. I have a full-time job, and my “coding time” is usually what’s left over at the end of a long day.
I didn’t build fIOx by spending years in programming textbooks. I built it with some basic understanding of programming, by being curious and, crucially, by using AI coding assistance. Using Claude Code as a pair programmer allowed me to describe what I wanted, test it, and iterate until it worked. If you have a clear goal and the patience to test your ideas and if you’re good at describing things, I truly believe you can build something like this too. This series, Building fIOx, is my attempt to document that journey.
Lessons from the Midnight Sessions
The journey hasn’t been without its stumbles. I’ve learned that building a system that is both powerful and stable requires a lot of discipline. I’ve learned that a “simple” idea often reveals much deeper complexities—like how hard it is to correctly sync a local music library or how much care is needed when letting an AI write to your files. I’ve learned that the best architecture is one that remains modular.
What’s Next?
This is just the beginning. fIOx is a living project. I have plans to improve each tool while I’m using it and as further refine the AI’s ability to act as a proactive assistant using and choosing the right tools rather than just a reactive tool. And if I find that a feature or a new tool is missing, I simply add one, brainstorm a bit with Claude, have it suggest some options, and then, as a “senior developer”, decide which direction to take.
I’m excited to take you along for the ride. I’ll be breaking down the specific tools, the architecture, and the logic behind the build in the coming weeks.