Building PHOTOS: Finding Memories in 200,000 Photos

I built a custom tool to manage my massive photo archive, featuring semantic AI search and smart organization, all part of my personal fIOx system.

It happened again last Tuesday, around 11:00 PM, right during one of those late-night private coding sessions that usually define my week. I wanted to find a specific shot from a trip to Ireland—something capturing the mood of the landscape on a cloudy day. I knew I had it. I’ve been photographing things for over three decades, and my archive has grown into a mountain of over 200,000 images.

More than one Solution for the “Where Is That Photo?” Problem

But finding it? That was a nightmare. I have Adobe Lightroom, which is professional-grade, but my collection is a massive collection of folders organized by import date rather than actual content. I never had the time to tag every single photo with keywords or create albums for special occasions, and I certainly didn’t want to upload 200,000 high-res files to a cloud provider just so I could search them on my phone. I felt buried under my own memories.

That frustration was the spark. As part of my ongoing journey in Building fIOx—the personal, 24/7 system I’ve been putting together on my Mac Mini—I decided to build a tool that could actually “understand” my life’s work.

A Digital Twin for My Memories

The concept behind “Photos” isn’t to replace my professional editing software, but to create a “smart mirror” of it. Instead of wrestling with massive, heavy files, this tool creates a lightweight digital twin of my entire library. It looks at my existing catalogs and copies the essential information into a lightning-fast database, including creating lightweight WebP versions of my photos.

This means I get all the benefits of a highly organized, searchable library without ever risking the integrity of my original high-resolution files. It’s a dedicated space just for browsing and finding, acting as a much faster, much smarter companion to my main storage. Thanks to Tailscale, I can even access it on my phone while on the go.

And the basic filtering options—such as by rating, date, camera, GPS coordinates (if available), and EXIF information like exposure time, ISO, focal length, and storage location—as well as the option to create albums—are all included.

Finding the “Feeling” of a Moment

The real magic—and the part that genuinely blew me away during testing—is the Intelligent Semantic Search. Most photo apps require you to type in specific tags like “dog” or “beach.” But life isn’t that organized.

With this tool, I can search using natural language, describing the vibe or specific details I remember. I can type in “yellow taxi in New York,” “sunsets at the beach with empty lounge chairs,” or even something more abstract like “melancholic mood in the forest.” Similar words are found; spelling errors are tolerated to a certain extent. Because the tool uses local AI to understand what is actually happening inside the pixels, it finds these moments instantly. And my photos still stay on my device—they aren’t sucked up by some big cloud service.

Technically speaking, I’m using SigLIP 2 (the So400m checkpoint). I chose this model because it offers excellent performance in what’s known as “zero-shot retrieval”—that is, the ability to find things that haven’t been explicitly labeled. Thanks to the special sigmoid loss function, it’s also perfect for automatically categorizing images. The entire logic runs on a PostgreSQL database with the pgvector extension, which is specifically designed to process these complex search queries extremely quickly.

Image Search Using a Picture or Drawing

I even pushed it a step further and implemented a way to search using an image itself. I can upload a photo I like, and the system finds similar shots from my own photo history. It felt like magic—a late-night win that made all those tired hours of tinkering with Claude and Python feel completely worth it.

Beyond just searching, the tool handles the tedious “housekeeping” tasks that humans hate. It automatically identifies faces so I can group photos by person, maps out my travels on an interactive map, and even detects duplicates or “burst” shots (where you take several photos in a row) so I can clean up the clutter.

Plugging into the fIOx Ecosystem

Because this tool is a part of my broader fIOx ecosystem (which, if you missed it, I explained in the overview article of this series), it doesn’t live in a vacuum. Since it’s connected to my central API/MCP—the “brain” that lets all my tools talk to each other—my LLM (a large language model, essentially a smart AI chatbot) can now “see” my photos.

I can literally talk to my AI assistant and say, “Find those photos of the last mountain holiday,” and it can pull them up. I can even instruct it to pass those photos to other tools in my system to analyze colors or prepare web-friendly versions for sharing. It turns a passive archive into an active, conversational assistant.

What’s Next?

The tool is still a work in progress. Right now, the process of importing new photos is a bit manual—I have to run a script to sync my Lightroom catalog with the new database. I’m working on making that import process fully automatic so it just happens in the background while I sleep.

I want it to be even more seamless, more intuitive, and even more integrated into my daily life. And I’ve also implemented a feature inspired by OneDrive and iPhone photos: ‘On This Day’ memories. With “dynamic albums” that automatically recompile their content from the database based on a filter rule (for example, “today’s tag,” “this month,” but with a flexible year).

And there are many, many more ideas for how to browse through my personal photo archive!