The indie web needs a search engine

Follow me on a journey. This post is essentially a brain dump, but there’s a point and an idea somewhere, I think.

This whole thing started with a post by Cassidy Williams.

She talks about how everyone metaphorically disappeared from the Internet after “social media™.” Personal blogs and websites became harder to come by, and human curation was replaced by algorithms and this “For You” page nonsense.

When I say “I don’t know where everyone went,” I know everyone’s out there surfing the web, of course, but it feels like it’s a different place now. When the algorithms are determining everything we should be seeing, it’s a much less personal internet.

It’s a great read; please check it out.

It got me thinking and sharing this idea on Mastodon:

Can we just make a new indie web search engine that uses tags and requires people submit their own site to be a part of it? Literally just make a page with a search bar and type in whatever kind of blog/site you’re looking for. Ship it.

The replies came in, and they pointed to examples of this or variations of it. Matt Stein even shared a link to their list in an article.

Some of these archives are what I had initially imagined, but they’re overwhelming in many ways, too. Landing on a page of thumbnails, links, or tags isn’t the most user-friendly. There’s also no source of truth to all of it, no standout example for others to follow. It’s potentially hundreds (or thousands?) of sites spread across dozens of lists.

So, I return to my original question: could we not create a single engine that these sorts of websites could hook into with a focused, minimal, user-friendly design akin to early (functional) Google? Could it federate? What about ActivityPub? Is there a scalable way to retain human curation?

I don’t have answers to these questions. I hope that something like this will prompt a discussion about it. Chris McLeod shared a post of his along the same lines from a few years ago. We can’t be the only ones thinking of this, right?

Several people have pointed me toward searchmysite.net, which I had in mind. I ran a couple of searches and found some compelling sites already. I’m excited to dig into it more.