This is part 2 in a pair of posts about building a heavy metal search bar; part 1 is here.
Last week I wrote about trying to build a simple search bar to look up bands in Metal Archives's database, and ran straight into a combination of bot blockers and no API. I got a proof of concept working locally on my network, but wanted to get it over the final hurdle of it being usable from anywhere.
A couple of days later, I heard from Metal Archives's webmaster: I’d reached out after finding an unofficial (albeit broken) API. They were aware the API might not be working, and said I was welcome to scrape their site myself, provided I didn’t kill their servers. An engineer I know suggested I scrape their data, save it into a database on my machine, and query that scraped data directly. I shared this with Claude, who said it was the best architecture idea of them all (humans are still sometimes the best problem solvers).
So that’s what I did - I built a scraper to slowly download their band name, subgenre, and country fields overnight (first time I'd heard of caffeinating my computer), and when I woke up the next day, info on 195K+ bands was in an SQLite database on my computer.
Then I built an API to query that data directly, deployed via Fly.io, and here it is - https://metal-search.fly.dev/
It’s a small but very specific thing I wanted - one big metal search bar I can use to look up bands from anywhere. Next, I’ll add some more details like city, styling (it has to be either black or bright pink, right?), a custom domain, and maybe I’ll set the scraper up to re-download the data weekly so it’s always fresh-ish.
