streaming has a history of being data intrusive. and buying from most online stores show itemized music receipts to the credit card company (and don't typically allow giftcards). buying in person is nice, but harder to get new music.
I ask because the easiest way to do what you’re asking is to have your local record store sell you shit and pay in cash (that you’ve laundered so the serial numbers don’t match the atm). You can even be like “I’m trying to get away from computers man, can you order me this off eBay?” And guaranteed if you spent a hundred bucks or so on used releases they’ll say “absolutely!”
Of course, you’ll stick out like a sore thumb and have a lie to keep up with, so you’ll not have any real measure of anonymity.
If you’re mechanically inclined and can work with small parts, the old Sony branded walkmans are generally good quality and have a decent supply of replacement parts. Some of the new portables have awful wow and flutter that will make it seem like that two step is a polyrhythm!
I listen on my phone in the world, cd and tape when I’m driving and on whatever at home. Today it was goat and escape-ism.