Downloading all archive.org metadata
Downloading all archive.org metadata
I'd love to know if anyone's aware of a bulk metadata export feature or repository. I would like to have a copy of the metadata and .torrent files of all items.
I guess one way is to use the CLI but this relies on knowing which item you want and I don't know if there's a way to get a list of all items.
I believe downloading via BitTorrent and seeding back is a win-win: it bolsters the Archive's resilience while easing server strain. I'll be seeding the items I download.
Edit: If you want to enumerate all item names in the entire archive.org repository, take a look at https://archive.org/developers/changes.html. This will do that for you!
You're viewing a single thread.
I just found out you can get all metadata with
ia metadata ID > metadata.json(replace ID with gamefaqs_txt in example). So from there you could extract any information too, if you know how to handle json. (Edit: Just load the metadata.json in your browser to see a better formatted list.)I like to pipe my json to
python -m json.toolfor quick formatting in the terminal.Take a look at jq, it's a really nice tool for handling json in the terminal, also gron for searching json