How the hell does no one talk about this?

Thanks for the detailed reply! :P

I'd like to converse with every part of what you pointed out -- real discussions are always exciting!

...they pay the journals, not the other way around...

Yes of course. It’s not at all relevant?

It's arguably relevant. Researchers pay journals to display their years of work, then these journals resell those years of work to AI companies who send indirect pressure to researchers for more work. It's a form of labor where the pay direction is reversed. Yes, researchers are aware that their papers can be used for profit (like medical tech) but they didn't conceive that it would be sold en masse to ethically dubious, historically copyright-violating, pollution-heavy server farms. Now, I see that you don't agree with this, since you say:

...not only is it very literally transparent and most models open-weight, and most libraries open-source, but it’s making knowledge massively more accessible.

but I can't help but feel obliged to share the following evidence.

1. Though a Stanford report notes that most new models are open source (Lynch, 2024), the models with the most market-share (see this Forbes list) are not open-source. Of those fifty, only Cleanlab, Cohere, Hugging Face (duh), LangChain (among other Python stuff like scikit-learn or tensorflow), Weaviate, TogetherAI and notably Mistral are open source. Among the giants, OpenAI's GPT-4 et al., Claude, and Gemini are closed-source, though Meta's LLaMa is open-source.
2. Transparency is... I'll cede that it is improving! But it's also lacking. According to the Stanford 2024 Foundation Model Transparency Index, which uses 100 indicators such as data filtration transparency, copyright transparency, and pollution transparency (Bommasani et al., 2024, p. 27 fig. 8), developers were opaque, including open-source developers. The pertinent summary notes that the mean FMTI company score improved from 37 to 58 over the last year, but information about copyright data, licenses, and guardrails have remained opaque.

I see you also argue that:

With [the decline of effort in average people's fact-finding] in mind I see no reason not to feed [AI] products of the scientific method, [which is] the most rigorous and highest solution to the problems of epistemology we’ve come up with thus far.

And... I partly agree with you on this. As another commenter said, "[AI] is not going back in the bottle", so might as well make it not totally hallucinatory. Of course, this should be done in an ethical way, one that respects the rights to the data of all involved.

But about your next point regarding data usage:

...if you actually read the terms and conditions when you signed up to Facebook... and if you listened to the experts then you and these artists would not feel like you were being treated unfairly, because not only did you allow it to happen, you all encouraged it. Now that it might actually be used for good, you are upset. It’s disheartening. I’m sorry, most of you signed it all away by 2006. Data is forever.

That's a mischaracterization of a lot of views. Yes, a lot of people willfully ignored surveillance capitalism, but we never encouraged it, nor did we ever change our stance from affirmatory to negative because the data we intentionally or inadvertently produced began to be "used for good". One of the earliest surveillance capitalism investigators, Harvard Business School professor Shoshana Zuboff, confirms that we were just scared and uneducated about these things outside of our control.

"Every single piece of research, going all the way back to the early 2000s, shows that whenever you expose people to what’s really going on behind the scenes with surveillance capitalism, they don’t want anything to do [with] it. The only reason we keep engaging with it is because we feel like we have no choice. ...[it] is a colossal market failure. Because it is not giving people what people want. ...everything that's inside that choice [i.e. the choice of picking between convenience and privacy] has been designed to keep us in ignorance." (Kulwin, 2019)

This kind of thing -- corporate giants giving up thousands of papers to AI -- is another instance of people being scared. But it's not fearmongering. Fearmongering implies that we're making up fright where it doesn't really exist; however, there is indeed an awful, fear-inducing precedent set by this action. Researchers now have to live with the idea that corporations, these vast economic superpowers, can suddenly and easily pivot into using all of their content to fuel AI and make millions. This is the same content they spent years on, that they intended for open use in objectively humanity-supporting manners by peers, the same content they had few alternative options for storage/publishing/hosting other than said publishers. Yes, they signed the ToS and now they're eating it. We're evolving towards the future at breakneck pace -- what's next? they worry, what's next?

(Comment 1/2)

This could be true if they were to give more weight to academic sources, but I fear it will probably treat them like any other source, so a published paper and some moron on Reddit will still get the same say on wether the Earth is round.

Hmm, that makes sense. The toothpaste can't go back into the tube, so they're going a bit deeper to get a bit higher.

That does shift my opinion a bit -- something bad is at least being made better -- although the "let's use more content-that-wants-to-be-open in our closed-content" is still a consternation.