A new wave of AI is poised to transform the technologies we use everyday. Trust must be at the core of how we develop and deploy AI, everyday, all the time. It is not an optional ‘add-on’. Mozilla has long championed a world where AI is more trustworthy, investing in startups, advocating for laws, a...
My mind immediately went to a horizon zero dawn like dystopia where the Mozilla AI is the only thing left protecting humans from various malevolent AIs bent on consuming the human race
I think by that point ChatGPT would be more like Apollo, keeping the knowledge of humanity. I feel like one of the more corporate AIs will go full HADES, I'm thinking Bard. It will get a mysterious signal from space that switches it's core protocol from "don't be evil" to "be evil."
Imagining the Mozilla AI as a personified Firefox and Thunderbird fighting off Cortana, some BARD (sorry) and a bunch of generic evil corporate AIs just makes me excited that Mozilla would be the one fending everyone off.
As much as I love Mozilla, I know they're going to censor it (sorry, the word is "alignment" now) the hell out of it to fit their perceived values. Luckily if it's open source then people will be able to train uncensored models
What in the world would an "uncensored" model even imply? And give me a break, private platforms choosing to not platform something/someone isn't "censorship", you don't have a right to another's platform. Mozilla has always been a principled organization and they have never pretended to be apathetic fence-sitters.
This is something I think a lot of people don't get about all the current ML hype. Even if you disregard all the other huge ethics issues surrounding sourcing training data, what does anybody think is going to happen if you take the modern web, a huge sea of extremist social media posts, SEO optimized scams and malware, and just general data toxic waste, and then train a model on it without rigorously pushing it away from being deranged? There's a reason all the current AI chatbots have had countless hours of human moderation adjustment to make them remotely acceptable to deploy publicly, and even then there are plenty of infamous examples of them running off the rails and saying deranged things.
Talking about an "uncensored" LLM basically just comes down to saying you'd like the unfiltered experience of a robot that will casually regurgitate all the worst parts of the internet at you, so unless you're actively trying to produce a model to do illegal or unethical things I don't quite see the point of contention or what "censorship" could actually mean in this context.
I fooled around with some uncensored LLaMA models, and to be honest if you try to hold a conversation with most of them they tend to get cranky after a while - especially when they hallucinate a lie and you point it out or question it.
I will never forget when one of the models tried to convince me that photosynthesis wasn't real, and started getting all snappy when I said I wasn't accepting that answer 😂
Most of the censorship "fine tuning" data that I've seen (for LoRA models anyway) appears to be mainly scientific data, instructional data, and conversation excerpts
There's a ton of stuff ChatGPT won't answer, which is supremely annoying.
I've tried making Dungeons and Dragons scenarios with it, and it will simply refuse to describe violence. Pretty much a full stop.
Open AI is also a complete prude about nudity, so Eilistraee (Drow godess that dances with a sword) just isn't an option for their image generation. Text generation will try to avoid nudity, but also stop short of directly addressing it.
Sarcasm is, for the most part, very difficult to do... If ChatGPT thinks what you're trying to write is mean-spirited, it just won't do it. However, delusional/magical thinking is actually acceptable. Try asking ChatGPT how licking stamps will give you better body positivity, and it's fine, and often unintentionally very funny.
There's plenty of topics that LLMs are overly sensitive about, and uncensored models largely correct that. I'm running Wizard 30B uncensored locally, and ChatGPT for everything else. I'd like to think I'm not a weirdo, I just like D&d... a lot, lol... and even with my use case I'm bumping my head on some of the censorship issues with LLMs.
Anything that prevents it from my answering my query. If I ask it how to make me a bomb, I don't want it to be censored. It's gathering this from public data they don't own after all. I agree with Mozilla's principles, but also LLMs are tools and should be treated as such.
If 'censored' means that underpaid workers in developing countries don't need to sift through millions of images of gore, violence, etc, then I'm for it
As an aside I'm in corporate. I love how gung ho we are on AI meanwhile there are lawsuits and potential lawsuits and investigative journalism coming out on all the shady shit AI and their companies are doing. Meanwhile you know the SMT ain't dumb they know about all this shit and we are still driving forward.
I'm so goddamn tired of "open source" turning into subscription models restricting use cases because the company wants to appease conservative investors.
Using copyrighted material for research is fair use. Any model produced by such research is not itself a derivative work of the training material. If people use it to create infringing (on the training or other material) they can be prosecuted in the exact same way they would if they created an infringing work via Photoshop or any other program. The same goes for other illegal uses such as creating harmful depictions of real people.
Accepting any expansion of IP rights, for whatever reason, would in fact be against the ethics of free software.
Using copyrighted material for research is fair use. Any model produced by such research is not itself a derivative work of the training material.
You're conflating AI research and the AI business. Training an AI is not "research" in a general sense, especially in the context of an AI that can be used to create assets for commercial applications.
Couldn't give a fuck, there's already far too much bad blood regarding any form of AI for me.
It's been shoved in my face, phone and computer for some time now.
The best AI is one that doesn't exist. AGI can suck my left nut too, don't fuckin care.
Give me livable wages or give me death, I care not for anything else at this point.
Edit: I care far more about this for privacy reasons than the benefits provided via the tech.
The fact these models reached "production ready" status so quickly is beyond concerning, I suspect the companies are hoping to harvest as much usable data as possible before being regulated into (best case) oblivion.
It really no longer seems that I can learn my way out of this, as I've been doing since the beginning, as the technology is advancing too quickly for users, let alone regulators to keep it in check.
I feel the issue with AI models isn’t their source not being open but the actual derived model itself not being transparent and auditable. The sheer number of ML experts who cannot explain how their model produces a given result is the biggest concern and requires a completely different approach to model architecture and visualisation to solve.
Coming from a company the preaches about privacy and rates privacy respecting businesses, while collecting telemetry and accepting 500M/ year to from google to promote their search engine... I'll take this as the puff up piece that is is.
The very little, basic telemetry Firefox collects can be easily disabled^1.
What alternative do you suggest to Mozilla? Reject the $500M and blowup everything they've worked so hard for decades to build? I feel like users having to click, at most, a whole 5 times to change their search engine (if they want) isn't that big of a sacrifice to have a major privacy-oriented, non-profit player in the tech sphere.
Its more so the principle. Many people that download Firefox are doing so to escape google, and if they are not born as cyber security experts they may download Firefox and continue with no real improvement to their privacy.
Secondly, the main thing you should look for is where a company gets its funding. If Mozilla gets almost 100% of its funding through google.. How much do you really expect them to push back against the data collection of their userbase?
I rank Mozilla with the likes of ExpressVPN, NordVPN, etc. They preach privacy and security against surveillance.. But its just theatre to make money in specific demographics.
Not only is telemetry easy to disable. In about:telemetry, you can see what's being send and many of these things are important to improve the user experience, make Firefox faster and also monitor privacy/security problems.
Without telemetry (use counter), how to decide whether a deprecated feature can be removed? Removing them is necessary to decrease maintenance work, be able to innovate and remove features that are less secure.
No?
The code for the model can be open-source - and that's pretty valuable. The training data can be made openly available too - and that's perhaps even more valuable. And the post-training weights for the model can be made open too.
Each of those things is very meaningful and useful. If those things are open, then the AI can be used and adjusted for different contexts. It can be run offline; it can be retrained or tweaked. It can be embedded into other software. etc. It is definitely meaningful to open source that stuff.