philoko @ philoko @lemmy.ml

Posts

0

Comments

6

Joined

2 yr. ago

Yes, all browsers on iOS are

You’d think that they’d stop being a child after 2000 years.

I’m a bit out of the loop with LLMs but it depends on what you’re doing.

Last I heard, you’re going to want to use a 65b or 70b model if you want something that runs as good as GPT 3.5 but good luck with getting a GPU with enough VRAM to hold it without breaking the bank. You could offload layers to system RAM or even swap but that can come with pretty steep performance implications.

I haven’t heard of a model that’s comparable to GPT 4 but like I said, I’m pretty out of the loop. But, you’d still probably have the same VRAM and performance issues but even worse since bigger models usually is better.

All that being said, you might not need some huge model depending on what you’re doing. There’s some smaller models that can fit on consumer GPUs that can perform surprisingly well in certain situations. There’s also uncensored variants of models that won’t give you some moral lecture if you ask it for something questionable. Then there’s also the privacy aspect; I absolutely would not trust OpenAI with any personal information. I believe there’s a way to opt out of them using your personal data for training for personal accounts but you’re still trusting them with whatever information you send them.

Yeah, Oobabooga let’s you easily download models from HuggingFace and has a chat interface.

From my experience, your posts on private subs are hidden so PowerDeleteSuite can’t delete them. When they open back up, those comments and submissions suddenly appear on your profile.

TopGrade is a good one stop upgrade utility.