DemifiendQueen @ DemifiendQueen @lemmy.world

Posts

2

Comments

10

Joined

2 mo. ago

I know, I don't need to prompt an entire book as lore, I am just asking for 15-20k.

Short answer: you are cooked. The current old language model (it's in the process of being replaced, but we don't know when) has a 4k context window token, aprox 2 pages of text. This is the what the model can remember. Best option is to use the non-fancy AI Chat (https://perchance.org/ai-chat) put less than 1k tokens in your character description or even better just put the essential, and put the rest of your lore in the Scenario & Lore and pray to the Old Gods you don't exceed 3k-4k characters or you will experience a new universe of roleplaying...the goldfish memory experience. I love Perchance but I gave up months ago about doing roleplay with strong scenarios and world building because with 4k tokens it's simply impossible. We all hope the new model will have at least 10k tokens window so you will have the bare minimum to create cool worlds and scenarios without having to describe each character in one line of text. More than 20k tokens will be fucking amazing...but one only can dream.

I know, but if a model can't handle, let's say 10k tokens, expanding the current shitty 4k tokens to 10K will mean nothing, that's why it's important to have a model with a bigger token number. About the newest model being Llama 3 is a long story. Check other posts mentioning the update; the owner talked about that about half a year ago.

You'll have to keep waiting. I am also thrilled to test the new text generator...but for now still the same old one, so he hasn't finished the update.

Well, then I have good news for you, the chat is down because it is being upgraded after 1.5 years with the new Llama 3. So you just need to wait and be happy because if we are lucky and it is Llama 3.3, we could have 128k tokens context instead of the shitty 4k tokens or at least 10k or 20k that would make it possible to finally have lores and world building, so goodbye to goldfish memory issues. But "let's not get ahead of ourselves". lol Speaking about running LLM locally. I have an RTX 3060, and it takes a minute to get an answer, which is why I generate images locally with comfy and use Perchance for roleplay..so you are fucked.

Yeah, there are already models with millions of tokens like Gemini. I've been thinking about that a lot lately, and I think that will be fucking amazing. Imagine losing yourself in the world of your favourite game or novel, interacting with all the characters, creatures, and places. I think I will become a crazy old cat lady lol

Well, I hope it's 3.3, after two years of waiting, and with the image generators using Chroma and Flux. I can't complain about a free service anyway. It will be a bittersweet victory; having 2x context will surely improve the roleplay experience, but it will still be impossible to create lore and world-building unless we have at least 50-60k context tokens. And yes, the actual model is amazing, but it's being held down by the context window; you cannot create a deep lore or word building, and worse of all, the chat forgets the key events around 3-4 pages of convo, having to rely on summaries that miss A LOT of important elements.

Thanks! I can't wait. Just imagine going from 4k tokens to 128k tokens. This is gonna revolutionize free chatbots.

Oh, that's great. So I was right. I'm not crazy, then. Is a new version coming out? I heard they were gonna update it in July, but it never happened.

No way this is llama 3: https://perchance.org/ai-chat Unless there are other "official" chatbots. This chat has a very short memory. I'm sure it is Llama 2; it would make sense, since I suspect it has around 4k token context, while Llama 3 has 128k. Maybe I am missing something. Are there other chats like this that use more advanced LLMs?