Bilinguals, does chatGPT sound different in other languages?
I have a theory that it should have a very different "personality" (probably more like writing style) depending on language because it's an entirely different set of training data
In English chatGPT is rather academic and has a recognisable style of writing, if you've used it a bit you can usually get hints something was written by it just by reading it.
Does it speak in a similar tone, with similar mannerisms in other languages? (where possible, obviously some things don't translate)
I don't know a second language well enough to have natural conversation so I'm unable to test this myself, and may have worded things awkwardly from a lack of understanding
In two languages I'm learning, German and Chinese, I've found it to suffer from "translationese". It's grammatically correct, but the sentence structure and word choice feel like the answer was first written in English then translated.
No single sentence is wrong, but overall it sounds unnatural and has none of the "flavor" of the language. That also makes it bad for learning - it avoids a lot of sentence patterns you'll see/hear in day to day life.
As a native German speaker I agree that ChatGPT is very English-flavored. I think it's just because the sheer amount of English training data is so much larger that the patterns it learned from that bleed over into other languages. Traditional machine translations are also often pretty obvious in German, but they are more fundamentally wrong in a way that ChatGPT isn't.
It's also somewhat cultural. The output you get from ChatGPT often sounds overly verbose and downright ass-kissing in German, even though I know I wouldn't get that impression from the same output in English, simply because the way you communicate in professional environments is vastly different. (There is no German equivalent to "I hope this email finds you well", for example.)
No single sentence is wrong, but overall it sounds unnatural and has none of the "flavor" of the language.
I've also found that it's often contextually wrong. Like it doesn't know what's going on around it or how to interpret the previous paragraph or even the previous sentence, let alone the sentence two pages back that was actually relevant to the sentence it's now working on.
Well probably because it does not know what's going on around it. It only knows the words. It can't interpret the words, only guess what is the most likely answer word by word.
Bard can't speak my language Tulu. I teach it some phrases in romanized letter, because the script is yet to be added in Unicode and might take another ten years. So far, only some improvements.
Here's an example:
Prompt: "Eerna poodar enchina?"
Exact translation: "Your name what?"
Normalized translation: "What is your name?"
Output:
"Eerna poodar enchina?" is a Tulu phrase that means "Whose house is this?". It is typically used to ask for directions or to find out who owns a particular house.
Here are some other ways to ask for directions in Tulu:
"Namma poodar enchina?" (Our house is where?)
"Nanna poodar enchina?" (My house is where?)
"Enda poodar enchina?" (Their house is where?)
"Enda poodar ninna?" (Whose house is this?)
"Enda poodar yen?" (Whose house is that?)
Expected output:
"Eerna poodar enchina?" is a Tulu phrase that means "What's your name?". My name is Bard, an AI assistant.
As you can see, everything about this is so wrong. It also mixes Kannada with Tulu. And none of this is intelligible - it's all gibberish. They're not even the same language.
Bard also sucks at Hindi/Urdu translation. I've asked it to translate "Bheegi Bheegi Raaton Mein", which is a pretty old Bollywood song, but it just fails. And even if it does after carefully crafting the prompt, it fails to capture the beauty of the lyrics in the actual source. I'm not good with French, although I've learnt about it before, so I've not bothered trying.
For now, folks are using the Kannada script. But I don't use it, because, let's just say that it is kinda similar to the French-Occitan situation. By the way, here is the developing script that will be added to Unicode soon.
Not all languages have scripts, and not all scripts are on Unicode (yet). So in this example Tulu speakers would use either the Kannada or the Latin script, both of which are on Unicode.
If you ask ChatGPT to communicate to you in a different writing style it can do a decent job of doing so. It will also respect requests to decrease verbosity and formality. The default writing style is some kind of specific configuration they have made for it, it’s not a fundamental characteristic of it.
I think the misunderstanding here is in thinking ChatGPT has "languages". It doesn't choose a language. It is always drawing from everything it knows. The 'configuration' hence is the same for all languages, it's just basically an invisible prompt telling it, in plain text, how to communicate.
When you change/add your personalized "Custom Instructions", this is basically the same thing.
I would assume that this invisible context is in English, no matter what. It should make no difference.
DeepL prompted a change in career choice for me, honestly. I was initially looking into finding work as a translator, since Cantonese is an in-demand language, but (while it is still not perfect) I have seen massive improvements in translation tech over time, and DeepL was my breaking point that helped me realize "Okay, maybe this can all be automated in the future".
While plain direct translation might be automated (though not necessarily because some things just don't translate), localization is a whole different deal. Can't speak much for Cantonese because I can't speak it, but as an Arabic speaker I can't see an AI being able to translate from Arabic to English as well as a human can anytime soon.
I can't tell apart the quality or "flavour" in English from Spanish. Spanish is my first language though, if that tells you anything. IMO the performance in those two languages is the same, with the caveat that I have used it only for generic purposes (writing resumes, rephrasing stuff)
If you give it instructions in English, then switch to Spanish does it continue to follow them in Spanish?
(As in if you ask it to play the character of John the cheese merchant then ask it what its name is in Spanish, does it respond in Spanish with the correct name?)
I haven't tried anything complicated, but it does switch languages when you do. I've only tried GPT 3.5 though, and only with prompts that "ended" in one answer (not something like asking the AI to play characters or answer in a certain way, but questions that can be answered in a single message)
It's kind of weird, but it can be way more specific in Spanish. I mostly use English for programming or tech related questions; Spanish for research and when its response in English isn't relevant or it's less informative than I expected.