I actually might be able to provide some insight to this. My wife is Balkan and the first time I called her a "Silly Goose" she became madly upset. It turns out in her language the phrase "Glupova Gusko" (Stupid Goose) is a common insult. It is considered incredibly harsh in her culture. My guess is that the phrase "Silly Goose" is borrowed from a Slavic Language and lost its harshness when it moved into English.
It’s a common insult in many languages - with a lot of theories about origins.
You can track medival usage of this all over Europe, but I have no knowledge about other continents.
Insults in general hold more power in honor-cultures. Your grandparents might have reacted the same way she did, while you might not feel the same.
The silliest person I know was deadly serious and no-nonsense at work. Their silly side only came out among friends. Maybe you just need to befriend a goose?
Believe it or not, I am actually really serious at work, unless we are friends. I have to be super accurate all the time and I hold myself to very high standards.
I asked my goose friend what he thinks about this and he just honked. Though I suspect he didn't hear me, since he seemed to be busy balancing on his unicycle (his feet can't reach the pedals, so he has to flap his wings to balance)
This is why you have to run when you find a goose among your ducks.
I thought up Goose as a superhero for a street level setting, essentially she's a tough scrapper, easy to underestimate, and is really good at blending into crowds or workstaff. Mistique without the mutant shapechangey stuff.
i think this would probably explain the phrase "silly goose"
I imagine the whole point is to say that someone is "sillier than a goose" famously known for, not being silly at all. Arguing that you are so silly, that you look like a goose being silly.