We don't have this problem in Australia, where a sandwich is narrowly understood as being between two slices of bread cut from a loaf.
If it's on a bread roll it's not a sandwich it's a roll. If it's on a burger bun it's not a sandwich it's a burger. A sub is not a sandwich, it's a sub. A hot dog cannot be considered a sandwich, nor could a cake or anything else that's clearly not a sandwich.
Sandwich cake is already a term that means the same thing as layer cake. The classic combination of two layers of Victoria sponge with strawberry jam and whipped cream in between is called a Victoria Sandwich. Anyone arguing that a layer cake isn't a sandwich is just illiterate, not a defender of semantic specificity.
In my house most baked goods are eaten without a fork, one small sliver/square at a time, while standing at the counter and repeatedly saying, "I'm just going to have one small bite."
If the cake layers don't get your fingers messy and don't crumble when you hold it, I'll allow it. It also has to be small enough to be able to take a bite out of all the layers at once.
Is that the definition of a sandwich, or is there something about 'sanwhich' that transcends its constituent parts? Could 'sandwich' be a cluster of different properties that, when considered as a whole, become 'sandwhich'? I think to get to the heart of this 'sandwhich' question, we need not look at the sandwhich but instead at 'cake'. What is 'cake' and do those propertie exclude sandwhich? What common aspects do cake and sandwhich have, and are both of those elements essential?
Words aren’t isomorphic to their dictionary definitions—words had commonly-accepted meanings long before the existence of dictionaries. Dictionary definitions are just an attempt to come up with a heuristic for identifying things as instances of the term in question, but they’re never perfect—and the real-world usage is ontologically prior.
If the dictionary definition of sandwich fails to distinguish cakes from sandwiches, it’s just an imperfect definition (like all definitions are)—and we can leave it at that.
Also, I think you might be hard-pressed to find anyone who would consider sugary cake sponge to be "bread". (Something something Ireland's supreme court ruled that Subway's bread isn't legally bread for tax purposes due to its high sugar content)
It'd be like layering up some sourdough slices with sweet sugary icing slathered between them, most people probably wouldn't call that a cake.