"fuse" implies that the CPU will stop working when it is overclocked, this seems to be more of a mechanism for AMD to let them know that the reason the CPU is not working anymore is because it was overclocked and fried.
No, this means something else in chip design. For example, an AVR microcontroller can be configured by blowing some fuses. Here is an introduction: https://www.ladyada.net/learn/avr/fuses.html
Understandable but this is not a fuse in common usage of the word, which is used to break a circuit to protect against over-current. Rather it's an part that changes state irreversibly (much like a fuse would) when something happens. There is no implication that it would cut off the power to the CPU in this sense.
But in the world of electronics fuses and circuit breakers exist to trip when too much voltage is applied to protect the circuit. That's their generally agreed upon definition.
A fuse is just an electronic component. It can be used for circuit protection, but it doesn't have to be. For example, a transistor doesn't have to be an amplifier, a resistor doesn't have to be for dimming bulbs, etc.