Small talk is the equivalent of dogs sniffing each others ass. The topic isn't really the point, it's just a quick and easy way to gauge another person's mood and attitude towards you. It's a skill worth developing. And I say this as someone with social anxiety.
But most people who are invested in small talk will be giving the signals they think the other person wants, making it less useful than not talking at all.
This is coming from someone who learned how to do small talk, watches other people completely flip their personality the moment they are out of the small talk, and only uses it when necessary because everyone else does. I do avoid common topics I have no interest in, like watching sports, and avoid getting into the weeds of topics and that works well enough for anyone I would want to talk to later.
But most people who are invested in small talk will be giving the signals they think the other person wants, making it less useful than not talking at all.
I don't think this is true. When I engage in small talk, I don't see it as me bending flexibly to the conversation partner's wants. I'm testing to see if there are common overlaps that we can talk about, and talking for the sake of being entertained. If the other person turns out not to be a good conversation partner for me in that moment, I don't think anything of just moving on. I'm not trying to please them, I'm trying to enjoy myself.