This is going to sound unbelievable, but aside from buying the pillow, this happens to me with both shoulders involuntarily. I've had to uncomfortably sleep in a chair for too long, else risking a dislocated shoulder.
I kinda learned to sleep on my back, but I hate it. Physical therapy has helped, but I always wind up forgetting to do the exercises at some point. And yeah, I mostly just sleep horribly.
You know how some people complain they go to the doctor because it hurts when they do something and the doctor's advice is "Don't Fucking Do That Then?"