Does it matter which lemmy server I join?

It does, to a degree.

One of the ways Lemmy sorts posts is by local. being on a bigger/busier instance will return more stuff there. Also 'All' will return stuff from communities that at least one person from your instance have subscribed, so same thing, more users more results.

Another thing to look for is instances position on defederating other communities, requirements for email/application, etc..

Ideally you will find a community that appeals to you.

I picked lemmy.fmhy.ml without knowing much. I did so because they allowed NSFW without being a NSFW community. I couldn't be happier with my decision

Good luck and enjoy the lemmyverse