Honestly not that big of a deal. When something is hyped up users who either didn't or barely used Twitter (X) probably joined in. It boosted their initial numbers but again they fell off. They'll grow as unique content grows. That happens with everything, as will the case be with Lemmy. Growth isn't linear.
I suppose it isn't linear but I suspect going from massive insane explosion in numbers to an 80% loss in a matter of weeks is pretty unusual. I think that growth was largely driven not by hype but by the automatic linking with other Zuckernedia properties.