so I get dark matter is special and all, but how can they say it might be dark matter instead of just a regular star? like how are they able to tell the difference
At the moment it doesn't look like they can. The paper is looking for two things that could make the objects "dark stars". The stars would look like point objects because they would be below JWST's angular resolution. The second is spectroscopic data which the writers don't have:
We note here that the spectra for those four objects, obtained in ref. 28, do not yet confidently identify any spectral lines, as they are too noisy (S/N ∼ 2). Follow-up spectroscopy is required in order to determine the presence of emission/absorption features.
The paper's conclusion summarizes what they are looking for in the lines:
we made predictions for the spectra of those SMDS candidates and suggested smoking gun signatures such as the He II λ1640 absorption line, a feature expected for all SMDSs but not for Pop III/II galaxies. We further note that the spectra of SMDS and early galaxies differ for wavelengths above ∼5 μm, so that future observatories (beyond JWST) might be able to differentiate the two types of objects in this way
EDIT:
I will add from the paper:
There are a set of undetermined parameters that control the formation and evolution of a Dark Star, and ultimately, its observable properties.