The absolute best arcade racer to me was always NFS Hot Pursuit 2 for PS2. The physics were so much fun and the cars were a curated selection of cool.
Always loved the Project Gotham Racing series, especially 3. Tons of fun to drift in those games. The Kudos system was definitely a unique feature.
Been playing through the Ridge Racer games most recently. Damn these are just fun to play.
Sega Rally Championship will always stand out as some of the best driving physics early on.
Art of Rally and Art of drift are hella fun "zen" games with a unique art style.
I was always a sucker for some of the cash-in Fast and Furious era car fad games. Juiced, Tokyo Extreme Racer (out before all of it) Street Racing Syndicate, NFS Underground, Midnight Club 3. They're all fun but driving is always just ok.
My short list though:
Gran Turismo 4
Forza Motorsport 4 or 6 (4 is less grindy I feel)
NFS Most Wanted 2