I never owned one, but played plenty of my friends N64’s. The controller has three different handle positions, so you had to move your hand from one spot (for pressing directional buttons) to another spot to use the joystick.
I’m sure eventually you’d get used to it and not think twice about it. But it was definitely a unique controller design
The three-prong layout was certainly an idea someone had. Having an analog stick on a console controller was innovative at the time, but the implementation of putting it on its own grip so you had to choose between it and the D-pad didn't go so well. There were games that used the D-pad, but most titles published for the system were the newfangled 3D games that were best played with the newfangled analog stick, so in practice most players held the right-hand 2/3rds of their controllers. There were a lot of games where having occasional access to the D pad would have been nice; imagine Majora's Mask with the transformation masks and ocarina bound permanently to the D-pad so you wouldn't have to keep menuing to replace C-button items, especially late in the game.
Sony very quickly studied what was right and what was wrong with Nintendo's approach and they created the Dualshock, which was almost entirely perfect.
How many shooters were there for the platform? There was Goldeneye, Doom64, and...?
Sure they made it work, but...adding the analog stick to the controller was pure innovation at the time, sure. But the vision for implementation was so fundamentally flawed that a bodge job from Sony bolting two analog sticks to their existing controller was 90 times better.