"Can i ask you for a favor?" Is it rude to say no?

Some random dude walked up to me while I was waiting outside a food place for my food and asked me this question.

I said "no, I dont give out random favors" and something along the lines of that's sus.

Is it rude to say no to random dude that asks for a favor?