Not much to it, The starch is essentially just a type of complex carbohydrate, and potatoes are known to carry them in abundance. So while a potato is still technically a vegetable, it's often referred to as a starch or starchy vegetable because of how much starch it contains.
It can be made with either! I think most vodka you get (at least in the US) is made with grain, but there are distillers who specifically make potato vodka, too.