It's definitely illegal to download some things. Copying audio to audio recording media is legal because of a levy we pay on the media, but software, movies, books are a different story.
What constitutes "audio recording medium" is still a question not completely answered by the courts and it seems that most copyright owners are not willing to test it in court and more than happy to just threaten with options to settle before legal action is taken and see who flinches.
Meanwhile the authorities have stated in the past that going after downloaders of any copyrighted material is not a priority unless they are also distributing copyrighted works.