I had a lot of trouble with this, but it works now, so hopefully I can help.
Just to check, when you say the calendars are set up to sync, do you mean they're set up that way in the phone's settings? Because there's also a setting in the Outlook app itself which needs to be on and it's not immediately obvious.
If you go into Outlook settings, then near the top, look for 'Mail Accounts'. You should see your email address - tap it, and you'll be taken to a screen called Account Info. The third option down is called 'Sync Calendars' and needs to be switched on.
This was what sorted it for me and allowed me to properly sync Outlook events with my phone calendar.