I would go to your installed apps in settings and look for apps you don't remember installing. A quick search will tell you what they are, then remove any suspect apps.
Install a "logcat" application. You might have to enable developer mode. Then you can configure the types of logs you want and look at them immediately after you get the alert or notification to see which app sent it.
Do you have a Samsung phone? You could maybe Try downloading Good Lock and get Nice Catch if so. That tracks things like apps that woke your screen or apps that vibrated your phone. Maybe there's something in that app that might catch it?