Looking for some selfhosted webscanner/scraper
Looking for some selfhosted webscanner/scraper
hi, i'm kinda out of ideas -maybe you know some selfhostable solution.
I want to monitor some of my pages for gdpr compliance and have reports on that.
So far i tried selenium container with k6, i tried selenium locally with some Python automation and even gave phantomJS with webXray a try. In any case sth. is missing...like selenium wont grab the thirdparty cookies, webxray code has outdated Python and so on.
You guys have any suggestion on how i could selfhost my own automated toolset to check entire webpages for all cookies and thirdpartyscripts? Ppl make changes which might have impact on the gdpr compliance and i dont want to check manually. How would you do it selfhosted?
The EDPS open-sourced a cookie inspection tool:
oh wow...how did Google not suggest this to me? thanks mate.
edit: got i running...works like a charm. just great. thanks so much.
You are welcome :)
You could try Puppeteer, but I don't know if you can check third party cookies.
Also I can recommend using Apache Airflow for automation. It is like a fancy crontab with a webui and email notifications, and lots of other stuff.
thanks. was thinking about airflow or n8n for automation but i still cant find a way to check the thirdpartycookies. someone wrote firefoxwebdriver could do it but you'd need to extract from the profile an stuff...nothing really worked.