How do you keep docker/podman containers up-to-date?
I used to simply use the 'latest' version tag, but that occasionally caused problems with breaking changes in major updates.
I'm currently using podman-compose and I manually update the release tags periodically, but the number of containers keeps increasing, so I'm not very happy with this solution. I do have a simple script which queries the Docker Hub API for tags, which makes it slightly easier to find out whether there are updates.
I imagine a solution with a nice UI for seeing if updates are available and possibly applying them to the relevant compose files. Does anything like this exist or is there a better solution?
Sorry if it's obvious, but I don't see a way to use Matrix for notifications on their documentation and my searching is coming up blank. Do you by chance have a tutorial for this?
I use DIUN (docker image update notifier). You can watch tags with it and it will notify you when updates are available. I have it email me Saturday morning. I like it a lot more than watchtower.
This looks great. I was looking at Watchtower again a few days ago, but I don't want to auto update my containers, just get notified for updates. I usually just keep the RSS feed of the project in my feed reader, but diun looks like a proper solution. Thanks!
The beer way I’ve found is to wait till something breaks.
Message around on forums asking why I’m getting errors till someone recommends update and restart.
Since my "homelab" is just that, a homelab, I'm comfortable with using :latest-tag on all my containers and just running docker-compose pull and docker-compose up -d once per week.
This is mostly my strategy too. Most of the time I don't have any issues, but occasionally I'll jump straight to a version with breaking changes. If I have time to fix I go find the patch notes and update my config, otherwise I just tag the older version and come back later.
I've recently been moving my containers from docker compose into pure ansible though since I can write roles/playbooks to push config files and cycle containers which previously required multiple actions on docker compose. It's also helped me to turn what used to be notes into actual code instead.
Just put all commands into a bash file. Starting with ‘’docker tag’’ changing tag to something else in case I need to revert and than pull, compose up. All run by crontab weekly. In case something breaks the latest working container is still there.
It kind of depends on what are your priorities. In my experience it's usually much easier to upgrade to latest version from previous version, than to jump couple versions ahead, because you didn't have time doing upgrades recently..
When you think about it, from the development point of view, the upgrade from previous to last version is the most tested path. The developers of the service probably did exactly this upgrade themselves. Many users probably did the same and reported bugs. When you're upgrading from version released many months ago to the current stable, you might be the only one with such combination of versions. The devs are also much more likely to consider all the changes that were introduced between the latest versions.
If you encounter issue upgrading, how many people will experience the same problem with your specific versions combination? How likely are you to see issue on GitHub compared to a bunch of people that are always upgrading to latest?
Also moving between latest versions, there's only limited set of changes to consider if you encounter issues. If you jumped 30 versions ahead, you might end up spending quite some time figuring out which version introduced the breaking change.
Also no matter how carefully you look at it, there's always a chance that the upgrade fails and you'll have to rollback. So if you don't mind a little downtime, you can just let the automation do the job and at worst you'll do the rollback from backup.
It's also pretty good litmus test. If service regularly breaks when upgrading to latest without any good reason, perhaps it isn't mature enough yet.
We're obviously talking about home lab where time is sometimes limited, but some downtime usually not a problem.
Just so I understand, you're using your compose file to handle updating images? How does that work? I'm using some hacked together recursive shell function I found to update all my images at once.
Essentially you create docker compose file for services, within which you have as many containers as you want set up like you would any other compose file. You ‘docker compose pull’ and ‘docker compose up -d’ to update/install just like you would for individual docker container, but it does them all together. It sounds like others in the thread have more automated someone with services dedicated to watching for updates and running those automatically but I just look for a flag in the app saying there’s an update available and pull/ up -d whenever it’s convenient/I realize there’s an update.
Compose is the best. Way more granular control. And makes migration entirely pain free. Just ran into the case for it. Set it and forget it, use the same compose for updates.
Ideally containers are provided with a major release version tag, so not just :latest but :0.18 for all 0.18.x releases that should in theory not break compatibility.
Then you can set your Podman systemd configuration file (I use Quadlet .container files) to automatically check for new versions and update them.
Well, most projects publish their dockerfiles so you could take ans rebuild them with the tags you want. And all the building can be built into a CI/CD pipeline so you just have to make a new push with the latest versions.
I originally used this too, but in the end had to write my own python script that basically does the same thing and is also triggered by systemd. The problem I had was that for some reason podman sometimes thinks there is a new image, but when it pulls it just gets the old image. This would then trigger restarts of the containers because auto-update doesn't check if it actually downloaded anything new. I didn't want those restarts so had to write my own script.
Edit: but I lock the version manually though e.g. nextcloud 27 and check once a month if I need to bump it. I do this manually in case the upgrade needs an intervention.
Auto update with "latest" version tag, and re-pull to a specific previous version if there are problems. Got too many containers to keep up with individual versions
If you pull 'latest' and then want to roll back, how do you know what version you were in before? Is there a way to see what version/tag actually got pulled when you pull latest?
Last time it happened was with one of the newer Nextcloud updates. It was a bit of trial and error, but I eventually went back to a version that worked and I could fix the underlying issue. There should be a list of version tags either on dockerhub or GitHub that list all versions that have been pushed to live and are available to pull
I pin versions and stick to stable releases as I want stability. Everything is behind a VPN so I'm not too worried. I check them and update once a week or so.
Kubernetes with ArgoCD declarative config and then Renovate. It automatically makes prs against my config repo for container/chart versions with the change log in the description
You obviously know a thing or two about Kubernetes. I'm trying to learn. I've been at the cloud native conference, I attended the vmware tanzu course, even played with microk8s on my laptop. I still look for the "aha!" moment, when I understand the point of it all, and everything clicks into place.
However, whenever I see somebody describe their setup, I just cringe. It all just feels like we're doing simple things in an obscure and difficult way.
The technology has been here for almost a decade, and it's obviously not going away. How can I escape the misery, and start loving k8s?
For sure, just stacking turtles all the way down... 🐢 It's definitely overkill for a home lab, but I'm an infra engineer, and it's what I use daily, so setting it up was worth it because I'm already really familiar with the stack. That said, I do absolutely love having declarative setup at home because I'll sometimes go months without touching things. Before I spent the time to make it declarative, I'd frequently forget how I set certain things up and waste time redoing, or figuring out where I left off. Now I just check commit history and I'm always moving forward.
You weren't asking me, but I've used K8s professionally and my take is that K8s is only suited for business environments, ones with a good number of devs and users and complex deployment/runtime needs. You're not finding that "aha!" with K8s for self-hosting at home because, simply put, you are not the target market. It's way overkill for your needs. The one exception is if you're trying to learn it at home so you can use it in a corporate environment. In that case, go wild. But just don't expect it to make sense for most modest home lab or self-hosting needs.
I use something called What's Up Docker to check for docker updates. It integrates nicely with Home Assistant, so I made a card on my server state dashboard that shows which containers have updates available. I'll check every so often and update my docker-compose files.
Watchtower updates most containers. They never break. If it leads to some breaking, it goes to the second option.
Update script that update the whole stack from portainer webhook. This did fix the only stack that used to give me issues with watchtower. The other stack is watchtower itself.
Manual update. Only for Homeassistant. I want to make sure to know about breaking changes. So I update it when I can and I read the patch notes.
I use a combination of flux and a python app that checks out everything running on my cluster and keeps me a list of what needs some attention from upgrades and kube-clarity as well. It's more kubernetes related though.