Estimate our needs for a container registry
While experimenting with GitLab CI (#17740) we noticed the following issues when using a container registry:
The default Docker registry (docker.io) has a limit of 100 pulls in 6h for anonymous users and 200 pulls in 6h for authenticated users (doc. We've been reaching the anonymous limit and this has been causing (Docker) jobs to fail.Solved by adding a local registry mirror.
- Many of our current CI jobs install dependencies inside a Docker image during the job, which might be too slow and non-deterministic (tails#18169).
- We've filled the VM disk once already with container images and had to increase disk space to avoid jobs from failing (#17824 (closed)).
- ... more? (please list here!)
If at some point we decide to use GitLab's container registry, we need to estimate mid-term disk space needs (tails#18169 (comment 165346)). Another option is to use our own container registry, and this option would also benefit from such estimation.
Current resource usage
After some months of use, on Feb 10 2022 we have the following situation in the GitLab Runner VM which also runs a proxy registry:
- 2.5 GB storage of dangling Docker images, reclaimed with:
docker system prune
- 6.0 GB storage of remaining Docker images after the above pruning.
- 7.0 GB storage of dangling Docker volumes, reclaimed with:
docker volume rm $( docker volume ls -f dangling=true -q )
- 3.5 GB storage base Debian VM image (yes, we have a libvirt runner!).
- 4 dangling VMs:
- each launched with 3 GB RAM but actually using a total of ~700 MB (summing all of them).
- They use
.qcow2images which will only grow as needed, each currently occupying a few hundred KB.
Specific to the registry proxy:
docker-registry(proxy) service was down since Nov 25 2021 when the VM was rebooted and the service failed to start. (We don't have monitoring for this yet as it's not in production yet.)
- The only image in the registry (proxy) is