Make failure to add entry to cache fatal again, and provide guidance

Since #20150 (comment 226306) we're now sure that in some cases, there is enough disk space to store the new cache entry, but still it fails with "no space left". My hunch is that we're short on inodes. So this MR adds logging to check this. And then, it makes adding to the cache fatal again, providing guidance to the version of ourselves who'll face this problem again. Hopefully they'll be better equipped to deal with it, thanks to the logs we added.

Yes, technically we could reproduce the problem (simply duplicate an existing cache entry a number of times) and know right now in which situation we are exactly. But there's little advantage in doing this now. And who knows, maybe for unrelated reasons this problem won't happen ever again.

Closes #20150 (closed)