|
|
[[!meta title="CI usability issues"]]
|
|
|
---
|
|
|
title: CI usability issues
|
|
|
---
|
|
|
|
|
|
|
|
|
Here, we collect CI usability issues.
|
|
|
|
|
|
This effort started with [[!tails_ticket 16959]], in order to have a better
|
|
|
This effort started with tails/tails#16959, in order to have a better
|
|
|
understanding of how our current setup feels to its users.
|
|
|
|
|
|
This data will help us define our strategy for the future of our CI (e.g.
|
|
|
switching to GitLab CI, switching to Jenkins pipelines, or merely removing some
|
|
|
UX stumbling blocks without changing the big picture of our setup).
|
|
|
|
|
|
[[!toc levels=2]]
|
|
|
|
|
|
[[_TOC_]]
|
|
|
|
|
|
|
|
|
# Cumbersome navigation
|
|
|
|
... | ... | @@ -17,10 +22,10 @@ UX stumbling blocks without changing the big picture of our setup). |
|
|
|
|
|
- Jenkins does not readily expose the full CI pipeline progress & outcome
|
|
|
⇒ developers and reviewers need to track/check the result of 3 different jobs.
|
|
|
([[!tails_gitlab tails/sysadmin/-/issues/17071]])
|
|
|
([tails/sysadmin/-/issues/17071](https://gitlab.tails.boum.org/tails/sysadmin/-/issues/17071))
|
|
|
|
|
|
- CI results are not visible on GitLab MRs
|
|
|
([[!tails_gitlab tails/sysadmin/-/issues/17070]])
|
|
|
([tails/sysadmin/-/issues/17070](https://gitlab.tails.boum.org/tails/sysadmin/-/issues/17070))
|
|
|
|
|
|
# Misleading output
|
|
|
|
... | ... | @@ -35,22 +40,21 @@ UX stumbling blocks without changing the big picture of our setup). |
|
|
# Missing information
|
|
|
|
|
|
* When a Jenkins build aborts due to a timeout, no summary of the scenarios
|
|
|
that did run is generated. ([[!tails_ticket 17678]])
|
|
|
that did run is generated. (tails/tails#17678)
|
|
|
|
|
|
# Suboptimal jobs prioritizing
|
|
|
|
|
|
* The fact all jobs are treated equally causes trouble during our release
|
|
|
process: the RM sometimes has to wait a long time for the builds they care
|
|
|
about to run, while our CI resources are kept busy by other builds that
|
|
|
arguably could wait a bit longer. ([[!tails_gitlab
|
|
|
tails/sysadmin/-/issues/9760]])
|
|
|
arguably could wait a bit longer. ([tails/sysadmin/-/issues/9760](https://gitlab.tails.boum.org/tails/sysadmin/-/issues/9760))
|
|
|
|
|
|
# Very long feedback loop
|
|
|
|
|
|
As of August 2020, a full CI pipeline takes almost 7 hours to run.
|
|
|
|
|
|
This problem is tracked on [[!tails_gitlab tails/sysadmin/-/issues/16960]] and
|
|
|
[[a dedicated blueprint|blueprint/hardware_for_automated_tests_take3]], which
|
|
|
This problem is tracked on [tails/sysadmin/-/issues/16960](https://gitlab.tails.boum.org/tails/sysadmin/-/issues/16960) and
|
|
|
[a dedicated blueprint](hardware_for_automated_tests_take3), which
|
|
|
focuses on the hard facts.
|
|
|
|
|
|
Here we focus on feelings and human perception.
|
... | ... | @@ -84,6 +88,6 @@ Note: this is not about robustness problems inherent to our test suite, |
|
|
that don't depend much on _how_ we run the test suite.
|
|
|
|
|
|
* When GitLab is down or connectivity between lizard and GitLab is poor, many
|
|
|
Jenkins jobs — if not all — fail. ([[!tails_gitlab
|
|
|
tails/sysadmin/-/issues/17715]])
|
|
|
Jenkins jobs — if not all — fail. ([tails/sysadmin/-/issues/17715](https://gitlab.tails.boum.org/tails/sysadmin/-/issues/17715))
|
|
|
|
|
|
|