Rationale:

 - Right now:

   - These pages' Markdown will be rendered when browsing our
     repository in the GitLab web interface

   - The .md extension is more common nowadays: GitHub, GitLab, and various
     other major players have settled on it. Let's get used to it.

 - If we decide to migrate our blueprints to GitLab (#18079),
   this will be a necessary first step.

HTTP_mirror_pool.md

+**Ticket**: [[!tails_ticket 7161]]
+
+[[!toc levels=3]]
+
+# The plan
+
+See the corresponding [[design document|contribute/design/mirrors]].
+
+<a id="speed"></a>
+
+# Speed
+
+This is mainly for [[!tails_ticket 10295]]. We keep this information
+around because it provides potentially useful data wrt. the reasons
+why we picked some mirrors, but not others, for the fallback DNS pool.
+
+## Fast & reliable enough mirrors
+
+i.e. those that I've seen provide good speed and that have had no
+reliability issue in the last N months.
+
+Note: measurements done from lizard are capped to 100Mbps due to
+upstream network configuration, so they can barely be used to reliably
+compare those fast mirrors with each other. For measurements done from
+Germany, upstream network should not be the limiting factor for most
+practical purposes here.
+
+* 5.45.108.219 aka https://tails.mirror.metalgamer.eu/tails/ (Germany):
+  - one glitch in August, 2016
+  - from lizard: 8.21 MB/s, 6.87 MB/s, 7.48 MB/s
+  - from D.C.: 10.1 MB/s, 9.84 MB/s, 10.1 MB/s
+  - from Germany: 37.7 MB/s, 43.4 MB/s, 37.2 MB/s
+  - from France: avg. 21.1 MB/s, stdev 4.1 MB/s
+  - from Netherlands: 50.4 MB/s, 41.3 MB/s, 43.7 MB/s
+* 85.93.216.116 aka https://tails.c3l.lu/tails/ (Luxembourg):
+  - from lizard: 6.58 MB/s, 6.72 MB/s, 3.73 MB/s, 5.52 MB/s, 2.97 MB/s, 5.31 MB/s, 4.46 MB/s, 4.50 MB/s, 3.15 MB/s
+  - from D.C.: 8.76 MB/s, 8.82 MB/s, 9.51 MB/s
+  - from Germany: 34.7 MB/s, 34.9 MB/s, 31.3 MB/s
+  - from France: avg. 14.5 MB/s, stdev 3.4 MB/s
+  - from Netherlands: 54.0 MB/s, 52.7 MB/s, 51.7 MB/s
+* 195.154.14.189 aka https://16.dl.amnesia.boum.org/tails/ (France):
+  - being moved to another IP (163.172.39.92 aka https://28.dl.amnesia.boum.org/tails/)
+    so we're not adding it yet
+  - from lizard: 5.08 MB/s, 5.25 MB/s, 6.26 MB/s, 6.33 MB/s, 6.17 MB/s
+  - from D.C.: 4.65 MB/s, 7.21 MB/s, 7.01 MB/s
+  - from Germany: 22.4 MB/s, 21.6 MB/s, 22.6 MB/s
+  - from France: avg. 25.4 MB/s, stdev. 1.5 MB/s
+  - from Netherlands: 17.2 MB/s, 17.5 MB/s, 18.4 MB/s
+* 5.104.106.180 aka https://dl2.crypto-rebels.de/tails/ (Germany):
+  - from lizard: 7.08 MB/s, 5.23 MB/s, 5.46 MB/s, 5.09 MB/s, 4.45 MB/s, 5.72 MB/s
+  - from D.C.: 7.58 MB/s, 7.98 MB/s, 7.09 MB/s
+  - from Germany (from the same network): 24.6 MB/s, 17.6 MB/s, 18.4 MB/s
+  - from France: avg. 15.7 MB/s, stdev. 2.1 MB/s
+  - from Netherlands: 38.0 MB/s, 37.9 MB/s
+* 212.110.161.69 aka http://mirror.bytemark.co.uk/tails/ (UK):
+  - from lizard: 5.31 MB/s, 6.62 MB/s, 4.61 MB/s, 6.70 MB/s, 6.34 MB/s, 6.26 MB/s
+  - from D.C.: 7.65 MB/s, 6.68 MB/s, 7.57 MB/s
+  - from Germany: 15.5 MB/s, 17.1 MB/s, 16.1 MB/s
+  - from France: avg. 10.4 MB/s, stdev. 2.7 MB/s
+  - from Netherlands: 25.2 MB/s, 66.3 MB/s, 43.9 MB/s
+- 198.145.20.143 and 149.20.37.36, aka mirrors.kernel.org
+- 208.80.154.15 aka mirrors.wikimedia.org
+* 169.229.226.30 aka https://mirrors.ocf.berkeley.edu/tails/ (California)
+  - from lizard: 10.3MB/s, 11.0MB/s, 11.2MB/s
+  - from D.C.: 8.23 MB/s, 7.45 MB/s, 8.50 MB/s
+  - from Germany: 5.01 MB/s, 6.25 MB/s, 5.90 MB/s
+  - from Netherlands: 16.0 MB/s, 12.7 MB/s, 14.7 MB/s
+
+## Too slow mirrors
+
+* 62.201.161.88 aka http://tails.mirror.iphh.net/tails/ (Germany):
+  - from lizard: 2.67 MB/s, 1.84MB/s, 1.82MB/s, 2.44MB/s
+  - from Germany: 56.7MB/s, 32.1MB/s, 13.7MB/s
+  - from France: avg. 11.6 MB/s, stdev 3.7 MB/s
+* 178.217.184.32 aka https://tails.uk.to/tails/ (Poland):
+  - from lizard: 4.96 MB/s, 4.96 MB/s
+  - from Germany: 17.1 MB/s, 18.9 MB/s, 16.7 MB/s
+  - from France: avg. 8.5 MB/s, stdev 2.1 MB/s
+* 176.9.38.37:
+  - from lizard: 2.81 MB/s, 2.74MB/s, 3.05MB/s, 2.74MB/s
+  - from Germany: 43.0MB/s, 22.1MB/s, 7.41MB/s
+  - from France: avg. 11.7 MB/s, stdev 2.6 MB/s
+* 195.154.188.146: 3.69 MB/s
+* 83.212.104.246:
+  - from lizard: 3.90 MB/s
+  - from France: avg. 4.8 MB/s, stdev 2.4 MB/s
+* 188.138.127.35 aka https://tails.bl0m.de/tails/: perf. varies too much
+* 45.33.79.99
+  - from France: avg. 4.3 MB/s, stdev 0.6 MB/s
+* 80.241.222.98 aka http://dl3.crypto-rebels.de/tails/ (Germany):
+  - from lizard: 9.18 MB/s, 3.90 MB/s, 6.62 MB/s
+  - from Germany: 7.83 MB/s, 8.00 MB/s, 7.54 MB/s
+* 213.136.84.245 aka https://dl1.crypto-rebels.de/tails/ (Germany):
+  - from lizard: 7.10 MB/s, 8.46 MB/s
+  - from Germany: 9.74 MB/s, 7.99 MB/s, 7.60 MB/s
+  - from France: avg. 6.5 MB/s, stdev. 2.4 MB/s
+* 81.7.10.29 aka https://tails.ybti.net/tails/ (Germany):
+  - from lizard: 6.28 MB/s, 5.17 MB/s, 5.26 MB/s
+  - from Germany: 7.92 MB/s, 6.04 MB/s, 6.92 MB/s
+  - from France: avg. 5.2 MB/s, stdev 1.5 MB/s
+* 96.126.119.95 aka https://tails.interpipe.net/tails/ (USA):
+  - from lizard: 6.10 MB/s, 7.04 MB/s
+  - from Germany: 4.99 MB/s, 4.62 MB/s, 4.59 MB/s
+  - from France: avg. 3.5 MB/s, stdev 0.1 MB/s
+* 5.135.66.221 aka http://24.dl.amnesia.boum.org/tails/ (France):
+  - from lizard: 3.27MB/s, 2.77MB/s, 2.89MB/s
+  - from Germany: 6.22MB/s, 6.93MB/s, 5.05MB/s
+  - from France: avg. 10.2 MB/s, stdev 1.7 MB/s
+* 151.80.190.129 (France):
+  - from lizard: 2.69MB/s, 1.24MB/s, 1.27MB/s
+  - from Germany: 2.40MB/s, 2.60MB/s, 4.42MB/s
+  - from France: avg. 7.0 MB/s, stdev 0.6 MB/s
+* 158.36.190.173 (Norway):
+  - from lizard: 3.17MB/s, 3.44MB/s, 2.44MB/s
+  - from Germany: 24.4MB/s, 23.1MB/s, 23.5MB/s
+  - from France: avg. 7.0 MB/s, stdev 1.1 MB/s
+* 192.42.116.116 aka http://192.42.116.116/tails/ (Netherlands):
+  - from lizard: 4.36 MB/s, 6.45 MB/s, 5.94 MB/s, 6.53 MB/s
+  - from D.C.: 3.72 MB/s, 2.80 MB/s, 2.86 MB/s
+  - from Germany: 45.0 MB/s, 45.5 MB/s, 38.0 MB/s
+  - from France: avg. 16.8 MB/s, stdev 2.0 MB/s
+  - from Netherlands: 89 MB/s, 94.8 MB/s, 88.0 MB/s
+* 141.138.141.28 aka http://25.dl.amnesia.boum.org/tails/ (Netherlands):
+  - from lizard: 3.35MB/s, 9.07MB/s, 6.00MB/s, 5.35 MB/s, 4.74 MB/s, 3.97 MB/s
+  - from D.C.: 5.82 MB/s, 6.37 MB/s, 7.13 MB/s
+  - from Germany: 16.7MB/s, 27.9MB/s, 24.5MB/s
+  - from France: avg. 11.9 MB/s, stdev 2.3 MB/s
+  - from Netherlands: 21.5 MB/s, 21.9 MB/s, 23.3 MB/s
+
+## Not reliable enough mirrors
+
+i.e. mirrors that have had issues at least once in the last 6 months;
+let's not include them in the fallback DNS pool:
+
+* 5.196.175.179
+* 77.70.69.9
+  - from France: avg. 0.3 MB/s, stdev 0.1 MB/s ⇒ **TODO** remove from the pool?
+* 80.90.43.162
+* 84.106.196.237
+* 86.59.119.84
+* 109.239.48.152
+* 137.226.34.46:
+  - from lizard: 798 KB/s, 9.80MB/s, 2.08MB/s, 1.37MB/s
+     connection closes in the middle of the download pretty often
+  - from Germany: 8.73MB/s, 10.4MB/s, 12.2MB/s
+  - from France: avg. 11.1 MB/s, stdev 2.5 MB/s
+* 141.138.136.78
+* 144.76.14.145
+* 149.202.98.175
+* 178.32.220.171
+* 192.99.131.144
+* 198.199.103.96
+* 212.47.229.219
+
+# Initial research
+
+See [[HTTP_mirror_pool/archive]].
+
+<a id="HTTPS"></a>
+
+# HTTPS mirrors
+
+We've already switched all our mirrors in the Javascript mirror-pool, handled
+by mirror-pool-dispatcher to HTTPS, but not all of our fallback mirrors
+([[!tails_ticket 12833]]).
+
+## Current problem space
+
+Round-Robin pool
+
+* we point to different IPs
+* round robin incompatible with different CNAMES
+* round robin uses IPs → incompatible with SSL certs
+* Asking mirror OPs to create SSL certs themselves and keep them updated is not
+  practicable.
+* Links to dl.a.b.o on website & UDFs point to the round robin. (used for
+  example on https://tails.boum.org/install/expert/usb/index.en.html)
+
+* Website, DAVE2 and IUKs use Javascript based mirror-pool-dispatcher.
+* Hardcoded URLs on the website need to be accessible & HTTPSified without
+  Javascript
+
+## Possible solutions
+
+### Server based solution
+
+We ruled this solution out when we first based the mirror-pool-dispatcher on
+Javascript. Likely, we'd want to avoir recreating such a complicated solution
+even if we will have to host our website ourselves and have this technical
+possibility.
+
+### One-mirror-only solution
+
+A very stable and big mirror should become the only fallback for non-JS users
+and the expert/wget installion method.
+
+* → We ditch the round-robin
+* → We monitor this server more often so that we can change it if ever it becomes inaccessible.
+
+## Todo now
+
+* deploy in lockstep on our live website:
+  - change fallback_download_url_prefix in mirror-pool-dispatcher [u]
+  - change all instances of http://dl.a.b.o → https://mirrors.wikimedia on our website [u]
+     - except in UDFs
+* ensure Tails 3.7 gets the updated mirror-pool-dispatcher submodule [i]
+* ensure Tails 3.7 gets an updated `tails-perl5lib` package (`lib/Tails/MirrorPool.pm`) [i]
+* prepare a branch in iuk.git that updates UDF generation code (replace dl.a.b.o with mirrors.wikimedia) [i]
+* keep the fallback DNS pool running: it's still used by Tails Upgrader and we "support" skipping an upgrade (from 3.6 to 3.7) so it must remain working until 3.6 users can upgrade directly to 3.8
+* prepare a branch against mirrors.git to document the new setup and drop the obsolete crap
+* prepare a branch against tails.git to update the design doc
+
+## Whenever we want
+
+* tell wikimedia.org admins about our plans (before or after the change, whatever) [u]
+* update the documentation for mirror operators in a dedicated Git branch: delete the part about dl.a.b.o [u]
+* prepare a branch against mirror-pool.git that drops support for the DNS fallback pool [i]
+* prepare a branch against puppet-tails.git that drops support for the DNS fallback pool [i]
+
+## When releasing Tails 3.7 [bertagaz]
+
+* all UDFs for upgrades must still have dl.a.b.o because Tails 3.6 and older
+only support that (nothing special to do for that, just follow the release
+process doc)
+
+## When releasing Tails 3.8 [i]
+
+* all UDFs for upgrades from 3.6 must have dl.a.b.o
+* all UDFs for upgrades from 3.7 must have mirrors.wikimedia
+
+## A few weeks after Tails 3.8 is released
+
+* drop the dl.a.b.o fallback pool
+* merge the branch into iuk.git
+* merge the branch into mirror-pool.git
+* merge the doc branch into mirrors.git
+* merge the doc branch for mirror operators into tails.git
+* merge the updated design doc branch into tails.git
+* merge the branch into puppet-tails.git