tor-browser.mdwn 9 KB
Newer Older
intrigeri's avatar
intrigeri committed
1
[[!meta title="Upgrading the Tor Browser"]]
2

3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
[[!toc levels=2]]

The big picture
===============

The Tails ISO build system [[!tails_gitweb
config/chroot_local-hooks/10-tbb desc="downloads"]] a set of Tor
Browser tarballs from a location specified in [[!tails_gitweb
config/chroot_local-includes/usr/share/tails/tbb-dist-url.txt]], and
compares their hash with previously verified ones found in
[[!tails_gitweb
config/chroot_local-includes/usr/share/tails/tbb-sha256sums.txt]].

Once released officially, Tor Browser tarballs can be found in
a [permanent (?)
location](http://archive.torproject.org/tor-package-archive/torbrowser/).
However, when upgrading Tor Browser for an imminent Tails release, we
generally have to use Tor Browser tarballs that are under QA and not
officially released yet. So, we have to retrieve them from another,
temporary location, such as
<http://people.torproject.org/~mikeperry/builds/>. If we hard-coded
this temporary URL in `tbb-dist-url.txt`, then our release tag would
only be buildable for as long the tarballs stay in that place, which
at best is a few months.

To solve this, we host ourselves the Tor Browser tarballs we need, and
point to [this permanent
location](http://torbrowser-archive.tails.boum.org/) for anything that
we tag.

Still, one can set an arbitrary download location in
`tbb-dist-url.txt`, which should provide all the flexibility needed
for development purposes.

Upgrade Tor Browser in Tails
============================

40
41
Have a look at

Tails developers's avatar
Tails developers committed
42
43
44
* <https://archive.torproject.org/tor-package-archive/torbrowser/>
* <https://www.torproject.org/dist/torbrowser/>
* <https://people.torproject.org/~mikeperry/builds/>
45
* <https://people.torproject.org/~gk/builds/>
46
* <https://people.torproject.org/~boklm/builds/>
Tails developers's avatar
Tails developers committed
47
* <https://people.torproject.org/~linus/builds/>
48

49
50
and see if the desired version is available. Set `TBB_DIST_URL` to the
chosen URL, and set `TBB_VERSION` to the desired Tor Browser version, for
51
example:
52

53
    TBB_DIST_URL=https://people.torproject.org/~mikeperry/builds/4.5-build5/
54
    TBB_VERSION=4.5-build5
55

56
57
58
59
<div class="caution">
Ensure you include the "-buildN" part.
</div>

Tails developers's avatar
Tails developers committed
60
61
Fetch the version's hash file and its detached signature, and verify
with GnuPG:
62

63
    wget ${TBB_DIST_URL}/sha256sums-unsigned-build.txt{.asc,} && \
anonym's avatar
anonym committed
64
    gpg --verify sha256sums-unsigned-build.txt{.asc,}
65
66

Filter the tarballs we want and make them available at build time,
Tails developers's avatar
Tails developers committed
67
when the tarballs are fetched:
68

69
70
71
    grep --color=never "\<tor-browser-linux64-.*\.tar.xz$" sha256sums-unsigned-build.txt \
    | grep -v '\<tor-browser-linux64-debug\.tar\.xz$' \
    > config/chroot_local-includes/usr/share/tails/tbb-sha256sums.txt
72

73
Then update the URL to the one chosen above:
74

75
    echo "${TBB_DIST_URL}" | sed "s,^https://,http://," > \
76
77
         config/chroot_local-includes/usr/share/tails/tbb-dist-url.txt

Tails developers's avatar
Tails developers committed
78
79
80
81
<div class="note">
<p>
We cannot use HTTPS due to limitations/bugs in
<code>apt-cacher-ng</code>, which often is used in Tails build
82
83
environments. However, it is of no consequence since we verify the
checksum file.
Tails developers's avatar
Tails developers committed
84
85
</p>
</div>
86
87
88

Lastly, commit:

Tails developers's avatar
Tails developers committed
89
    git commit config/chroot_local-includes/usr/share/tails/tbb-*.txt \
90
91
        -m "Upgrade Tor Browser to ${TBB_VERSION}." && \
    git show
92
93
94
95
96
97
98
99
100

<div class="caution">
<p>
If this new Tor Browser is meant to be included in a Tails
release, then that's not enough: as explained above, we need to host
the corresponding tarballs ourselves, so read on the next section.
</p>
</div>

101
Sync with the upstream wrapper scripts
102
103
======================================

104
105
106
Adapt our `config/chroot_local-includes/usr/local/bin/tor-browser`
and/or
`config/chroot_local-includes/usr/local/lib/tails-shell-library/tor-browser.sh`
107
108
for recent changes made in the
[Tor Browser build Git repo](https://git.torproject.org/builders/tor-browser-build.git):
109

110
111
112
113
    git log -p \
        projects/firefox/abicheck.cc \
        projects/firefox/start-firefox \
        projects/tor-browser/RelativeLink/start-tor-browser
114

115
116
117
118
119
120
121
Then apply any relevant change, e.g. to:

 - environment variables;
 - commandline options passed to the `firefox` executable;
 - required libstdc++6 version bumps; if there's been any change upstream,
   look for `abicheck` in `config/chroot_local-hooks/10-tbb` and adjust
   that hook as needed.
122

123
124
125
126
127
128
Self-hosted Tor Browser tarballs archive
========================================

Initial setup
-------------

129
First, install [[!debpts git-annex]].
130
131

Then, make sure you have an entry for `git.puppet.tails.boum.org` in
132
your `~/.ssh/config`. See `systems/ISO_history.mdwn` in the internal Git repo
133
134
135
136
137
138
for details.

Then, clone the metadata repository and initialize git-annex:

	git clone gitolite@git.puppet.tails.boum.org:torbrowser-archive.git && \
	cd torbrowser-archive && \
intrigeri's avatar
intrigeri committed
139
	git annex init
140

intrigeri's avatar
intrigeri committed
141
You now have a lot of (dangling) symlinks in place of the files that are
142
143
available in this git-annex repo.

intrigeri's avatar
intrigeri committed
144
To synchronize your local git-annex metadata with the remote, run:
145
146
147

	git annex sync

148
149
Set up environment variables
----------------------------
150

151
152
153
154
1. Make sure you still have the environment variables defined in the
   previous section set.

2. Make `TAILS_GIT_REPO` point to the main Tails Git repository
155
156
   checkout where `tbb-dist-url.txt` is being worked on, for example:

157
        TAILS_GIT_REPO="$HOME/tails/git"
158

159
3. Make `TBB_ARCHIVE` point to your local git annex working
160
161
   copy of our Tor Browser archive, for example:

162
        TBB_ARCHIVE="$HOME/tails/torbrowser-archive"
163

164
4. Make `TBB_IMPORT_BRANCH` point to the branch where you want to
165
166
   import the new Tor Browser's metadata, for example:

167
        TBB_IMPORT_BRANCH=feature/123456-torbrowser-42.3.4
168

169
170
171
172
173
Import a new set of Tor Browser tarballs
----------------------------------------

1. Download and verify all the tarballs we need:

174
        DL_DIR=$(mktemp --tmpdir -d "tor-browser-${TBB_VERSION}.XXXXXXXXXX")
175
176
177
178
179
180
181
182
183
184
        CHROOT_INCLUDES="config/chroot_local-includes"
        TBB_SHA256SUMS_FILE="${CHROOT_INCLUDES}/usr/share/tails/tbb-sha256sums.txt"
        TBB_DIST_URL_FILE="${CHROOT_INCLUDES}/usr/share/tails/tbb-dist-url.txt"
        cd "$TAILS_GIT_REPO" && git checkout "$TBB_IMPORT_BRANCH"
        TBB_TARBALLS_BASE_URL="$(cat "${TBB_DIST_URL_FILE}" | sed "s,^http://,https://,")"
        current_branch=$(git -C "$TAILS_GIT_REPO" branch | awk '/^\* / { print $2 }')
        for branch in "$current_branch" ; do
           git -C "$TAILS_GIT_REPO" show "$branch:$TBB_SHA256SUMS_FILE" \
           | while read expected_sha256 tarball; do
              (
185
                 cd "$DL_DIR"
186
187
188
189
190
191
                 echo "Retrieving '${TBB_TARBALLS_BASE_URL}/${tarball}'..."
                 curl --remote-name --continue-at - \
                    "${TBB_TARBALLS_BASE_URL}/${tarball}"
              )
           done
           (
192
              cd "$DL_DIR" && \
193
194
195
196
              git -C "$TAILS_GIT_REPO" show "$branch:$TBB_SHA256SUMS_FILE" \
                 | sha256sum -c -
           )
        done
197

198
2. Move the tarballs into your local Git annex:
199

200
201
        cd "$TBB_ARCHIVE" && \
        mkdir "$TBB_VERSION" && cd "$TBB_VERSION" && \
202
        git annex import --duplicate "$DL_DIR/"* "$TAILS_GIT_REPO/"sha256sums-*
203
204
205
206

Commit and push your changes
----------------------------

207
208
	cd "$TBB_ARCHIVE" && \
	git commit -m "Add Tor Browser ${TBB_VERSION}." && \
209
	git annex sync && \
210
	git annex copy --to origin -- "${TBB_VERSION}"
211
212
213
214
215
216
217
218

Wait for the synchronization
----------------------------

Once you've gone through these steps, a cronjob that runs every
5 minutes will download the tarballs and make them available on
<http://torbrowser-archive.tails.boum.org/>.

219
Wait for this to happen before you proceed with the next steps.
intrigeri's avatar
intrigeri committed
220
221
222

In the meantime, you might want to import the new Tor Browser tarballs
into your `apt-cacher-ng` local cache.
223
224
225
226
227

Adjust the URL in the main Git repository
-----------------------------------------

    cd "$TAILS_GIT_REPO" && \
228
    git checkout "$TBB_IMPORT_BRANCH"
229
    current_branch=$(git branch | awk '/^\* / { print $2 }')
230
    for branch in "$current_branch" ; do
231
       git checkout "$branch" && \
232
       echo "http://torbrowser-archive.tails.boum.org/${TBB_VERSION}/" > \
233
234
            config/chroot_local-includes/usr/share/tails/tbb-dist-url.txt && \
       git commit config/chroot_local-includes/usr/share/tails/tbb-dist-url.txt \
235
236
           -m "Fetch Tor Browser from our own archive." && \
       git show
237
    done
238
239
240
241
242
243
244

Clean up
--------

	cd "$TBB_ARCHIVE" && \
	git annex drop -- "${TBB_VERSION}" && \
    rm -rf "$DL_DIR"
245
246
247
248

Update the htpdate User Agent
=============================

intrigeri's avatar
intrigeri committed
249
We want to use the same user agent in our htpdate script (see the
250
[[Time syncing design|contribute/design/Time_syncing]]
251
252
253
254
for more info on that) as in Tor Browser.

To find out the User Agent of the new Tor Browser:

intrigeri's avatar
intrigeri committed
255
1. Start Tor Browser (outside of Tails, if there is no ISO yet with the new
256
   Tor Browser)
intrigeri's avatar
intrigeri committed
257
2. Open the _Network_ tab in the _Developer Tools_ (Ctrl+Shift+E)
258
3. Load a website (e.g. <https://tails.boum.org>)
intrigeri's avatar
intrigeri committed
259
260
4. Select one of the GET requests in the _Developer Tools_
5. Scroll down to `User-Agent` in the _Request headers_ section
261
262

Now replace the User Agent in `config/chroot_local-includes/etc/default/htpdate.user-agent` with the one you found above.