Commit 6ffd8e94 authored by anonym's avatar anonym
Browse files

Use a more (line) diff friendly way to store the uBlock Origin database dump.

The command I used for generating our dump was:

    echo '.dump' | sqlite3 ublock0.sqlite | \
      grep -v "cached_asset_content://cache://compiled-" | \
      awk '!/^INSERT/; /^INSERT/ {print $0 | "sort -n"}' | \
      sed 's_\\n_\\n\r\n_g' | \
      sed "/^INSERT INTO \"settings\" VALUES('\(remoteBlacklists\|cached_asset_entries\)'/"'s_,_,\r\n_g' > \

The idea is to split lists (including huge strings with escaped
newlines) into separate lines, leaving a CRLF at the end of the line
as a marker -- removing these markers completely restores the file. We
also have to sort the "INSERT" lines since uBlock Origin doesn't save
them in the same order.
parent d0ada0a3
......@@ -6,12 +6,16 @@ apt-get install sqlite3
echo "Converting uBlock database dump into sqlite blob"
mkdir /etc/tor-browser/profile/extension-data/
cp /usr/share/tails/ublock-origin/ublock0.dump /etc/tor-browser/profile/extension-data/
cat /etc/tor-browser/profile/extension-data/ublock0.dump | sqlite3 /etc/tor-browser/profile/extension-data/ublock0.sqlite
mkdir -p "$(dirname "${DATABASE}")"
rm /etc/tor-browser/profile/extension-data/ublock0.dump
# The sed expression simply means: remove all CRLF ("\r\n"). The use
# of labels is simply to make this able to remove multiple CRLF to
# create a single (long) line. In the end, this restores the
# diff-friendly dump to the original sqlite dump.
sed ':a;N;$!ba;s_\r\n__g' "${DUMP}" | sqlite3 "${DATABASE}"
echo "Created uBlock sqlite blob successfully"
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment