flypig.co.uk

It's been a week since I last posted a diary entry. It turns out that not writing a diary entry everyday is almost as hard as writing one. If I'm not wandering aimlessly around the house trying to find a purpose, I find I'm sitting on the sofa quietly mumbling gdb backtrace nonsense to myself.

Thankfully there are still some tasks to do, addressing review comments and reviewing others' commits. I'll talk about these happenings today. But I also want to share some retrospective analysis of the project: what went well, what went badly and how the whole process felt as an experience. I'll be expanding on all that in a future post.

First up I want to highlight the really nice fix from from Vlad G (vlagged) that's fixed certificate overrides. When the browser accesses a site using HTTPS but receives a bad certificate, it'll present the user with an error page. This is to avoid the user mistakenly believing that the data being sent from the site, or that they're sending back to it, is being done so securely (i.e. using a channel that maintains integrity and confidentiality).

There are still some sites that don't support HTTPS which provide no more security than a site with a broken or bad certificate. But the latter is worse in that could give the user a false sense of security. If there's a failure it could be an attack. Exposing this to the user in an overt way is critical for them to make an informed decision about how to proceed.

That means that Firefox typically offers the option to view the site anyway, as long as the warning page has been prominently presented. It's hard to imagine anyone could miss the bad certificate page given the way it's presented on Sailfish OS.

Changes to the underlying API however mean that the ESR 91 upgrade broke the option to bypass the certificate check. With this broken code, selecting the option to "view the page anyway" caused the browser to return the same certificate warning page rather than actually going to the site. That's because the button is supposed to record a "certificate override" stating that the browser should ignore the bad certificate. But the method to perform this has changed parameters, so it was throwing up an error rather than recording the site details.

Here we can see the blame for the interface file where the new parameter was added by R. Martinho Fernandes:

$ git blame -c --date=short \
    security/manager/ssl/nsICertOverrideService.idl -L 84,113
[...]
345598abc1d6f (Masatoshi Kimura 2019-06-11)  void rememberValidityOverride(
                                                     in AUTF8String aHostName,
e368dc9c853cf (Ehsan Akhgari  2012-08-22)            in int32_t aPort,
7ea1bcd018a32 (R. Martinho Fernandes 2021-06-01)     in jsval aOriginAttributes,
61d16d4cd9588 (kaie@kuix.de   2007-10-03)            in nsIX509Cert aCert,
e368dc9c853cf (Ehsan Akhgari  2012-08-22)            in uint32_t aOverrideBits,
f2e2a270fb9a4 (kaie@kuix.de   2007-11-19)            in boolean aTemporary);

If we check out the commit 7ea1bcd018a32 that introduced the change, we can see it takes us to differential revision D91962:

$ git log -1 7ea1bcd018a32
commit 7ea1bcd018a32a8e98216cf9f675c49d582950ca
Author: R. Martinho Fernandes <bugs@rmf.io>
Date:   Tue Jun 1 06:55:07 2021 +0000

    Bug 1597600 - make certificate overrides depend on origin attributes 
    r=keeler,geckoview-reviewers,smaug,agi
    
    Differential Revision: https://phabricator.services.mozilla.com/D91962

This was noticed by Vlad who dug down into the code and found the source of the error, fixing it by adding the missing parameter. It's great detective work and a nice fix.

As I write this I've approved the change myself and hopefully it'll be picked up and reviewed by Jolla so that it can be merged swiftly in to the ESR 91 branch.

Besides this I also received really nice review feedback from sailors Raine (rainemak) and Mal (mlehtima) in relation to my gecko-dev pull request. This pull request is the one that actually performs the upgrade from ESR 78 to ESR 91. The upgrade takes the form of redirecting a submodule to a different upstream commit so is a super-simple change. The bigger part is the set of patches that then has to be applied on top of the upstream code to make it all work. This is all in the pull request as well.

Raine and Mal clearly spent some time going through the commits and I'm really grateful for their suggestions. The result was better preference handling and several commits being merged or removed entirely. On Mal's request I also rebased the code to bring in recent changes needed to support Jolla's updated SDK. The totality of these changes all make for a much cleaner commit history and impoved code.

This pull request, along with the pull requests for the sailfish-browser and sailfish-components-webview repositories, are now awaiting a second review from the Jolla team. As far as I'm aware I've completed the changes requested, but there may still be a few more review rounds to go before they get merged in.

The other change this week has been to the package tarballs available from my website. These are very much still testing packages and you should read all of the info in the installation instructions before you use them. Nevertheless these new packages should provide some nice improvements over the previous ones. The main changes are:

1. The user agent patch has now been applied, so problems with pages serving desktop rather than mobile variants should now be largely resolved.
2. Vlad's HTTPS fix is included so you can now view pages with broken TLS certificates if you so choose.

If you're still on ESR 78 you can just follow the installation instructions from before. If you already have the previous ESR 91 packages installed you can follow the same instructions but should skip steps 4 (copying the browser profile) and 7 (creating a symlink to the ESR 78 library directory).

You should, however, still do step 5 to clean out the old settings files. This won't clear cookies, tabs, history or anything like that, but it will reset any preferences you've changed using about:config. In summary, if you already have ESR 91 installed you should complete the following steps:

1. Download the tarball to your phone.

   $ curl -O https://www.flypig.co.uk/dnload/dnload/sailfishos/gecko/gecko-dev-esr91-release.tar.bz2
   

2. Unpack the tarball:

   $ tar -xvf gecko-dev-esr91-release.tar.bz2
   

3. Ensure the browser and any apps using the WebView are closed.
4. [Skip this step]
5. Remove settings that need to be updated for ESR 91:

   $ rm ~/.local/share/org.sailfishos/browser/.mozilla/ua-update.json
   $ rm -rf ~/.local/share/org.sailfishos/browser/.mozilla/startupCache
   $ rm ~/.local/share/org.sailfishos/browser/.mozilla/prefs.js
   $ rm ~/.local/share/org.sailfishos/browser/__PREFS_WRITTEN__
   

6. Install the ESR 91 packages:

   $ devel-su rpm -U --force \
       xulrunner-qt5-91.*.rpm \
       xulrunner-qt5-misc-91.*.rpm \
       qtmozembed-qt5-1.*.rpm \
       sailfish-components-webview-qt5-1.*.rpm \
       sailfish-components-webview-qt5-pickers-1.*.rpm \
       sailfish-components-webview-qt5-popups-1.*.rpm \
       embedlite-components-qt5-1.*.rpm \
       sailfish-browser-2.*.rpm \
       sailfish-browser-settings-2.*.rpm \
       mapplauncherd-booster-browser-0.*.rpm
   

7. [Skip this step]
8. Finally kill the browser booster:

   $ killall booster-browser
   

9. You're good to go!

As always, if you run in to trouble, drop me an email, create an issue on GitHub, or post a message in the Sailfish OS forum.

That summarises the changes this week. I'm still writing up my retrospective of the project so far and will post it once it's ready.

If you'd like to read any of my other gecko diary entries, they're all available on my Gecko-dev Diary page.