flypig.co.uk

It looks like I'm beginning to wrap things up. Don't get me wrong, I'm not under any illusion that Sailfish OS browser development will ever be "done", but the objective here is just to get ESR 91 into a usable state. Then I'm hoping Jolla can perform its magic to get from that into a production-ready state.

I still need to arrange the patches and create pull requests, but otherwise the work done over the last few days, including the help I've received from Raine, Frajo and mal at Jolla, mean that things have progressed well.

Where does this leave us today? The browser is now working nicely on Sailfish OS 4.6. Actually, I'd go so far as to say I'm really pleased with the result. It feels responsive and usable on the majority of sites I've tested. Under the light load I've subjected them to, the browser and WebView have both been pretty stable. By no means all perfect, but usable.

So I've decided to take the plunge and install it on my daily device. This is no small step for me, because for better or worse I use my phone a lot and the browser has become an essential feature. If I break my phone I'm going to regret it.

Installing it on my daily phone will allow me to write-up the steps needed to install the packages, alongside the steps needed to revert a phone back to its previous state. It'll also be a convenient way to test its functionality.

Several people have also asked to try it out so I'll also be releasing the packages and instructions. It could certainly benefit from the testing, but this will only be for the brave and/or those with a second "disposable" device.

First up I'm going to create a tarball containing all of the packages. I'm just going to include the release packages at this stage to keep things small and tidy.

$ tar -czf \
    gecko-dev-esr91-release.tar.bz2 \
    xulrunner-qt5-91.9.1-1.aarch64.rpm \
    xulrunner-qt5-misc-91.9.1-1.aarch64.rpm \
    qtmozembed-qt5-1.53.9-1.aarch64.rpm \
    sailfish-components-webview-qt5-1.5.21-1.aarch64.rpm \
    sailfish-components-webview-qt5-pickers-1.5.21-1.aarch64.rpm \
    sailfish-components-webview-qt5-popups-1.5.21-1.aarch64.rpm \
    embedlite-components-qt5-1.23.0-1.aarch64.rpm \
    sailfish-browser-2.2.45-1.aarch64.rpm \
    sailfish-browser-settings-2.2.45-1.aarch64.rpm \
    mapplauncherd-booster-browser-0.2.1-1.aarch64.rpm

This produces a neat 37 MiB compressed tarball. Some people might want all of the packages, including the debug symbols, so I'll make a tarball available with everything in as well:

$ tar -czf \
    gecko-dev-esr91-debug.tar.bz2 \
    xulrunner-qt5-91.9.1-1.aarch64.rpm \
    xulrunner-qt5-debuginfo-91.9.1-1.aarch64.rpm \
    xulrunner-qt5-debugsource-91.9.1-1.aarch64.rpm \
    xulrunner-qt5-devel-91.9.1-1.aarch64.rpm \
    xulrunner-qt5-misc-91.9.1-1.aarch64.rpm \
    qtmozembed-qt5-1.53.9-1.aarch64.rpm \
    qtmozembed-qt5-debuginfo-1.53.9-1.aarch64.rpm \
    qtmozembed-qt5-debugsource-1.53.9-1.aarch64.rpm \
    qtmozembed-qt5-devel-1.53.9-1.aarch64.rpm \
    qtmozembed-qt5-tests-1.53.9-1.aarch64.rpm \
    sailfish-components-webview-qt5-1.5.21-1.aarch64.rpm \
    sailfish-components-webview-qt5-debuginfo-1.5.21-1.aarch64.rpm \
    sailfish-components-webview-qt5-debugsource-1.5.21-1.aarch64.rpm \
    sailfish-components-webview-qt5-devel-1.5.21-1.aarch64.rpm \
    sailfish-components-webview-qt5-doc-1.5.21-1.aarch64.rpm \
    sailfish-components-webview-qt5-examples-1.5.21-1.aarch64.rpm \
    sailfish-components-webview-qt5-pickers-1.5.21-1.aarch64.rpm \
    sailfish-components-webview-qt5-popups-1.5.21-1.aarch64.rpm \
    sailfish-components-webview-qt5-tests-1.5.21-1.aarch64.rpm \
    sailfish-components-webview-qt5-ts-devel-1.5.21-1.aarch64.rpm \
    embedlite-components-qt5-1.23.0-1.aarch64.rpm \
    embedlite-components-qt5-debuginfo-1.23.0-1.aarch64.rpm \
    embedlite-components-qt5-debugsource-1.23.0-1.aarch64.rpm \
    sailfish-browser-2.2.45-1.aarch64.rpm \
    sailfish-browser-debuginfo-2.2.45-1.aarch64.rpm \
    sailfish-browser-debugsource-2.2.45-1.aarch64.rpm \
    sailfish-browser-settings-2.2.45-1.aarch64.rpm \
    sailfish-browser-tests-2.2.45-1.aarch64.rpm \
    sailfish-browser-ts-devel-2.2.45-1.aarch64.rpm \
    mapplauncherd-booster-browser-0.2.1-1.aarch64.rpm \
    mapplauncherd-booster-browser-debuginfo-0.2.1-1.aarch64.rpm \
    mapplauncherd-booster-browser-debugsource-0.2.1-1.aarch64.rpm

This debug tarball is quite a bit larger at 670 MiB, but that's okay because I don't expect most people will want or need anything other than the release tarball.

$ ls -lh gecko-dev-esr91*.tar.bz2
-rw-rw-r-- 1 flypig flypig 670M Aug 23 16:54 gecko-dev-esr91-debug.tar.bz2
-rw-rw-r-- 1 flypig flypig  37M Aug 23 16:49 gecko-dev-esr91-release.tar.bz2

Alright, time to test it out. I've uploaded the tarballs to my website which means I can download them to my phone using curl:

$ curl https://www.flypig.co.uk/dnload/dnload/sailfishos/gecko/gecko-dev-esr91-release.tar.bz2 \
    -O gecko-dev-esr91-release.tar.bz2

Next it needs unpacking:

$ tar -xvf gecko-dev-esr91-release.tar.bz2
embedlite-components-qt5-1.23.0-1.aarch64.rpm
mapplauncherd-booster-browser-0.2.1-1.aarch64.rpm
qtmozembed-qt5-1.53.9-1.aarch64.rpm
sailfish-browser-2.2.45-1.aarch64.rpm
sailfish-browser-settings-2.2.45-1.aarch64.rpm
sailfish-components-webview-qt5-1.5.21-1.aarch64.rpm
sailfish-components-webview-qt5-pickers-1.5.21-1.aarch64.rpm
sailfish-components-webview-qt5-popups-1.5.21-1.aarch64.rpm
xulrunner-qt5-91.9.1-1.aarch64.rpm
xulrunner-qt5-misc-91.9.1-1.aarch64.rpm

Before moving on I need to close down the browser if it's running and any apps that might be using a WebView. The next few steps are going to be destructive, in that they'll mess up some of the ESR 78 configuration settings. It's therefore crucial that I make a copy of my ESR 78 configuration before the upgrade, so that I can restore it later if I need to switch back.

$ cp ~/.local/share/org.sailfishos/browser/ ~/.local/share/org.sailfishos/browser.esr78.bak/

Next I need to clean a few things up, in particular by removing various configuration files that I want the ESR 91 code to regenerate.

$ rm ~/.local/share/org.sailfishos/browser/.mozilla/ua-update.json
$ rm -rf ~/.local/share/org.sailfishos/browser/.mozilla/startupCache
$ rm -rf ~/.local/share/org.sailfishos/browser/.mozilla/prefs.js
$ rm -rf ~/.local/share/org.sailfishos/browser/__PREFS_WRITTEN__

In the final version these steps are likely to be performed by a oneshot script, but creating that is a future task. Now I'm ready to install the packages. This will require root privileges, so I've prefixed the installation command with devel-su:

$ devel-su rpm -U --force \
    xulrunner-qt5-91.*.rpm \
    xulrunner-qt5-misc-91.*.rpm \
    qtmozembed-qt5-1.*.rpm \
    sailfish-components-webview-qt5-1.*.rpm \
    sailfish-components-webview-qt5-pickers-1.*.rpm \
    sailfish-components-webview-qt5-popups-1.*.rpm \
    embedlite-components-qt5-1.*.rpm \
    sailfish-browser-2.*.rpm \
    sailfish-browser-settings-2.*.rpm \
    mapplauncherd-booster-browser-0.*.rpm

Finally we need to kill the browser booster to avoid the old libraries lingering in runtime memory:

$ killall booster-browser

I originally thought this would be enough to get things working, but it turns out this breaks the email app as it's unable to find the libxul.so library. As I discussed yesterday, this is due to the jolla-email binary having an rpath that points to the ESR 78 directory. The simplest way I've found to work around this is to create a symlink to the new directory at the location of the previous one:

$ ln -s /usr/lib64/xulrunner-qt5-91.9.1/ /usr/lib64/xulrunner-qt5-78.15.1

And with that the browser should be ready to test! It works for me either from the command line, or directly by selecting the icon in the app drawer.

In case that gives a bad result, I also need to figure out the right way to revert back to ESR 78. I'll come on to that, but first, here's a summary of the steps I've taken to complete the installation of ESR 91:

1. Download the tarball to your phone.

   $ curl -O https://www.flypig.co.uk/dnload/dnload/sailfishos/gecko/
       gecko-dev-esr91-release.tar.bz2
   

2. Unpack the tarball:

   $ tar -xvf gecko-dev-esr91-release.tar.bz2
   

3. Ensure the browser and any apps using the WebView are closed.
4. Make a copy of your browser profile:

   $ cp -r ~/.local/share/org.sailfishos/browser/ ~/.local/share/org.sailfishos/
       browser.esr78.bak/
   

5. Remove settings that need to be updated for ESR 91:

   $ rm ~/.local/share/org.sailfishos/browser/.mozilla/ua-update.json
   $ rm -rf ~/.local/share/org.sailfishos/browser/.mozilla/startupCache
   $ rm -rf ~/.local/share/org.sailfishos/browser/.mozilla/prefs.js
   $ rm -rf ~/.local/share/org.sailfishos/browser/__PREFS_WRITTEN__
   

6. Install the ESR 91 packages:

   $ devel-su rpm -U --force \
       xulrunner-qt5-91.*.rpm \
       xulrunner-qt5-misc-91.*.rpm \
       qtmozembed-qt5-1.*.rpm \
       sailfish-components-webview-qt5-1.*.rpm \
       sailfish-components-webview-qt5-pickers-1.*.rpm \
       sailfish-components-webview-qt5-popups-1.*.rpm \
       embedlite-components-qt5-1.*.rpm \
       sailfish-browser-2.*.rpm \
       sailfish-browser-settings-2.*.rpm \
       mapplauncherd-booster-browser-0.*.rpm
   

7. Symlink the old ESR 78 library directory to the new location:

   $ devel-su ln -s /usr/lib64/xulrunner-qt5-91.9.1/ /usr/lib64/
       xulrunner-qt5-78.15.1
   

8. Finally kill the browser booster:

   $ killall booster-browser
   

9. You're good to go!

For completeness you can use the following to download the much larger debug tarball:

$ curl https://www.flypig.co.uk/dnload/dnload/sailfishos/gecko/gecko-dev-esr91-debug.tar.bz2 \
    -O gecko-dev-esr91-debug.tar.bz2

Then, having unpacked it and after having completed the other instructions, you can then install all of the debug packages with the following command:

$ rpm -U --force \
    xulrunner-qt5-91.*.rpm \
    xulrunner-qt5-debuginfo-91.*.rpm \
    xulrunner-qt5-debugsource-91.*.rpm \
    xulrunner-qt5-misc-91.*.rpm \
    qtmozembed-qt5-1.*.rpm \
    qtmozembed-qt5-debuginfo-1.*.rpm \
    qtmozembed-qt5-debugsource-1.*.rpm \
    sailfish-components-webview-qt5-1.*.rpm \
    sailfish-components-webview-qt5-debuginfo-1.*.rpm \
    sailfish-components-webview-qt5-debugsource-1.*.rpm \
    sailfish-components-webview-qt5-pickers-1.*.rpm \
    sailfish-components-webview-qt5-popups-1.*.rpm \
    embedlite-components-qt5-1.*.rpm \
    embedlite-components-qt5-debuginfo-1.*.rpm \
    embedlite-components-qt5-debugsource-1.*.rpm \
    sailfish-browser-2.*.rpm \
    sailfish-browser-debuginfo-2.*.rpm \
    sailfish-browser-debugsource-2.*.rpm \
    sailfish-browser-settings-2.*.rpm \
    mapplauncherd-booster-browser-0.*.rpm \
    mapplauncherd-booster-browser-debuginfo-0.*.rpm \
    mapplauncherd-booster-browser-debugsource-0.*.rpm

Alright, let's move on to the process you need to follow if you want to revert the changes and drop back to ESR 78. Here's the way I did it on my phone. Note that I'm using zypper here which you'll need for this. You can install it with devel-su pkcon install zypper if you don't already have it.

1. Ensure the browser and any apps using the WebView are closed.
2. Restore your original ESR 78 profile directory.

   $ mv ~/.local/share/org.sailfishos/browser/ ~/.local/share/org.sailfishos/browser.esr91.bak/
   $ cp -r ~/.local/share/org.sailfishos/browser.esr78.bak/ ~/.local/share/org.sailfishos/browser
   

3. Remove the symlink:

   $ devel-su rm /usr/lib64/xulrunner-qt5-78.15.1
   

4. Install all of the original packages from the Jolla repository.

   $ devel-su zypper install --oldpackage \
       xulrunner-qt5-78.15.1+git39-1.19.1.jolla \
       xulrunner-qt5-misc-78.15.1+git39-1.19.1.jolla \
       qtmozembed-qt5-1.53.25-1.22.2.jolla \
       sailfish-components-webview-qt5-1.5.21-1.13.1.jolla \
       sailfish-components-webview-qt5-pickers-1.5.21-1.13.1.jolla \
       sailfish-components-webview-qt5-popups-1.5.21-1.13.1.jolla \
       embedlite-components-qt5-1.24.35-1.26.1.jolla \
       sailfish-browser-2.2.63-1.12.1.jolla \
       sailfish-browser-settings-2.2.63-1.12.1.jolla \
       mapplauncherd-booster-browser-0.2.2-1.1.1.jolla
   

5. Finally kill the booster if it's running.

   $ killall booster-browser
   

6. You're back to ESR 78!

When you run the zypper command you'll be presented with a list of packages which it says are going to change vendor and which you'll need to agree to. This is normal; for me it looked something like this:

The following 8 packages are going to be upgraded:
  embedlite-components-qt5 mapplauncherd-booster-browser qtmozembed-qt5 
    sailfish-browser sailfish-browser-settings sailfish-components-webview-qt5
  sailfish-components-webview-qt5-pickers sailfish-components-webview-qt5-popups

The following 2 packages are going to be downgraded:
  xulrunner-qt5 xulrunner-qt5-misc

The following 10 packages are going to change vendor:
  embedlite-components-qt5                  -> meego
  mapplauncherd-booster-browser             -> meego
  qtmozembed-qt5                            -> meego
  sailfish-browser                          -> meego
  sailfish-browser-settings                 -> meego
  sailfish-components-webview-qt5           -> meego
  sailfish-components-webview-qt5-pickers   -> meego
  sailfish-components-webview-qt5-popups    -> meego
  xulrunner-qt5                             -> meego
  xulrunner-qt5-misc                        -> meego

8 packages to upgrade, 2 to downgrade, 10  to change vendor.
Overall download size: 39.5 MiB. Already cached: 0 B. After the operation, 94.8 
    MiB will be freed.
Continue? [y/n/v/...? shows all options] (y): y
[...]

If you installed the full set of debug packages you'll need to use an even longer command to restore ESR 78. This should revert the lot of them:

$ zypper install --oldpackage \
    xulrunner-qt5-78.15.1+git39-1.19.1.jolla \
    xulrunner-qt5-debuginfo-78.15.1+git39-1.19.1.jolla \
    xulrunner-qt5-debugsource-78.15.1+git39-1.19.1.jolla \
    xulrunner-qt5-misc-78.15.1+git39-1.19.1.jolla \
    qtmozembed-qt5-1.53.25-1.22.2.jolla \
    qtmozembed-qt5-debuginfo-1.53.25-1.22.2.jolla \
    qtmozembed-qt5-debugsource-1.53.25-1.22.2.jolla \
    sailfish-components-webview-qt5-1.5.21-1.13.1.jolla \
    sailfish-components-webview-qt5-debuginfo-1.5.21-1.13.1.jolla \
    sailfish-components-webview-qt5-debugsource-1.5.21-1.13.1.jolla \
    sailfish-components-webview-qt5-pickers-1.5.21-1.13.1.jolla \
    sailfish-components-webview-qt5-popups-1.5.21-1.13.1.jolla \
    embedlite-components-qt5-1.24.35-1.26.1.jolla \
    embedlite-components-qt5-debuginfo-1.24.35-1.26.1.jolla \
    embedlite-components-qt5-debugsource-1.24.35-1.26.1.jolla \
    sailfish-browser-2.2.63-1.12.1.jolla \
    sailfish-browser-debuginfo-2.2.63-1.12.1.jolla \
    sailfish-browser-debugsource-2.2.63-1.12.1.jolla \
    sailfish-browser-settings-2.2.63-1.12.1.jolla \
    mapplauncherd-booster-browser-0.2.2-1.1.1.jolla \
    mapplauncherd-booster-browser-debuginfo-0.2.2-1.1.1.jolla \
    mapplauncherd-booster-browser-debugsource-0.2.2-1.1.1.jolla

So far I've been doing all of this on my development phone. The next step is for me to install them on my main daily phone. This is an Xperia 10 III (a different one) running Sailfish OS 4.6.

Working through the instructions step-by-step gave the correct results: the browser immediately works fine.

I've created a gecko install page with all of these details summarised on it (there's a link at the top of the main gecko blog page too).

If you want to give this a go I recommend you follow the steps there. If you notice any problems with the installation steps please let me know. However if you find a problem with the browser itself, it would be preferable if you could please create an issue on GitHub about it. That'll be easier for me and others to handle.

Having now installed ESR 91 on my daily phone I'm expecting to find lots of issues and bugs. But in my short time doing real-world testing I've not yet found it to be unusable.

Tomorrow I'm going to start arranging the patches.

If you'd like to read any of my other gecko diary entries, they're all available on my Gecko-dev Diary page.