Gecko-dev Diary
Starting in August 2023 I'll be upgrading the Sailfish OS browser from Gecko version ESR 78 to ESR 91. This page catalogues my progress.
Latest code changes are in the gecko-dev sailfishos-esr91 branch.
There is an index of all posts in case you want to jump to a particular day.
Click the icon for the Gecko-dev Diary RSS feed.
Gecko
5 most recent items
28 May 2024 : Day 246 #
Hooray! After a lot of to-ing and fro-ing, I now have a successful build and a bunch of fresh packages. Now to test them. I've transferred them over to my phone, installed the packages and set the harbour-webview test application running.
After my epic train journey building these packages yesterday, and in the hope that these packages might work, thigg, AI artist in residence, has kindly created a picture of the aftermath. I think this captures the vibe rather nicely! Thank you thigg as always for your brilliant work.
Astonishingly, after all this when I execute my harbour-webview test app using the new packages I still just get a blank screen. That's okay though. Presumably it means the NotifyDidPaintForSubtree events still aren't being called. I now have all the structure for that to happen, so I'll just have to do a bit of debugging to find out why. That, at least, is the plan on paper.
So let's get stuck in with the debugger.
And if I restart the app without the debugger it's now working fine as well. The page is being rendered correctly to the screen. Weird. So although it didn't work the first time, it is now working correctly.
There are a bunch of reasons why this might happen, such as a profile folder that needed refreshing or an executable that hadn't quite been properly quit. So I'll put it down to an anomaly. Whatever the reason It looks like the WebView is now working correctly.
It feels like this justifies a bit of a celebration and thankfully thigg is on hand once again to provide us with the content needed for just that. After the rather sombre image above, this feels a lot more upbeat and I really do like it. Finally, the Firefox has yielded its rendering riches!
Thank you again thigg for your amazing work. I'd happily continue the celebrations, but in practice there's more work to be done. Although that's all for the harbour-webview test app it's time now to try a proper app that uses the WebView to render. The obvious example of this is the email application, but I don't have an email account set up on this phone as I only use it for development. I'll set up an email account to check this in due course, but I'm looking for a quick test and entering my account details will take some time.
The good news is that the Settings app also uses the WebView, specifically in the account creation pages, depending on the account type. Google, VK, DropBox, OneDrive and Twitter all use a WebView to show Terms and Conditions pages, and in the case of Twitter it's used for OAuth configuration as well.
But when I try to use any of them the Settings app crashes. Here's the backtrace I get when the crash occurs:
The first time was back on Day 50 where the problem followed from the fact that gfx::gfxVars::UseWebRender() was set to true when it was expected to be set to false.
The second time was on Day 160 when the problem also amounted to the WebView layer manager being used. It took a couple of days of investigation (I eventually figured it out on Day 163) to figure out the reason, but it turned out to be because the gfx.webrender.force-disabled preference was set to false when it should have been set to true.
This preference is managed by the pref.js file inside the .mozilla profile folder. This folder is stored in different places for different apps. In the case of the Settings app the profile folder looks like this:
I need to keep my problems partitioned, otherwise they become too big and never get resolved. I'll create an issue for this pref.js problem and continue concentrating on getting the rendering wrapped up.
So what is there still to wrap up on the rendering front? Well, specifically I need to now go through the code and try to simplify things, partition everything into a series of clean commits so that I can turn them into nice patches.
I've had to make a really quite huge number of changes to get this WebView offscreen rendering working. Now I have to see if it can be simplified. This is a big task and not one I'm going to start today, so that'll have to do for now. I'll pick up the next steps tomorrow.
If you'd like to read any of my other gecko diary entries, they're all available on my Gecko-dev Diary page.
After my epic train journey building these packages yesterday, and in the hope that these packages might work, thigg, AI artist in residence, has kindly created a picture of the aftermath. I think this captures the vibe rather nicely! Thank you thigg as always for your brilliant work.
Astonishingly, after all this when I execute my harbour-webview test app using the new packages I still just get a blank screen. That's okay though. Presumably it means the NotifyDidPaintForSubtree events still aren't being called. I now have all the structure for that to happen, so I'll just have to do a bit of debugging to find out why. That, at least, is the plan on paper.
So let's get stuck in with the debugger.
(gdb) info break
Num Type Address What
1 0x0000007ff2c81914 in nsRootPresContext::EnsureEventualDidPaintEvent(
BaseTransactionId<TransactionIdType>) at nsPresContext.cpp:2689
breakpoint already hit 4 times
2 0x0000007ff7ee8ff0 in QMozViewPrivate::OnFirstPaint(int, int) at
qmozview_p.cpp:1122
breakpoint already hit 1 time
3 <MULTIPLE>
breakpoint already hit 2 times
3.1 0x0000007ff367468c in mozilla::embedlite::EmbedLiteViewChild::
OnFirstPaint(int, int) at EmbedLiteViewChild.cpp:1479
3.2 0x0000007ff3674774 EmbedLiteViewChild.h:60
(gdb) c
[...]
Thread 7 "GeckoWorkerThre" hit Breakpoint 1, nsRootPresContext::
EnsureEventualDidPaintEvent (this=0x7fc5112ba0,
aTransactionId=aTransactionId@entry=...)
at nsPresContext.cpp:2689
2689 {
(gdb) c
Continuing.
Thread 7 "GeckoWorkerThre" hit Breakpoint 3, non-virtual thunk to
mozilla::embedlite::EmbedLiteViewChild::OnFirstPaint(int, int) ()
at EmbedLiteViewChild.h:60
60 NS_DECL_NSIEMBEDBROWSERCHROMELISTENER
(gdb) c
Continuing.
Thread 7 "GeckoWorkerThre" hit Breakpoint 3, mozilla::embedlite::
EmbedLiteViewChild::OnFirstPaint (this=0x7fc4ae79c0, aX=0, aY=0)
at EmbedLiteViewChild.cpp:1479
1479 {
(gdb) c
Continuing.
Thread 1 "harbour-webview" hit Breakpoint 2, QMozViewPrivate::
OnFirstPaint (this=0x55557db0f0, aX=0, aY=0) at qmozview_p.cpp:1122
1122 mIsPainted = true;
(gdb) c
[...]
As you can see from this all of the expected methods are being called. So it's a bit strange that it's not rendering anything. And in fact... the odd thing is that now, with the debugger running, it is rendering something. If I clear the breakpoints, the render appears just fine.And if I restart the app without the debugger it's now working fine as well. The page is being rendered correctly to the screen. Weird. So although it didn't work the first time, it is now working correctly.
There are a bunch of reasons why this might happen, such as a profile folder that needed refreshing or an executable that hadn't quite been properly quit. So I'll put it down to an anomaly. Whatever the reason It looks like the WebView is now working correctly.
It feels like this justifies a bit of a celebration and thankfully thigg is on hand once again to provide us with the content needed for just that. After the rather sombre image above, this feels a lot more upbeat and I really do like it. Finally, the Firefox has yielded its rendering riches!
Thank you again thigg for your amazing work. I'd happily continue the celebrations, but in practice there's more work to be done. Although that's all for the harbour-webview test app it's time now to try a proper app that uses the WebView to render. The obvious example of this is the email application, but I don't have an email account set up on this phone as I only use it for development. I'll set up an email account to check this in due course, but I'm looking for a quick test and entering my account details will take some time.
The good news is that the Settings app also uses the WebView, specifically in the account creation pages, depending on the account type. Google, VK, DropBox, OneDrive and Twitter all use a WebView to show Terms and Conditions pages, and in the case of Twitter it's used for OAuth configuration as well.
But when I try to use any of them the Settings app crashes. Here's the backtrace I get when the crash occurs:
Thread 10 "GeckoWorkerThre" received signal SIGSEGV, Segmentation
fault.
[Switching to LWP 2476]
0x0000007fc4fe8fdc in mozilla::embedlite::PuppetWidgetBase::Invalidate (
this=0x7fb4e698b0, aRect=...)
at mobile/sailfishos/embedshared/PuppetWidgetBase.cpp:274
274 MOZ_CRASH("Unexpected layer manager type");
(gdb) bt
#0 0x0000007fc4fe8fdc in mozilla::embedlite::PuppetWidgetBase::Invalidate (
this=0x7fb4e698b0, aRect=...)
at mobile/sailfishos/embedshared/PuppetWidgetBase.cpp:274
#1 0x0000007fc4fed870 in mozilla::embedlite::PuppetWidgetBase::UpdateBounds (
this=0x7fb4e698b0, aRepaint=aRepaint@entry=true)
at mobile/sailfishos/embedshared/PuppetWidgetBase.cpp:395
#2 0x0000007fc4ff6a5c in mozilla::embedlite::EmbedLiteWindowChild::
CreateWidget (this=0x7fb4e19370)
at xpcom/base/nsCOMPtr.h:851
#3 0x0000007fc4fe6fc8 in mozilla::detail::RunnableMethodArguments<>::
applyImpl<mozilla::embedlite::EmbedLiteWindowChild, void (mozilla::
embedlite::EmbedLiteWindowChild::*)()>(mozilla::embedlite::
EmbedLiteWindowChild*, void (mozilla::embedlite::EmbedLiteWindowChild::*)(
), mozilla::Tuple<>&, std::integer_sequence<unsigned long>) (args=...,
m=<optimized out>, o=<optimized out>)
at ${PROJECT}/obj-build-mer-qt-xr/dist/include/nsThreadUtils.h:1151
#4 mozilla::detail::RunnableMethodArguments<>::apply<mozilla::embedlite::
EmbedLiteWindowChild, void (mozilla::embedlite::EmbedLiteWindowChild::*)()>
(
m=<optimized out>, o=<optimized out>, this=<optimized out>)
at ${PROJECT}/obj-build-mer-qt-xr/dist/include/nsThreadUtils.h:1154
#5 mozilla::detail::RunnableMethodImpl<mozilla::embedlite::
EmbedLiteWindowChild*, void (mozilla::embedlite::EmbedLiteWindowChild::*)(
), true, (mozilla::RunnableKind)1>::Run (this=<optimized out>)
at ${PROJECT}/obj-build-mer-qt-xr/dist/include/nsThreadUtils.h:1201
#6 0x0000007fc214e63c in mozilla::RunnableTask::Run (this=0x55562b6ce0)
at ${PROJECT}/obj-build-mer-qt-xr/dist/include/mozilla/RefPtr.h:313
[...]
#28 0x0000007ff6f4289c in ?? () from /lib64/libc.so.6
(gdb)
And here's where that error is coming from in PuppetWidgetBase.cpp according to this backtrace:
if (mozilla::layers::LayersBackend::LAYERS_CLIENT == lm->GetBackendType()) {
// No need to do anything, the compositor will handle drawing
} else {
MOZ_CRASH("Unexpected layer manager type");
}
As we can see from this, it's an enforced crash which happens whenever the layer backend is set to anything other than LAYERS_CLIENT. According to the debugger, ours is set to LAYERS_WR, so it's no wonder the crash is happening:
(gdb) p lm->GetBackendType() $1 = mozilla::layers::LayersBackend::LAYERS_WR (gdb)The interesting thing is that this isn't the first time we've hit this error. In fact, it's not the second time either.
The first time was back on Day 50 where the problem followed from the fact that gfx::gfxVars::UseWebRender() was set to true when it was expected to be set to false.
The second time was on Day 160 when the problem also amounted to the WebView layer manager being used. It took a couple of days of investigation (I eventually figured it out on Day 163) to figure out the reason, but it turned out to be because the gfx.webrender.force-disabled preference was set to false when it should have been set to true.
This preference is managed by the pref.js file inside the .mozilla profile folder. This folder is stored in different places for different apps. In the case of the Settings app the profile folder looks like this:
$ ls -a /home/defaultuser/.cache/org.sailfishos/Settings/
. .. .mozilla
$ ls -a /home/defaultuser/.cache/org.sailfishos/Settings/.mozilla/
. .parentlock cert9.db cookies.sqlite-wal
pkcs11.txt startupCache storage.sqlite
.. cache2 cookies.sqlite key4.db
security_state storage ua-update.json
$ rm -rf /home/defaultuser/.cache/org.sailfishos/Settings/.mozilla
It's notable that there is no pref.js file to be found here. That contrasts with the harbour-webview configuration file, which not only exists, but which also includes a setting for gfx.webrender.force-disabled:
$ pwd /home/defaultuser/.cache/harbour-webview/harbour-webview/.mozilla $ grep "gfx\.webrender\.force-disabled" prefs.js user_pref("gfx.webrender.force-disabled", true);But as we noted, there's no equivalent settings file for the Settings app:
$ pwd /home/defaultuser/.cache/org.sailfishos/Settings/.mozilla $ grep "gfx\.webrender\.force-disabled" prefs.js grep: prefs.js: No such file or directoryThere's obviously something to fix here, but right now I just want to test the rendering. So to move things forwards I'm going to copy over the preferences file from the harbour-webview app and put it in the configuration folder for the Settings app.
$ cp ../../../harbour-webview/harbour-webview/.mozilla/prefs.js .
$ ls
cache2 cookies.sqlite key4.db prefs.js
startupCache storage.sqlite
cert9.db cookies.sqlite-wal pkcs11.txt security_state
storage ua-update.json
$ grep "gfx\.webrender\.force-disabled" prefs.js
user_pref("gfx.webrender.force-disabled", true);
After copying this file over and re-starting the Settings app I find things are now working as expected. I can happily view the Terms and Conditions pages for the different accounts and even start to enter my Twitter credentials if I want to. There is clearly a problem to solve here, but it's not directly related to rendering. The problem is that the pref.js file isn't being created when it should be.I need to keep my problems partitioned, otherwise they become too big and never get resolved. I'll create an issue for this pref.js problem and continue concentrating on getting the rendering wrapped up.
So what is there still to wrap up on the rendering front? Well, specifically I need to now go through the code and try to simplify things, partition everything into a series of clean commits so that I can turn them into nice patches.
I've had to make a really quite huge number of changes to get this WebView offscreen rendering working. Now I have to see if it can be simplified. This is a big task and not one I'm going to start today, so that'll have to do for now. I'll pick up the next steps tomorrow.
If you'd like to read any of my other gecko diary entries, they're all available on my Gecko-dev Diary page.
Comments
Uncover Disqus comments