List items
Items from the current list are shown below.
Gecko
15 Jan 2024 : Day 139 #
This morning I transferred the DuckDuckGo testing website from S3 where we had it yesterday to Amazon CloudFront. This still uses the S3 bucket to serve the pages as a static site, but now with a cloudfront.net URL (which shouldn't make any difference for the tests) and using HTTPS (which might make a difference). I want to use HTTPS not because I'm worried about the integrity of the site, but because I want to eliminate differences between how I'm accessing the real DuckDuckGo and the test page version I'm using.
A small, overly-optimistic, bit of me was hoping that the switch to HTTPS might cause the site to break when using ESR 91, but it didn't. The DuckDuckGo logo shows just fine, the page looks sensible, albeit still showing the peculiar differences in functionality that I described yesterday.
The next steps are to capture the full output from accessing the original site using ESR 91, followed by the same process accessing the test site using ESR 91. In order to do this I'll need to build and run the recent changes I made to the EmbedLiteConsoleListener.js code so that it can be installed on my ESR 91 device. I made the changes manually on my ESR 78 device, and although I did already manually copy those changes over to the local repository on my development machine, I've not yet built and installed them for my other phone.
Having done that, I now need to capture a copy of the site using the ESR 91 version of the browser. I'll capture both a copy of the real site and a copy of the replicated version of the site (the version downloaded using ESR 78):
I also tried downloading a copy using ESR 91 and the iPhone user agent string from yesterday. But the site downloaded was the same.
What I want to do now is create a copy of the site downloaded when I use ESR 91. This is the site that's broken (showing just a blank page) when rendered using the ESR 91 renderer. But although the page is blank it is still downloading a bunch of files, so there's definitely something to replicate.
Having done this process before I'm getting quite proficient at it now. The process goes like this:
There's one final check to make and that's to access the copy of the site originally downloaded using ESR 91 and now being served on CloudFront (the ddg9 version of the site), but using ESR 91. Why do this? Once I've got this I can compare the files downloaded this way with the files downloaded directly from DuckDuckGo using ESR 91.
If the copy is good, the files downloaded should be very similar, if not identical.
I've done this now, so have two copies of the log output. Let's compare them using the comparison command I put together a few days back.
Visiting this copy of the site in other browsers, including ESR 78 and the desktop version of Firefox, shows that the site doesn't render there either.
It's been a long journey to get to this point, but this is clear evidence that the problem is the site being served to ESR 91, rather than the ESR 91 rendering engine or JavaScript interpreter getting stuck after the site has been received.
This means I have to concentrate on persuading DuckDuckGo to serve the same version of the page that it's serving to ESR 78. It's taken far too long to get here, but at least I feel I've learnt something in the process about how to perform these checks, how to download accurate copies of a website and how to serve them using CloudFront so that they don't have to go in a sub-directory.
I've already tried with multiple different user agents and that hasn't been enough to persuade DuckDuckGo to serve the correct version of the page, so I'm not quite sure how to get around this issue. One possibility is that I'm not actually using the user agents I think I am. So this will be something to check tomorrow.
If you'd like to read any of my other gecko diary entries, they're all available on my Gecko-dev Diary page.
A small, overly-optimistic, bit of me was hoping that the switch to HTTPS might cause the site to break when using ESR 91, but it didn't. The DuckDuckGo logo shows just fine, the page looks sensible, albeit still showing the peculiar differences in functionality that I described yesterday.
The next steps are to capture the full output from accessing the original site using ESR 91, followed by the same process accessing the test site using ESR 91. In order to do this I'll need to build and run the recent changes I made to the EmbedLiteConsoleListener.js code so that it can be installed on my ESR 91 device. I made the changes manually on my ESR 78 device, and although I did already manually copy those changes over to the local repository on my development machine, I've not yet built and installed them for my other phone.
Having done that, I now need to capture a copy of the site using the ESR 91 version of the browser. I'll capture both a copy of the real site and a copy of the replicated version of the site (the version downloaded using ESR 78):
$ rm -rf ~/.local/share/org.sailfishos/browser
$ EMBED_CONSOLE="network" sailfish-browser \
https://d3gf5xld99gmbj.cloudfront.net/ 2>&1 > esr91-ddg-copy.txt
[...]
$ rm -rf ~/.local/share/org.sailfishos/browser
$ EMBED_CONSOLE="network" sailfish-browser \
https://duckduckgo.com/ 2>&1 > esr91-ddg-orig.txt
[...]
Looking through the output of both it's clear that they're quite different. Just starting with the index.html page, the very first line of each differ significantly. So it really does seem to be the case that DuckDuckGo is serving a different version of the page.I also tried downloading a copy using ESR 91 and the iPhone user agent string from yesterday. But the site downloaded was the same.
What I want to do now is create a copy of the site downloaded when I use ESR 91. This is the site that's broken (showing just a blank page) when rendered using the ESR 91 renderer. But although the page is blank it is still downloading a bunch of files, so there's definitely something to replicate.
Having done this process before I'm getting quite proficient at it now. The process goes like this:
- Take the log output from loading the page, with the full network dump enabled
- Work through this log file and whenever there's some text content in the log, cut and paste this out into its own file. This is then a copy of the file as it was downloaded by the Sailfish Browser.
- Carefully save this file out using the same file structure as served by the server (matching the suffix of the URL of the file downloaded).
- Having recreated all of these files, create an S3 bucket on AWS and copy all of these files in to it.
- Create a CloudFront distribution of the bucket. To reiterate, I do it this way rather than just serving the bucket as a static site so that I can offer the site with HTTPS access.
- ddg8: the original DuckDuckGo site as downloaded by ESR 78: https://d3gf5xld99gmbj.cloudfront.net/.
- ddg9: the original DuckDuckGo site as downloaded by ESR 91: https://dd53jyxmgchu8.cloudfront.net/.
There's one final check to make and that's to access the copy of the site originally downloaded using ESR 91 and now being served on CloudFront (the ddg9 version of the site), but using ESR 91. Why do this? Once I've got this I can compare the files downloaded this way with the files downloaded directly from DuckDuckGo using ESR 91.
If the copy is good, the files downloaded should be very similar, if not identical.
I've done this now, so have two copies of the log output. Let's compare them using the comparison command I put together a few days back.
$ diff --side-by-side <(sed -e 's/^.*\/\(.*\)/\1/g' <(grep "duckduckgo" esr91-ddg-orig-urls.txt) | sort) \ <(sed -e 's/^.*\/\(.*\)/\1/g' <(grep "cloudfront" esr91-ddg-esr91-urls.txt) | sort) 18040-1287342b1f839f70.js 18040-1287342b1f839f70.js 38407-070351ade350c8e4.js 38407-070351ade350c8e4.js 39337-cd8caeeff0afb1c4.js 39337-cd8caeeff0afb1c4.js 41966-c9d76895b4f9358f.js 41966-c9d76895b4f9358f.js 55015-29fec414530c2cf6.js 55015-29fec414530c2cf6.js 55672-0a2c33e517ba92f5.js 55672-0a2c33e517ba92f5.js 61754-cfebc3ba4c97208e.js 61754-cfebc3ba4c97208e.js 6a4833195509cc3d.css 6a4833195509cc3d.css 703c9a9a057785a9.css 703c9a9a057785a9.css 81125-b74d1b6f4908497b.js 81125-b74d1b6f4908497b.js 93432-ebd443fe69061b19.js 93432-ebd443fe69061b19.js 94623-d5bfa67fc3bada59.js 94623-d5bfa67fc3bada59.js 95665-f2a003aa56f899b0.js 95665-f2a003aa56f899b0.js a2a29f84956f2aac.css a2a29f84956f2aac.css _app-a1aac13e30ca1ed6.js _app-a1aac13e30ca1ed6.js _buildManifest.js _buildManifest.js c89114cfe55133c4.css c89114cfe55133c4.css ed8494aa71104fdc.css ed8494aa71104fdc.css f0b3f7da285c9dbd.css f0b3f7da285c9dbd.css framework-f8115f7fae64930e.js framework-f8115f7fae64930e.js home-34dda07336cb6ee1.js home-34dda07336cb6ee1.js main-17a05b704438cdd6.js main-17a05b704438cdd6.js ProximaNova-Bold-webfont.woff2 ProximaNova-Bold-webfont.woff2 ProximaNova-ExtraBold-webfont.woff2 ProximaNova-ExtraBold-webfont.woff2 ProximaNova-RegIt-webfont.woff2 ProximaNova-RegIt-webfont.woff2 ProximaNova-Reg-webfont.woff2 ProximaNova-Reg-webfont.woff2 ProximaNova-Sbold-webfont.woff2 ProximaNova-Sbold-webfont.woff2 _ssgManifest.js _ssgManifest.js webpack-96503cdd116848e8.js webpack-96503cdd116848e8.jsThe two sets of downloaded files are identical. This is really good, because it means that the ddg9 version of the site is an accurate reflection of what's being served to ESR 91 when Sailfish Browser accesses the real DuckDuckGo site using the ESR 91 engine.
Visiting this copy of the site in other browsers, including ESR 78 and the desktop version of Firefox, shows that the site doesn't render there either.
It's been a long journey to get to this point, but this is clear evidence that the problem is the site being served to ESR 91, rather than the ESR 91 rendering engine or JavaScript interpreter getting stuck after the site has been received.
This means I have to concentrate on persuading DuckDuckGo to serve the same version of the page that it's serving to ESR 78. It's taken far too long to get here, but at least I feel I've learnt something in the process about how to perform these checks, how to download accurate copies of a website and how to serve them using CloudFront so that they don't have to go in a sub-directory.
I've already tried with multiple different user agents and that hasn't been enough to persuade DuckDuckGo to serve the correct version of the page, so I'm not quite sure how to get around this issue. One possibility is that I'm not actually using the user agents I think I am. So this will be something to check tomorrow.
If you'd like to read any of my other gecko diary entries, they're all available on my Gecko-dev Diary page.
Comments
Uncover Disqus comments