flypig.co.uk

This morning I've picked up where I left off, attempting to get the library to link after apply equivalents for patches 77, 78 and 79. These three patches represent the first of two stages needed to get WebRTC back up and running. Once these changes are going through I'll then need to apply patches 80 and 81, which represent the second stage, which should hopefully be enough to get things working.

Having applied the first three patches I was still left with a single undefined reference during linking. Apparently the following constructor is missing an implementation:

webrtc::EncoderSimulcastProxy::EncoderSimulcastProxy()

Checking through the code I discovered that in ESR 78 the EncoderSimulcastProxy class isn't used, with a VP8EncoderSimulcastProxy class being used in its stead. It seems likely this has something to do with the problem. In ESR 78 this class is defined in the vp8_encoder_simulcast_proxy.cc and vp8_encoder_simulcast_proxy.h files, whereas the ESR 91 code is found in the encoder_simulcast_proxy.cc and encoder_simulcast_proxy.h files.

If you've been following along for the last few days you'll know that the WebRTC code is being built using the recipes in the following GN build files:

$ ls dom/media/webrtc/third_party_build/gn-configs/
arm64_False_arm64_freebsd.json  x64_False_x64_netbsd.json
arm64_False_arm64_linux.json    x64_False_x64_openbsd.json
arm64_False_arm64_netbsd.json   x64_False_x64_win.json
arm64_False_arm64_openbsd.json  x64_False_x86_android.json
arm64_True_arm64_freebsd.json   x64_False_x86_linux.json
arm64_True_arm64_linux.json     x64_True_arm64_android.json
arm64_True_arm64_netbsd.json    x64_True_arm64_mac.json
arm64_True_arm64_openbsd.json   x64_True_arm64_win.json
arm_False_arm_freebsd.json      x64_True_arm_android.json
arm_False_arm_linux.json        x64_True_x64_android.json
arm_False_arm_netbsd.json       x64_True_x64_dragonfly.json
arm_False_arm_openbsd.json      x64_True_x64_freebsd.json
arm_True_arm_freebsd.json       x64_True_x64_linux.json
arm_True_arm_linux.json         x64_True_x64_mac.json
arm_True_arm_netbsd.json        x64_True_x64_netbsd.json
arm_True_arm_openbsd.json       x64_True_x64_openbsd.json
mips64_False_mips64_linux.json  x64_True_x64_win.json
mips64_True_mips64_linux.json   x64_True_x86_android.json
ppc64_False_ppc64_linux.json    x64_True_x86_linux.json
ppc64_True_ppc64_linux.json     x86_False_x86_freebsd.json
x64_False_arm64_android.json    x86_False_x86_linux.json
x64_False_arm64_mac.json        x86_False_x86_netbsd.json
x64_False_arm64_win.json        x86_False_x86_openbsd.json
x64_False_arm_android.json      x86_False_x86_win.json
x64_False_x64_android.json      x86_True_x86_freebsd.json
x64_False_x64_dragonfly.json    x86_True_x86_netbsd.json
x64_False_x64_freebsd.json      x86_True_x86_openbsd.json
x64_False_x64_linux.json        x86_True_x86_win.json
x64_False_x64_mac.json

Of these the arm64_False_arm64_linux.json, arm_False_arm_linux.json and x86_False_x86_linux.json files were all transferred over from ESR 78, having been amended by patch 78. When I examined these files I notice something relevant: they differ slightly from all of the other files in that in the ESR 91 files encoder_simulcast_proxy is referenced where vp8_encoder_simulcast_proxy was referenced on ESR 78. So this could well be our problem.

And it makes sense as well: in the code the V8 versions of the files aren't used any more, having been replaced by the generic encoder simulcaast proxy implementation. So it makes sense that the build scripts might need updating to reference the new files rather than the previous ones.

If this is the case, then the solution should be straightforward. All I need to do is switch the file names in the GN build files, then regenerate the moz.build files from these.

So, here is the change I've made to the GN build files. This mimics the changes that have already been made to the other GN build files (but which we're not actually using).

$ git diff -U0
diff --git a/dom/media/webrtc/third_party_build/gn-configs/
    arm64_False_arm64_linux.json b/dom/media/webrtc/third_party_build/
    gn-configs/arm64_False_arm64_linux.json
index f1e7bfac33b8..beeded9b8602 100644
--- a/dom/media/webrtc/third_party_build/gn-configs/arm64_False_arm64_linux.json
+++ b/dom/media/webrtc/third_party_build/gn-configs/arm64_False_arm64_linux.json
@@ -4143,2 +4143,2 @@
-                "//media/engine/vp8_encoder_simulcast_proxy.cc",
-                "//media/engine/vp8_encoder_simulcast_proxy.h",
+                "//media/engine/encoder_simulcast_proxy.cc",
+                "//media/engine/encoder_simulcast_proxy.h",
diff --git a/dom/media/webrtc/third_party_build/gn-configs/
    arm_False_arm_linux.json b/dom/media/webrtc/third_party_build/gn-configs/
    arm_False_arm_linux.json
index 9885557f56cf..f3a87ac3b260 100644
--- a/dom/media/webrtc/third_party_build/gn-configs/arm_False_arm_linux.json
+++ b/dom/media/webrtc/third_party_build/gn-configs/arm_False_arm_linux.json
@@ -4408,2 +4408,2 @@
-                "//media/engine/vp8_encoder_simulcast_proxy.cc",
-                "//media/engine/vp8_encoder_simulcast_proxy.h",
+                "//media/engine/encoder_simulcast_proxy.cc",
+                "//media/engine/encoder_simulcast_proxy.h",
diff --git a/dom/media/webrtc/third_party_build/gn-configs/
    x86_False_x86_linux.json b/dom/media/webrtc/third_party_build/gn-configs/
    x86_False_x86_linux.json
index 27b49b928398..796fc1e920a5 100644
--- a/dom/media/webrtc/third_party_build/gn-configs/x86_False_x86_linux.json
+++ b/dom/media/webrtc/third_party_build/gn-configs/x86_False_x86_linux.json
@@ -4124,2 +4124,2 @@
-                "//media/engine/vp8_encoder_simulcast_proxy.cc",
-                "//media/engine/vp8_encoder_simulcast_proxy.h",
+                "//media/engine/encoder_simulcast_proxy.cc",
+                "//media/engine/encoder_simulcast_proxy.h",

Following this I'll need to regenerate the gecko build files that are actually used during the build process.

$ source `pwd`/obj-build-mer-qt-xr/rpm-shared.env
$ cd gecko-dev/
$ ./mach build-backend -b GnMozbuildWriter

I can see this has resulted in a large number of changes to the moz.build files, which I take to be a good sign. Now time to do a full clean rebuild of the code.

$ sfdk build -d -p --with git_workaround
[...]
Wrote: RPMS/SailfishOS-devel-aarch64/
    xulrunner-qt5-devel-91.9.1+git1+sailfishos.esr91.20240707214235.e32f3a780e7e-1.aarch64.rpm
Wrote: RPMS/SailfishOS-devel-aarch64/
    xulrunner-qt5-misc-91.9.1+git1+sailfishos.esr91.20240707214235.e32f3a780e7e-1.aarch64.rpm
Wrote: RPMS/SailfishOS-devel-aarch64/
    xulrunner-qt5-91.9.1+git1+sailfishos.esr91.20240707214235.e32f3a780e7e-1.aarch64.rpm
Wrote: RPMS/SailfishOS-devel-aarch64/
    xulrunner-qt5-debugsource-91.9.1+git1+sailfishos.esr91.20240707214235.e32f3a780e7e-1.aarch64.rpm
Wrote: RPMS/SailfishOS-devel-aarch64/
    xulrunner-qt5-debuginfo-91.9.1+git1+sailfishos.esr91.20240707214235.e32f3a780e7e-1.aarch64.rpm
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.B5jeR8
+ umask 022
+ cd gecko-dev-esr91/gecko-dev/gecko-dev
+ /bin/rm -rf /home/deploy/installroot
+ RPM_EC=0
++ jobs -p
+ exit 0

Fantastic! The build has gone through without errors either during compilation or linking. This is exciting progress. But it's not time to test the packages just yet as I also have to apply the remaining two patches. Both of these are marked as being part of task JB#53982, so I should apply them both together.

As we saw a couple of days ago, I've already updated patch 81 to use the new location of the libwebrtc code and I didn't notice any obvious issues with patch 80, so with any luck these will now apply without too much drama.

$ git am --3way ../rpm/
    0080-sailfishos-webrtc-Enable-GMP-for-encoding-decoding.-.patch
Applying: Enable GMP for encoding/decoding. JB#53982
Using index info to reconstruct a base tree...
M       dom/media/gmp/GMPSharedMemManager.cpp
A       media/webrtc/signaling/src/media-conduit/GmpVideoCodec.cpp
A       media/webrtc/signaling/src/media-conduit/GmpVideoCodec.h
A       media/webrtc/signaling/src/media-conduit/VideoConduit.cpp
A       media/webrtc/signaling/src/media-conduit/WebrtcGmpVideoCodec.cpp
A       media/webrtc/signaling/src/media-conduit/WebrtcGmpVideoCodec.h
A       media/webrtc/signaling/src/peerconnection/PeerConnectionImpl.cpp
Falling back to patching base and 3-way merge...
Auto-merging dom/media/webrtc/libwebrtcglue/WebrtcGmpVideoCodec.h
Auto-merging dom/media/webrtc/libwebrtcglue/WebrtcGmpVideoCodec.cpp
CONFLICT (content): Merge conflict in dom/media/webrtc/libwebrtcglue/
    WebrtcGmpVideoCodec.cpp
Auto-merging dom/media/webrtc/libwebrtcglue/VideoConduit.cpp
Auto-merging dom/media/webrtc/libwebrtcglue/GmpVideoCodec.h
Auto-merging dom/media/webrtc/jsapi/PeerConnectionImpl.cpp
Auto-merging dom/media/gmp/GMPSharedMemManager.cpp
error: Failed to merge in the changes.
Patch failed at 0001 Enable GMP for encoding/decoding. JB#53982
hint: Use 'git am --show-current-patch=diff' to see the failed patch
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am 
    --abort".

Ah, sadly it's not going to be quite as straightforward as I'd hoped. No matter. There are three hunks that have failed to apply cleanly to the WebrtcGmpVideoCode.cpp file. It looks like the reason might be an upstream change that has slightly amended the way strings are handled. The line that used to look like this:

tags.AppendElement(NS_LITERAL_CSTRING("h264"));

Now looks like this:

tags.AppendElement("h264"_ns);

We've seen this before all the way back on Day 35. Thankfully fixing it is a straightforward process, but I notice that NS_LITERAL_CSTRING() is used in several places in the new code as well, so I'll need to change those as well.

With these changes tackled the merge now goes through successfully.

$ git add dom/media/webrtc/libwebrtcglue/WebrtcGmpVideoCodec.cpp
$ git am --continue
Applying: Enable GMP for encoding/decoding. JB#53982

Now on to patch 81. Unfortunately this also fails to apply cleanly.

$ git am --3way ../rpm/
    0081-sailfishos-webrtc-Implement-video-capture-module.-JB.patch
Applying: Implement video capture module. JB#53982
Using index info to reconstruct a base tree...
M       old-configure.in
M       third_party/libwebrtc/webrtc/modules/video_capture/BUILD.gn
M       third_party/libwebrtc/webrtc/modules/video_capture/
    video_capture_internal_impl_gn/moz.build
error: patch failed: third_party/libwebrtc/webrtc/modules/video_capture/
    video_capture_internal_impl_gn/moz.build:60
error: third_party/libwebrtc/webrtc/modules/video_capture/
    video_capture_internal_impl_gn/moz.build: patch does not apply
error: Did you hand edit your patch?
It does not apply to blobs recorded in its index.
Patch failed at 0001 Implement video capture module. JB#53982
hint: Use 'git am --show-current-patch=diff' to see the failed patch
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am 
    --abort".

In this case the issue is different. It's specifically the moz.build file it fails on. But as we've seen, these moz.build files are autogenerated. In this case, it's the change to BUILD.gn that we can see in the patch (and which applied without issue) that I believe should then be propagated to moz.build when the build files are regenerated.

So it looks like the simple solution will be to remove the changes to moz.build from the patch, apply the patch, then regenerate the build files. I should be left with some similar changes to moz.build autogenerated for me. I've therefore manually edited patch 81 to remove the moz.build changes listed there.

$ git am --3way ../rpm/0081-sailfishos-webrtc-Implement-video-capture-modul
e.-JB.patch
Applying: Implement video capture module. JB#53982

Patch goes through okay. Now to regenerate the build files.

$ sfdk engine exec
$ sb2 -t SailfishOS-esr78-aarch64.default
$ source `pwd`/obj-build-mer-qt-xr/rpm-shared.env
$ cd gecko-dev/
$ ./mach build-backend -b GnMozbuildWriter
[...]
Finished reading 1165 moz.build files in 12.36s
Read 13 gyp files in parallel contributing 0.00s to total wall time
Processed into 7258 build config descriptors in 10.27s
GnMozbuildWriter backend executed in 25.27s
  1 total backend files; 1 created; 0 updated; 0 unchanged; 0 deleted
Total wall time: 48.95s; CPU time: 47.91s; Efficiency: 98%; Untracked: 1.06s
Glean could not be found, so telemetry will not be reported. You may need to 
    run |mach bootstrap|.

I'm a bit perturbed to discover that this hasn't changed the moz.build file in the way I was expecting. To be clear: it hasn't changed it at all. I'm not happy with this and it's going to require a bit more investigation.

Perhaps I need to clean out the build before attempting to regenerate the build files?

$ sfdk engine exec
$ sb2 -t SailfishOS-esr78-aarch64.default
$ cd ..
$ source `pwd`/obj-build-mer-qt-xr/rpm-shared.env
$ cd gecko-dev/
$ ./mach build-backend -b GnMozbuildWriter
config.status not found.  Please run |mach configure| or |mach build| prior to 
    building the ['GnMozbuildWriter'] build backend.
Glean could not be found, so telemetry will not be reported. You may need to 
    run |mach bootstrap|.

Well, no, clearly that's not going to do the job. After doing a complete rebuild I've tried again, but still without any changes resulting in the moz.build files.

$ ./mach build-backend -b GnMozbuildWriter
 0:03.89 ${PROJECT}/obj-build-mer-qt-xr/_virtualenvs/common/bin/python 
    ${PROJECT}/obj-build-mer-qt-xr/config.status --backend GnMozbuildWriter
Reticulating splines...
 0:04.87 File already read. Skipping: ${PROJECT}/gecko-dev/intl/components/
    moz.build
 0:08.18 File already read. Skipping: ${PROJECT}/gecko-dev/gfx/angle/targets/
    angle_common/moz.build

There are some warnings about files already having been read. I did some reading around this but there's no indication that it could be part of the problem. Part of the output generated from the process contains a list of all the GN build files it's consuming. Here's the list (edited for clarity):

Writing moz.build files based on the following gn configs: [
    'x64_False_arm_android.json', 'arm_True_arm_freebsd.json',
    'mips64_True_mips64_linux.json', 'arm64_True_arm64_linux.json',
    'x64_False_x64_openbsd.json', 'x64_True_x64_mac.json', 
    x64_False_x64_win.json', 'x64_True_x86_linux.json',
    'x86_True_x86_win.json', 'mips64_False_mips64_linux.json',
    'x64_True_arm64_win.json', 'arm64_True_arm64_openbsd.json',
    'ppc64_False_ppc64_linux.json', 'x64_False_arm64_mac.json',
    'x86_False_x86_linux.json', 'x86_False_x86_win.json',
    'arm_False_arm_linux.json', 'x64_False_x64_mac.json',
    'arm64_True_arm64_freebsd.json', 'x64_True_x64_netbsd.json',
    'arm64_False_arm64_linux.json', 'x64_False_x86_android.json',
    'x86_False_x86_openbsd.json', 'x64_True_x64_win.json',
    'arm64_False_arm64_openbsd.json', 'x86_True_x86_netbsd.json',
    'x86_True_x86_openbsd.json', 'x64_False_x64_android.json',
    'x64_False_x64_freebsd.json', 'x64_True_x86_android.json',
    'x64_True_x64_openbsd.json', 'arm_True_arm_netbsd.json',
    'x64_False_x64_netbsd.json', 'arm64_False_arm64_netbsd.json',
    'x64_True_arm_android.json', 'x64_False_x64_dragonfly.json',
    'x64_False_arm64_android.json', 'x64_True_x64_freebsd.json',
    'arm64_False_arm64_freebsd.json', 'x64_True_arm64_mac.json',
    'x64_False_x64_linux.json', 'x86_False_x86_freebsd.json',
    'arm_False_arm_openbsd.json', 'ppc64_True_ppc64_linux.json',
    'x64_True_x64_linux.json', 'x64_False_x86_linux.json',
    'x64_True_arm64_android.json', 'arm_True_arm_openbsd.json',
    'x64_False_arm64_win.json', 'arm64_True_arm64_netbsd.json',
    'arm_False_arm_netbsd.json', 'arm_False_arm_freebsd.json',
    'x64_True_x64_android.json', 'x64_True_x64_dragonfly.json',
    'x86_False_x86_netbsd.json', 'x86_True_x86_freebsd.json',
    'arm_True_arm_linux.json'
]

The three files of importance to us are the ones we've amended, which are the arm64_False_arm64_linux.json, arm_False_arm_linux.json and x86_False_x86_linux.json files. All of these appear in the list, which I'd take as a good sign if it weren't for the fact it doesn't appear to be having any effect.

In spite of all this I left the build running all day and it went through fine, so I thought I'd have a go at running it and testing it with some WebRTC test sites. First up I tried the LiveKit WebRTC Browser Test. The results are a bit of a mixed bag. Although the video codecs seem to be working correctly, when the site requests audio input it gets stuck in a holding position indefinitely. When I perform the same action on my desktop browser a permission box opens and the process continues once I've agreed for the permissions to be granted.


There are some errors output to the console as well:

Created LOG for EmbedLiteLayerManager
JavaScript warning: https://livekit.io/_next/static/chunks/
    main-4e25e037b50f6258.js, line 1: unreachable code after return statement
JavaScript warning: https://livekit.io/_next/static/chunks/
    main-4e25e037b50f6258.js, line 1: unreachable code after return statement
JavaScript error: resource://gre/modules/amInstallTrigger.jsm, line 43: 
    TypeError: this.mm is null
JavaScript error: resource://gre/modules/amInstallTrigger.jsm, line 43: 
    TypeError: this.mm is null
JavaScript error: resource://gre/modules/amInstallTrigger.jsm, line 43: 
    TypeError: this.mm is null
JavaScript error: file:///usr/lib64/mozembedlite/components/
    EmbedLiteWebrtcUI.js, line 293: TypeError: 
    contentWindow.navigator.mozGetUserMediaDevices is not a function

###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, 
    messages will be lost

This all suggests to me that the permission dialogues are currently broken on the ESR 91 build, so this is something I'll need to look into. I also had a play with Mozilla's WebRTC Tests.

Testing out the getUserMedia example shows similar problems. Pressing the camera button has no effect, while pressing the Microphone button generates a similar error to the console:

JavaScript error: file:///usr/lib64/mozembedlite/components/
    EmbedLiteWebrtcUI.js, line 293: TypeError: 
    contentWindow.navigator.mozGetUserMediaDevices is not a function

I have better success with the Single host DataChannels page. In this case I'm able to send messages backwards and forwards just as I can on a desktop browser.

So this is so far all pointing to an issue with the page for requesting permissions. I'm not going to have time to check this now, but there is one other thing I'd like to try.

Although I wasn't able to get the moz.build files updated automatically by generating them from the GN build scripts, I do have the patch that shows what ought to be changing in the moz.build. So I'm going to apply this change manually and run another rebuild. It'll be interesting to establish whether this has any effect. If all goes to plan, we can find out tomorrow.

If you'd like to read any of my other gecko diary entries, they're all available on my Gecko-dev Diary page.