List items
Items from the current list are shown below.
Gecko
23 Aug 2023 : Day 7 #
Last night we were left with a bit of a mystery: is 14 greater than 5? If so, why does our build error claim our version of libclang is too old when it's asking for at least version 5 and we have version 14 installed?
You'll be happy to hear 14 is greater than five and maths isn't broken. The answer to the mystery should have occurred to me earlier because back in August 2021 I hit exactly the same problem building ESR 78. We can see that in the old patch 0036 which is also designed to circumvent this exact same clang version check.
I'll go into the details in shortly, but first I must tip my hat to mal who figured this out and let me know on IRC:
Mal is correct of course. Kudos also to filip.k on the Sailfish Forum who also gave a nice suggestion for what might be going wrong, saying that it might be a single digit check.
I'm immediately reminded of the legendary tale that Microsoft skipped Windows 9 for precisely this reason. The claim was that third party software was breaking because it tested for Windows 95 or 98 by just checking the first digit of the version number. The story is no doubt apocryphal, but also deliciously plausible.
In our case it's slightly different in that it's not checking the version number at all. Instead it's checking whether a particular function is present in the library (a function that's present in version 5, but not in version 4). This is failing for us for very specific reasons, which is why we have to remove the version check entirely.
The reason it's failing is because of the cross-compiling dance we do, building for multiple platforms (armv7hl, aarch64, i486) all on an i486 machine (which is what OBS and my local Sailfish SDK are both running on). We need to use i486 clang for the build process, but one of the other variants (in the case of the arm builds) for the actual compiled code.
As such the Python build process is expecting to find an arm version of the library, picks up the i486 version and chokes on it. We can see this if we negotiate ourselves inside the build engine and check the file it's complaining about:
So no need to rebuild libclang. And the other libraries we need have all now been rebuilt.
However I hit a bit of a problem with icu 69.1. Upgrading icu has a cascading effect on other packages, all of which need to be rebuilt against it. This is what OBS is great at, but I'm doing all this locally right now. So I'm going to continue using the older version until it becomes absolutely necessary.
Happily the NSPR upgrade works without a hitch.
So now we're off with the build again and very quickly there's a problem with the generation of our custom EmbedLite IPDL files. It seems the file format has actually changed since ESR 78. Not significantly, but enough to trigger an error.
IPDL is gecko's "Inter-process/thread-communication Protocol Definition Language". It's used by gecko to pass messages between different parts of itself and can be thought of a little like internal RPC.
The developer creates an IPDL file which describes classes and methods. These are processed to generate C++ header files for two classes: a parent and a child class which represent the parent and child process respectively. Glued together with some internal gecko magic, the parent can now communicate safely with the child (and vice versa) by calling the methods defined in the IPDL file.
Anyhow, the issue seems to be that the IPDL syntax was changed so that method annotations such as compress, priority and tainted are now prefixes rather than suffixes, as explained in Bugzilla bug #1689147 and which we can see in the upstream change set D103368.
Happily it's just a syntax change, so fixing the Embedlite IPDL files is just a case of fixing the syntax, no logic or API changes.
That seems like a good place to stop for today. Tomorrow I'll run the build to find out if the IPDL changes fixed things, and move on to the next error we're up against.
If you want to catch up on any previous entries, check out the full Gecko-dev Diary page.
You'll be happy to hear 14 is greater than five and maths isn't broken. The answer to the mystery should have occurred to me earlier because back in August 2021 I hit exactly the same problem building ESR 78. We can see that in the old patch 0036 which is also designed to circumvent this exact same clang version check.
I'll go into the details in shortly, but first I must tip my hat to mal who figured this out and let me know on IRC:
<mal> flypig: there is a patch in gecko to disable clang version check
<mal> patch 36
<mal> patch 36
Mal is correct of course. Kudos also to filip.k on the Sailfish Forum who also gave a nice suggestion for what might be going wrong, saying that it might be a single digit check.
I'm immediately reminded of the legendary tale that Microsoft skipped Windows 9 for precisely this reason. The claim was that third party software was breaking because it tested for Windows 95 or 98 by just checking the first digit of the version number. The story is no doubt apocryphal, but also deliciously plausible.
In our case it's slightly different in that it's not checking the version number at all. Instead it's checking whether a particular function is present in the library (a function that's present in version 5, but not in version 4). This is failing for us for very specific reasons, which is why we have to remove the version check entirely.
The reason it's failing is because of the cross-compiling dance we do, building for multiple platforms (armv7hl, aarch64, i486) all on an i486 machine (which is what OBS and my local Sailfish SDK are both running on). We need to use i486 clang for the build process, but one of the other variants (in the case of the arm builds) for the actual compiled code.
As such the Python build process is expecting to find an arm version of the library, picks up the i486 version and chokes on it. We can see this if we negotiate ourselves inside the build engine and check the file it's complaining about:
$ sfdk engine exec [mersdk@f1c636fdd533]$ sb2 -t SailfishOS-devel-aarch64.default [SB2 sdk-install] I have no name!@f1c636fdd533 # file gecko-dev/../obj-build-mer-qt-xr/lib/libclang.so.10 gecko-dev/../obj-build-mer-qt-xr/lib/libclang.so.10: ELF 32-bit LSB shared object, Intel 80386, version 1 (GNU/Linux), dynamically linked, BuildID[sha1]=53fda5b6446347702ed3b6163ba0f59b69a0c53e, strippedWe can even perform the same python steps that are in the build script manually and see that they fail. The code is trying to dynamically load the library and check whether a particular function exists, but this assumes that the library is in the right format, which it's not.
$ sfdk engine exec
[mersdk@f1c636fdd533]$ sb2 -t SailfishOS-devel-aarch64.default
[SB2 sdk-build] I have no name!@f1c636fdd533 gecko-dev $ python3
Python 3.8.11 (default, Aug 26 2022, 00:00:00)
[GCC 8.3.0 20190222 (Sailfish OS gcc 8.3.0-7)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import CDLL
>>> lib = CDLL("obj-build-mer-qt-xr/lib/libclang.so.10")
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib64/python3.8/ctypes/__init__.py", line 373, in __init__
self._handle = _dlopen(self._name, mode)
OSError: obj-build-mer-qt-xr/lib/libclang.so.10: wrong ELF class: ELFCLASS32
The good news is that we're already enforcing the correct library availability through our spec file, so we can safely skip this check. The check makes sense for the general build process, but we don't really need it. At least the error highlighted the fact we need to apply the same patch as for ESR 78 again.So no need to rebuild libclang. And the other libraries we need have all now been rebuilt.
However I hit a bit of a problem with icu 69.1. Upgrading icu has a cascading effect on other packages, all of which need to be rebuilt against it. This is what OBS is great at, but I'm doing all this locally right now. So I'm going to continue using the older version until it becomes absolutely necessary.
Happily the NSPR upgrade works without a hitch.
So now we're off with the build again and very quickly there's a problem with the generation of our custom EmbedLite IPDL files. It seems the file format has actually changed since ESR 78. Not significantly, but enough to trigger an error.
IPDL is gecko's "Inter-process/thread-communication Protocol Definition Language". It's used by gecko to pass messages between different parts of itself and can be thought of a little like internal RPC.
The developer creates an IPDL file which describes classes and methods. These are processed to generate C++ header files for two classes: a parent and a child class which represent the parent and child process respectively. Glued together with some internal gecko magic, the parent can now communicate safely with the child (and vice versa) by calling the methods defined in the IPDL file.
Anyhow, the issue seems to be that the IPDL syntax was changed so that method annotations such as compress, priority and tainted are now prefixes rather than suffixes, as explained in Bugzilla bug #1689147 and which we can see in the upstream change set D103368.
Happily it's just a syntax change, so fixing the Embedlite IPDL files is just a case of fixing the syntax, no logic or API changes.
That seems like a good place to stop for today. Tomorrow I'll run the build to find out if the IPDL changes fixed things, and move on to the next error we're up against.
If you want to catch up on any previous entries, check out the full Gecko-dev Diary page.
Comments
Uncover Disqus comments