flypig.co.uk

For various reasons I had to clean out the repository and kick off a completely clean build. It takes a while, but is otherwise an unremarkable thing.

However, now it no longer fails on the errors I hit at the end of yesterday's session. There could be one of two reasons for this. It could be that the previous issue has been fixed by the cleanout. This is always a possibility if, due to the incremental build, a change doesn't have all the cascading effects it ought to have. Alternatively it could be because of the parallel build process. We're building with sixteen parallel jobs, meaning up to sixteen components can be building simultaneously. A small change can therefore cause a re-ordering of the failures.

Instead I now get the following error (abridged for clarity).

10:27.28 error: linking with `i686-unknown-linux-gnu-gcc` failed: exit status: 1
10:27.28   |
10:27.28   = note: "i686-unknown-linux-gnu-gcc" "-m32"
           "obj-build-mer-qt-xr/release/build/webrender-e0dea14bb1724795
           /build_script_build-e0dea14bb1724795.12luv6p7bzvqqwun.rcgu.o"
[...]
           "SailfishOS-devel-aarch64.default/usr/lib/rustlib
           /i686-unknown-linux-gnu/lib/libcore-a463ad6716e26c15.rlib"
           "-Wl,--end-group" "SailfishOS-devel-aarch64.default/usr/lib/rustlib
           /i686-unknown-linux-gnu/lib/libcompiler_builtins-1e7df035dceb93b6.rlib"
           "-Wl,-Bdynamic" "-lstdc++" "-lgcc_s" "-lutil" "-lrt" "-lpthread"
           "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-L"
           "SailfishOS-devel-aarch64.default/usr/lib/rustlib/i686-unknown-linux-gnu/lib"
           "-o" "obj-build-mer-qt-xr/release/build/webrender-e0dea14bb1724795
           /build_script_build-e0dea14bb1724795"
           "-Wl,--gc-sections" "-pie" "-Wl,-zrelro,-znow" "-nodefaultlibs"
10:27.29   = note: /usr/bin/ld: obj-build-mer-qt-xr/release/deps
           /libglslopt-e6e81689b2825cc3.rlib(glsl_optimizer.o): 
           Relocations in generic ELF (EM: 183)
[...]
10:27.32           /usr/bin/ld: obj-build-mer-qt-xr/release/deps
                   /libglslopt-e6e81689b2825cc3.rlib: error adding symbols:
                   file in wrong format
10:27.32           collect2: error: ld returned 1 exit status

This looks like a Rust issue: that is, the rust compiler building an object file for x86 when it should be building it for aarch64. We had to deal with these during the ESR 78 build as well and they were a real pain to fix (maybe because of my poor understanding of the Rust toolchain). But maybe this will turn out to be the same as something we saw back then. If that's the case, knowing the details could make this a whole lot easier to fix. So I need to check the patches.

A quick grep -rIn "glslopt" rpm/* throws up something that looks promising. Patch 0037 has the title "Patch glslopt to build on arm". That definitely looks like the same problem, so the next step is to try out the same patch.

A quick visual inspection suggests the patch will fail, but let's try it anyway.

$ patch -d gecko-dev -p1 < rpm/0037-sailfishos-configure-Patch-glslopt-to-build-on-arm.patch
patching file third_party/rust/glslopt/.cargo-checksum.json
Hunk #1 FAILED at 1.
1 out of 1 hunk FAILED -- saving rejects to file third_party/rust/glslopt/.cargo-checksum.json.rej
patching file third_party/rust/glslopt/build.rs
Hunk #1 FAILED at 1.
Hunk #2 succeeded at 26 with fuzz 2 (offset 7 lines).
Hunk #3 FAILED at 140.
2 out of 3 hunks FAILED -- saving rejects to file third_party/rust/glslopt/build.rs.rej

I'm not too worried about the checksum patch; that was always going to fail. The code in build.rs has changed a bit, so I'm not surprised the patch didn't apply completely, but the single hunk that did apply saved me a bit of time manually copying it over. Applying the remaining two failed hunks manually was pretty straightforward.

Kicking the build off I find I'm following it far more intensely than usual. This is a critical patch, it would be really great if it succeeded, and given the unpredictable ordering of the process I'd really like to know when (if) it's gone through. That is, when the webwender component, the one that needs this library, goes through.

Sadly, it doesn't. It seems the patch will need some more work after all.

21:03.89    Compiling glslopt v0.1.9
21:03.95 error[E0433]: failed to resolve: use of undeclared crate or module `env`
21:03.95   --> ${PROJECT}/gecko-dev/third_party/rust/glslopt/build.rs:36:11
21:03.95    |
21:03.95 36 |     match env::var("TARGET") {
21:03.95    |           ^^^ use of undeclared crate or module `env`
21:03.96 error[E0433]: failed to resolve: use of undeclared crate or module `env`
21:03.96   --> ${PROJECT}/gecko-dev/third_party/rust/glslopt/build.rs:45:11
21:03.96    |
21:03.96 45 |     match env::var("SB2_TARGET") {
21:03.96    |           ^^^ use of undeclared crate or module `env`
21:03.96 error[E0433]: failed to resolve: use of undeclared crate or module `env`
21:03.96   --> ${PROJECT}/gecko-dev/third_party/rust/glslopt/build.rs:53:19
21:03.96    |
21:03.96 53 |     let out_dir = env::var("OUT_DIR").unwrap();
21:03.96    |                   ^^^ use of undeclared crate or module `env`
21:03.96 error[E0433]: failed to resolve: use of undeclared crate or module `env`
21:03.96    --> ${PROJECT}/gecko-dev/third_party/rust/glslopt/build.rs:101:19
21:03.96     |
21:03.96 101 |     let out_dir = env::var("OUT_DIR").unwrap();
21:03.96     |                   ^^^ use of undeclared crate or module `env`
21:03.96 error[E0433]: failed to resolve: use of undeclared crate or module `env`
21:03.96    --> ${PROJECT}/gecko-dev/third_party/rust/glslopt/build.rs:129:19
21:03.97     |
21:03.97 129 |     let out_dir = env::var("OUT_DIR").unwrap();
21:03.97     |                   ^^^ use of undeclared crate or module `env`
21:03.99    Compiling whatsys v0.1.2
21:03.99 For more information about this error, try `rustc --explain E0433`.
21:04.00    Compiling audioipc v0.2.5 (https://github.com/mozilla/audioipc-2
            ?rev=7537bfadad2e981577eb75e4f13662fc517e1a09#7537bfad)
21:14.57    Compiling mozglue-static v0.1.0 (${PROJECT}/gecko-dev/mozglue/static/rust)
21:14.58    Compiling rand v0.7.3
21:14.62 dom/media/mediasession
21:15.26    Compiling rust_cascade v0.6.0
21:15.34    Compiling mp4parse v0.11.5 (https://github.com/mozilla/mp4parse-rust
            ?rev=1bb484e96ae724309e3346968e8ffd4c25e61616#1bb484e9)
21:15.45    Compiling tokio-udp v0.1.1
21:15.48    Compiling tokio-tcp v0.1.1
21:15.52    Compiling tokio-uds v0.2.5
21:15.53 error: could not compile `glslopt` due to 5 previous errors

From the error the issue is immediately clear though. The std::env crate was previously included in the original code, so there was no need to add it using the patch. But that's changed: the original code doesn't use it. Adding it back in should help.

It's suddenly started to rain heavily here in Cambridgeshire. Storm Betty was in the news earlier today, so it's not a surprise and personally I've been looking forward to it. My window is open, I can hear the rain outside and the air is cooling after a hot day. This has nothing to do with the code of course, except that the local environment has a big impact on my coding frame of mind.

As the rain continues the build now throws up a new error.

 2:06.58 error: the listed checksum of `/home/flypig/Documents/Development/jolla
         /gecko-dev-esr91/gecko-dev/gecko-dev/third_party/rust/glslopt/build.rs` has changed:
 2:06.58 expected: 8bcf41e15f780dcda7da689faf08d65ef4973827a0ca17faecce98dbc404b270
 2:06.58 actual:   1efe2b58d7281c5c8cb5d9a912871015625523281e9c38d0d0a8d89f33c918fb

New errors (or, more specifically, not the old errors) are always encouraging. The .cargo_checksum.json file that's causing the error has at least one enormously long line it it, which causes massive slowdown in gedit, the editor I'm using to make these changes. So I fire it up in vim, make the checksum edit from there, and set the build off again.

Sadly this isn't enough though. The custom glslopt build code we just changed has further problems.

11:02.28 error: failed to run custom build command for `glslopt v0.1.9`
11:02.31 Caused by:
11:02.31   process didn't exit successfully: `obj-build-mer-qt-xr/release/build
           /glslopt-1c70b99116f08314/build-script-build` (exit status: 101)
11:02.31   --- stdout
11:02.31   ####################### Compiling: glsl-optimizer/src/compiler/glsl
           /glcpp/glcpp-lex.c
11:02.31   Build command: "host-cc" "-isystem" "obj-build-mer-qt-xr/release
           /build/glslopt-3bb4f22cadf6e656/out/../../../../include"
           "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g"
           "-fno-omit-frame-pointer" "-m32" "-march=i686" "-I"
           "glsl-optimizer/include" "-I" "glsl-optimizer/src/mesa" "-I"
           "glsl-optimizer/src/mapi" "-I" "glsl-optimizer/src/compiler" "-I"
           "glsl-optimizer/src/compiler/glsl" "-I"
           "glsl-optimizer/src/gallium/auxiliary" "-I"
           "glsl-optimizer/src/gallium/include" "-I" "glsl-optimizer/src" "-I"
           "glsl-optimizer/src/util" "-D__STDC_FORMAT_MACROS" "-DHAVE_ENDIAN_H"
           "-DHAVE_PTHREAD" "-DHAVE_TIMESPEC_GET" "-o"
           "obj-build-mer-qt-xr/release/build/glslopt-3bb4f22cadf6e656/out
           /glsl-optimizer/src/compiler/glsl/glcpp/glcpp-lex.o"
           "-c" "glsl-optimizer/src/compiler/glsl/glcpp/glcpp-lex.c"
11:02.32   Compile status: exit status: 0
11:02.32   ######## stdout:
11:02.32   ######## stderr:
11:02.32   #######################
[...]
11:02.49   ####################### Compiling: glsl-optimizer/src/mesa/main/imports.c
11:02.49   Build command: "host-cc" "-isystem"
           "obj-build-mer-qt-xr/release/build/glslopt-3bb4f22cadf6e656/out/..
           /../../../include"
           "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g"
           "-fno-omit-frame-pointer" "-m32" "-march=i686" "-I"
           "glsl-optimizer/include" "-I" "glsl-optimizer/src/mesa" "-I"
           "glsl-optimizer/src/mapi" "-I" "glsl-optimizer/src/compiler" "-I"
           "glsl-optimizer/src/compiler/glsl" "-I"
           "glsl-optimizer/src/gallium/auxiliary" "-I"
           "glsl-optimizer/src/gallium/include" "-I" "glsl-optimizer/src" "-I"
           "glsl-optimizer/src/util" "-D__STDC_FORMAT_MACROS" "-DHAVE_ENDIAN_H"
           "-DHAVE_PTHREAD" "-DHAVE_TIMESPEC_GET" "-o"
           "obj-build-mer-qt-xr/release/build/glslopt-3bb4f22cadf6e656/out
           /glsl-optimizer/src/mesa/main/imports.o"
           "-c" "glsl-optimizer/src/mesa/main/imports.c"
11:02.49   Compile status: exit status: 1
11:02.49   ######## stdout:
11:02.49   ######## stderr:
11:02.50   #######################
11:02.50   --- stderr
11:02.50   In file included from glsl-optimizer/src/util/u_queue.h:43,
11:02.50                    from glsl-optimizer/src/mesa/main/glthread.h:48,
11:02.50                    from glsl-optimizer/src/mesa/main/mtypes.h:42,
11:02.50                    from src/compiler/glsl/glcpp/glcpp-parse.y:32:
11:02.50   glsl-optimizer/src/util/u_thread.h: In function ‘u_thread_setname’:
11:02.51       pthread_setname_np(pthread_self(), name);
11:02.51       ^~~~~~~~~~~~~~~~~~
11:02.51       u_thread_setname
11:02.51   In file included from glsl-optimizer/src/util/u_queue.h:43,
11:02.52                    from glsl-optimizer/src/mesa/main/glthread.h:48,
11:02.52                    from glsl-optimizer/src/mesa/main/mtypes.h:42,
11:02.52                    from glsl-optimizer/src/compiler/glsl/glcpp
                            /pp_standalone_scaffolding.h:34,
11:02.52                    from glsl-optimizer/src/compiler/glsl/glcpp
                            /pp_standalone_scaffolding.c:30:
11:02.52   glsl-optimizer/src/util/u_thread.h: In function ‘u_thread_setname’:
11:02.53       pthread_setname_np(pthread_self(), name);
11:02.53       ^~~~~~~~~~~~~~~~~~
11:02.53       u_thread_setname
11:02.53   In file included from glsl-optimizer/src/util/u_queue.h:43,
11:02.53                    from glsl-optimizer/src/mesa/main/glthread.h:48,
11:02.53                    from glsl-optimizer/src/mesa/main/mtypes.h:42,
11:02.53                    from glsl-optimizer/src/compiler/glsl/glcpp/pp.c:28:
11:02.54   glsl-optimizer/src/util/u_thread.h: In function ‘u_thread_setname’:
11:02.54       pthread_setname_np(pthread_self(), name);
11:02.54       ^~~~~~~~~~~~~~~~~~
11:02.54       u_thread_setname
11:02.54   In file included from glsl-optimizer/src/util/u_queue.h:43,
11:02.54                    from glsl-optimizer/src/mesa/main/glthread.h:48,
11:02.54                    from glsl-optimizer/src/mesa/main/mtypes.h:42,
11:02.55                    from glsl-optimizer/src/mesa/main/extensions.h:39,
11:02.55                    from glsl-optimizer/src/mesa/main/extensions_table.c:26:
11:02.55   glsl-optimizer/src/util/u_thread.h: In function ‘u_thread_setname’:
11:02.55       pthread_setname_np(pthread_self(), name);
11:02.55       ^~~~~~~~~~~~~~~~~~
11:02.55       u_thread_setname
11:02.55   cc: error: glsl-optimizer/src/mesa/main/imports.c:
           No such file or directory
11:02.56   cc: fatal error: no input files
11:02.56   compilation terminated.

This looks like there's at least a problem with the glsl-optimizer/src/mesa/main/imports.c file I'm trying to import in the code. Going through the removed code and the new code I can see that yes, indeed, some of the includes have changed. I just applied the patch and hoped, but now I'll have to go through it line-by-line and compare.

It turns out it's not so much code, so not really such a big undertaking after all. But it does require some care and attention.

Since there are quite a few changes to both the source and include files. I've been through them all and tried to make them manually align once again. Once that's done and after updating the checksum once again, I start the build. Time to listen to the rain once more in the background.

While it's chugging away it might be worth taking a moment to consider why this patch is needed. The problem here is to do with the target the Rust code is being built for. The code that gets patched is a build.rs. That's a file written in Rust that's used to build something else. As such the binary that's output gets linked in to the build chain, rather than being linked in to the final executable.

When this code is executed, it itself executes various parts of the build tooling in order to build something that does get linked into the final executable.

The problem is, which tooling should it be built with and which should it invoke? We want it to be built with the tooling for the x86 target so that the binary output can be run without emulation on the host. But we want that code to invoke the aarch64 target tooling. To get this latter part right, we have to amend the code to use the correct tools.

The changes needed aren't huge, but they are scratchbox2-specific. For example, rather than calling gcc for the build, we want to call host-cc instead. That ensures we use the correct toolchain. We also need to point to the relevant system header files and so on. And we need to do different things depending on whether we're building for an x86, armv7hl or aarch64 target, so we need some code to check this too.

If you have a look at the patch you'll see this is exactly what it's intended to achieve.

I'm now back to watching the build as it gradually progresses. Often slowly, but then with a sudden surge of text rushing past the screen. I'm really hoping webrender goes through this time. But it's also late, so I've decide to let the build complete overnight. Knowing whether or not this fixed this step of the build will have to wait until morning.

So that's it for today. See my Gecko Dev Diary page for all the other posts on this topic.