flypig.co.uk

Before I get in to my diary entry today I want to add a reminder that I'll not be posting diaries next week, or the week after. Next week I'll be attending a conference and I need a bit of time to sort out a few other things in my wider life. So this will give me the chance to do that. But this is only a temporary gap; I'll be back straight after to continue where I left off.

With that out of the way, let's get on with the entry for today. If you've read any of my diary entries over the last few days you'll know I've been trying to extract something useful from the GLScreenBuffer::Swap() method. I added some code in to the method to read off pixel data from the render surface which generated some plausible looking output.

But I wasn't totally convinced: it looked to me like there were too many zero entries that I couldn't explain

So today I've been trying to do a couple of things. First I tried to get something to compare against; second I tried to make the code that captures the colour at a point a little more general by also saving out the entire image buffer to disk.

Let's tackle these in order. The obvious way to get something to compare against is to add the same code to the ESR 78 library and try running that. And this is exactly what I've been doing. The surrounding code in ESR 91 is almost identical to that in ESR 78, so it's a pretty straightforward task to just copy over the code changes from ESR 91 to ESR 78.

Having done this, built the library and deployed it to my phone, here's what's output when I now execute the browser:

=============== Preparing offscreen rendering context ===============
Colour before: (0, 0, 0, 255), 1
Colour after: (0, 0, 0, 255), 1
Colour before: (0, 0, 0, 255), 1
Colour after: (0, 0, 0, 255), 1
Colour before: (0, 0, 0, 255), 1
Colour after: (0, 0, 0, 255), 1
Colour before: (0, 0, 0, 255), 1
Colour after: (0, 0, 0, 255), 1
Colour before: (0, 0, 0, 255), 1
[...]
Colour before: (0, 0, 0, 255), 1
Colour after: (0, 0, 0, 255), 1
Colour before: (0, 0, 0, 255), 1
Colour after: (0, 0, 0, 255), 1
Colour before: (45, 91, 86, 255), 1
Colour after: (45, 91, 86, 255), 1
Colour before: (0, 0, 0, 255), 1
Colour after: (0, 0, 0, 255), 1
Colour before: (0, 0, 0, 255), 1
Colour after: (0, 0, 0, 255), 1
Colour before: (0, 0, 0, 255), 1
Colour after: (0, 0, 0, 255), 1
Colour before: (0, 0, 0, 255), 1
Colour after: (0, 0, 0, 255), 1
Colour before: (0, 0, 0, 255), 1
Colour after: (0, 0, 0, 255), 1
Colour before: (128, 13, 160, 255), 1
Colour after: (128, 13, 160, 255), 1
Colour before: (0, 0, 0, 255), 1
Colour after: (0, 0, 0, 255), 1
Colour before: (0, 0, 0, 255), 1
Colour after: (0, 0, 0, 255), 1
[...]

There are a couple of lines of interesting output here, but most of them just show black pixels and nothing else. That's not ideal. To be clear, this is output from a WebView app that is fully working. So I'd have expected to see a lot more colour data coming out in the debug output.

This has left me more confused than enlightened, so I've gone on to implement the second idea as well: exporting the image data to file so I can check what the image actually looks like.

Once again the code for this is pretty straightforward. All I'm doing is taking a copy of the entire surface, rather than just one pixel of it. This is provided as a contiguous block of memory: a raw buffer with the pixel values stored in it. So I'm just dumping this out to a file. To avoid completely thrashing my phone I've set it up to output an image only once in every ten renders. The filenames increment each time an image is exported, so I should capture several steps as the page completes is render.

Here's the code I'm using for this, added to the GLScreenBuffer::Swap() method. This is hacky code at best, but it's also temporary, so I'm not too concerned about the style here. Something quick and dirty is fine, as long as... well, as long as it works!

  static int count = 0;
  static int filecount = 0;
  size_t bufferSize;
  uint8_t* buf;
  bool result;
  int xpos;
  int ypos;
  int pos;
  volatile char red;
  volatile char green;
  volatile char blue;
  volatile char alpha;

  if (count % 10 == 0) {
    //bool GLScreenBuffer::ReadPixels(GLint x, GLint y, GLsizei width, GLsizei 
    height, GLenum format, GLenum type, GLvoid* pixels)
    bufferSize = sizeof(char) * size.width * size.height * 4;
    buf = static_cast<uint8_t*>(calloc(sizeof(uint8_t), bufferSize));
    result = ReadPixels(0, 0, size.width, size.height, LOCAL_GL_RGBA, 
    LOCAL_GL_UNSIGNED_BYTE, buf);

    xpos = size.width / 2;
    ypos = size.height / 2;
    pos = (xpos + (size.width * ypos)) * 4;

    red = buf[pos];
    green = buf[pos + 1];
    blue = buf[pos + 2];
    alpha = buf[pos + 3];

    printf_stderr("Colour before: (%d, %d, %d, %d), %d\n", red, 
    green, blue, alpha, result);

    #define FORMAT "/home/defaultuser/Documents/Development/gecko/
    frame%03d.dat"

    // Export out the pixel data
    int const len = 61 + 10;
    char filename[61 + 10];
    snprintf(filename, len, FORMAT, filecount);
    FILE *fh = fopen(filename, "w");
    fwrite(buf, sizeof(char), bufferSize, fh);
    fclose(fh);
    free(buf);
    filecount += 1;
  }
  count += 1;

After building and running the code I get some sensible looking output. As you can see there are sixteen frames generated before I quit the application. The first five look empty, but eventually some colours start coming through.

frame000.data: Colour before: (0, 0, 0, 255), 1
frame001.data: Colour before: (0, 0, 0, 255), 1
frame002.data: Colour before: (0, 0, 0, 255), 1
frame003.data: Colour before: (0, 0, 0, 255), 1
frame004.data: Colour before: (0, 0, 0, 255), 1
frame005.data: Colour before: (187, 125, 127, 255), 1
frame006.data: Colour before: (67, 115, 196, 255), 1
frame007.data: Colour before: (18, 0, 240, 255), 1
frame008.data: Colour before: (162, 225, 0, 255), 1
frame009.data: Colour before: (128, 202, 255, 255), 1
frame010.data: Colour before: (0, 0, 0, 255), 1
frame011.data: Colour before: (240, 255, 255, 255), 1
frame012.data: Colour before: (255, 159, 66, 255), 1
frame013.data: Colour before: (68, 115, 196, 255), 1
frame014.data: Colour before: (0, 0, 0, 255), 1
frame015.data: Colour before: (0, 192, 7, 255), 1

After copying the frame data from my phone over to my laptop I'm able to load them into GIMP (the GNU Image Manipulation Package) using the raw plugin. This is activated for files with a .data extension and allows the data to be loaded in as if it were pure pixel data without any header or metadata. Because there's no header you have to specify certain parameters manually, such as the width, height and format of the image data.

I always forget exactly what the dimensions of the screens on my development Xperia 10 II devices are, but thankfully GSMArena is only a few clicks away to check:
 
Adding the dimensions into the raw data loader seems to do the trick.

The code I added to gecko requested a texture format of RGBA, so that's what I need to use when loading the data in. Sadly the results are not what I had hoped. There are clearly some data related to the render and it's worth noting that the buffer where these are stored is initialised to contain zeroes each frame, so the data is real, not just artefacts from the memory or previous render.

But most of the pixels are black, the colours are wrong and the images seem to be mangled in very strange ways as you can see in the screenshots.

It's too late for me to figure this out tonight so I'll have a think about it overnight and come back to it in the morning.

As always, if you'd like to read any of my other gecko diary entries, they're all available on my Gecko-dev Diary page.

I spent a bit of time yesterday trying to extract colour data from the SharedSurface_EGLImage object that should, as far as I understand it, be providing the raster image to be rendered to the screen. My plan has been to override the SharedSurface::ReadPixels() method — a method that currently just returns false and does nothing more — in order to make it perform the useful task of returning colour information. Since I've already set up the code to call this, once this method is working everything else should magically fall in to place.

I tried adding code into the method body to perform the task like this:

bool SharedSurface_EGLImage::ReadPixels(GLint x, GLint y, GLsizei width, 
    GLsizei height,
                        GLenum format, GLenum type, GLvoid* pixels) {
  const auto& gl = GLContextEGL::Cast(mDesc.gl);

  gl->fBindTexture(LOCAL_GL_TEXTURE_2D, mProdTex);
  gl->fGetTexImage(LOCAL_GL_TEXTURE_2D, 0, format, type, pixels);
  return true;
}

But I hit a problem when the glGetTexImage() symbol, needed for the call to fGetTexImage(), wasn't available to load dynamically from the EGL library. The error kept on coming back:

Can't find symbol 'glGetTexImage'.
Crash Annotation GraphicsCriticalError: |[0][GFX1-]: Failed to create 
    EGLConfig! (t=1.32158) [GFX1-]: Failed to create EGLConfig!

As I'm sure many of you will have noted, there is in fact no support for glGetTexImage() in EGL, as is explained very succinctly on Stack Overflow. So it doesn't matter how many different permutations I was going to try, this was never going to work.

Thankfully the Stack Overflow post has an alternative approach with methods that are all shown to already be dynamically loaded in GLContext. Consequently I've updated the code to align with the suggested approach and it now looks like this:

bool SharedSurface_EGLImage::ReadPixels(GLint x, GLint y, GLsizei width, 
    GLsizei height,
                        GLenum format, GLenum type, GLvoid* pixels) {
  const auto& gl = GLContextEGL::Cast(mDesc.gl);

  // See https://stackoverflow.com/a/53993894
  GLuint fbo;
  gl->raw_fGenFramebuffers(1, &fbo);
  gl->raw_fBindFramebuffer(LOCAL_GL_FRAMEBUFFER, fbo);
  gl->fFramebufferTexture2D(LOCAL_GL_FRAMEBUFFER, LOCAL_GL_COLOR_ATTACHMENT0, 
    LOCAL_GL_TEXTURE_2D, mProdTex, 0);

  gl->raw_fReadPixels(x, y, width, height, format, type, pixels);

  gl->raw_fBindFramebuffer(LOCAL_GL_FRAMEBUFFER, 0);
  gl->raw_fDeleteFramebuffers(1, &fbo);

  return true;
}

This doesn't seem to be entirely what I want because it's really just copying the texture data back into a framebuffer and reading from there, which is very similar to what I've already done but in a slightly different place in the code. But this at least brings us one step close to the screen, which is where this all eventually is supposed to end up.

Although all of the EGL symbols used in the code I've added exist and are technically usable, not all of them are available because of the way visibility of them is configured in GLContext. The raw_fBindFramebuffer() and raw_fDeleteFramebuffers() methods are both marked as private. I should explain that this is because these are all the raw variants of the methods. There are non-raw versions which have public visibility, but these have additional functionality I want to avoid triggering. I really do just want to use the raw versions.

To make them accessible to SharedSurface_EGLImage outside of GLContext I've therefore hacked their visibility like so:

//private:
public:
  void raw_fBindFramebuffer(GLenum target, GLuint framebuffer) {
    BEFORE_GL_CALL;
    mSymbols.fBindFramebuffer(target, framebuffer);
    AFTER_GL_CALL;
  }
[...]
// TODO: Make private again	
public:
  void raw_fDeleteFramebuffers(GLsizei n, const GLuint* names) {
    BEFORE_GL_CALL;
    mSymbols.fDeleteFramebuffers(n, names);
    AFTER_GL_CALL;
  }
private:
  void raw_fDeleteRenderbuffers(GLsizei n, const GLuint* names) {
    BEFORE_GL_CALL;
    mSymbols.fDeleteRenderbuffers(n, names);
    AFTER_GL_CALL;
  }
[...]

This wouldn't be especially noteworthy or interesting were it not for the fact that the changes I'm making are only temporary while I debug things, so I'll want to change them back before I commit any changes to the public repository. As we've already seem before, taking a note in these diary entries of things I need to reverse is a great way for me to keep track and make sure I can reverse the changes as easily as possible.

Having made all of these changes it's time to test things out. I've built and uploaded the code, and set the WebView app running:

[JavaScript Warning: "Layout was forced before the page was fully loaded. 
    If stylesheets are not yet loaded this may cause a flash of unstyled 
    content." {file: "https://sailfishos.org/wp-includes/js/jquery/
    jquery.min.js?ver=3.5.1" line: 2}]
Colour before: (0, 0, 0, 255), 1
Colour after: (0, 0, 0, 255), 1
Colour before: (0, 0, 0, 255), 1
Colour after: (0, 0, 0, 255), 1
Colour before: (0, 0, 0, 255), 1
Colour after: (0, 0, 0, 255), 1
Colour before: (0, 0, 0, 255), 1
Colour after: (0, 0, 0, 255), 1
Colour before: (0, 0, 0, 255), 1
Colour after: (0, 0, 0, 255), 1
Colour before: (0, 0, 0, 255), 1
[...]
Colour after: (107, 79, 139, 255), 1
Colour before: (137, 207, 255, 255), 1
Colour after: (137, 207, 255, 255), 1
Colour before: (135, 205, 255, 255), 1
Colour after: (135, 205, 255, 255), 1
Colour before: (132, 204, 254, 255), 1
Colour after: (132, 204, 254, 255), 1
Colour before: (0, 0, 0, 255), 1
Colour after: (0, 0, 0, 255), 1
Colour before: (10, 144, 170, 255), 1
Colour after: (10, 144, 170, 255), 1
Colour before: (0, 0, 0, 255), 1
Colour after: (0, 0, 0, 255), 1
Colour before: (59, 128, 13, 255), 1
Colour after: (59, 128, 13, 255), 1
Colour before: (108, 171, 244, 255), 1
Colour after: (108, 171, 244, 255), 1
Colour before: (185, 132, 119, 255), 1
Colour after: (185, 132, 119, 255), 1
Colour before: (0, 0, 0, 255), 1
Colour after: (0, 0, 0, 255), 1
[...]

As you can see, the surface starts off fully black but gets rendered until eventually it's a mix of colours. There are still a lot of cases where the pixels are just black, which makes me a bit nervous, but the colours in general look intentional. I think this is a good sign.

But I'd like to have some proper confirmation, so tomorrow I might see if I can get a full copy of the texture written out to disk so I can see what's really happening there.

If you'd like to read any of my other gecko diary entries, they're all available on my Gecko-dev Diary page.

I'm still in the process of trying to extract pixel colours from the surface used for transferring the offscreen buffer to the onscreen render. The snag I hit yesterday is that I wanted to use glGetTexImage(), but this symbol isn't being dynamically loaded for use with GLContext.

Consequently I've added it to the end of the coreSymbols table like this:

    const SymLoadStruct coreSymbols[] = {
        { (PRFuncPtr*) &mSymbols.fActiveTexture, {{ 
    "glActiveTexture", "glActiveTextureARB" }} },
        { (PRFuncPtr*) &mSymbols.fAttachShader, {{ "glAttachShader", 
    "glAttachShaderARB" }} },
        { (PRFuncPtr*) &mSymbols.fBindAttribLocation, {{ 
    "glBindAttribLocation", "glBindAttribLocationARB" }} },
        { (PRFuncPtr*) &mSymbols.fBindBuffer, {{ "glBindBuffer", 
    "glBindBufferARB" }} },
        { (PRFuncPtr*) &mSymbols.fBindTexture, {{ "glBindTexture", 
    "glBindTextureARB" }} },
[...]
        { (PRFuncPtr*) &mSymbols.fGetTexImage, {{ "glGetTexImage", 
    "glGetTexImageARB" }} },

        END_SYMBOLS
    };

I've also searched through the rest of the code for glBindBuffer, which is the reference I'm using for how to introduce glGetTexImage. Although it comes up in many places in the code, none of them look relevant to what I'm doing, so I'm going to guess that the change I've made above is all that's needed.

At least I may as well build it and find out.

Now when I run it I get this:

Created LOG for EmbedLiteLayerManager
Can't find symbol 'glGetTexImage'.
Crash Annotation GraphicsCriticalError: |[0][GFX1-]: Failed to create 
    EGLConfig! (t=1.32158) [GFX1-]: Failed to create EGLConfig!
Segmentation fault

That's not a great sign. I'm going to have to put some more thought into this.

This error message appears twice in the code so I can't be entirely certain, but it looks like the error is resulting in a failure in the following method:

bool CreateConfig(EglDisplay& egl, EGLConfig* aConfig, int32_t depth,
                  bool aEnableDepthBuffer, bool aUseGles, int aVisual);

At this stage I'm not really sure what I'm doing, but I've changed my earlier change so it now looks like this:

    const SymLoadStruct coreSymbols[] = {
        { (PRFuncPtr*) &mSymbols.fActiveTexture, {{ 
    "glActiveTexture", "glActiveTextureARB" }} },
        { (PRFuncPtr*) &mSymbols.fAttachShader, {{ "glAttachShader", 
    "glAttachShaderARB" }} },
        { (PRFuncPtr*) &mSymbols.fBindAttribLocation, {{ 
    "glBindAttribLocation", "glBindAttribLocationARB" }} },
        { (PRFuncPtr*) &mSymbols.fBindBuffer, {{ "glBindBuffer", 
    "glBindBufferARB" }} },
        { (PRFuncPtr*) &mSymbols.fBindTexture, {{ "glBindTexture", 
    "glBindTextureARB" }} },
[...]
        { (PRFuncPtr*) &mSymbols.fGetTexImage, {{ "glGetTexImage" }} 
    },

        END_SYMBOLS
    };

Sadly after rebuilding, re-uploading and re-running, that didn't fix it. I'm out of ideas for today, but maybe after a good night's sleep I'll have some better ideas tomorrow. Let's see in the morning.

If you'd like to read any of my other gecko diary entries, they're all available on my Gecko-dev Diary page.

Before I get in to my diary entry today I want to first mention a change that will be happening in a week. Between Saturday 4th May and Sunday 12th May I'll be attending a conference, which means I'll be pretty tied up throughout. So as I've done a couple of times in the past already, I'll be taking a break from these dev diaries during that period. It'll basically be a break of a week and a couple of days (Saturday to Sunday), but I'll get back on track again straight after again.

With that announcement out of the way, let's get on with the entry for today.

I'm currently digging around in GLScreenBuffer::Swap(), the idea being to put in some code to print out pixel colours. I added some yesterday, but it returned only zeros. Looking at the code the reason became clear pretty quickly: the ReadPixels() method from SharedSurface was being called, which always fails and just returns false.

Even though the surface is shown in the debugger as a SharedSurface it's supposed to be a SharedSurface_EGLImage. But SharedSurface_EGLImage doesn't override the ReadPixels() method, so even if it were, it still wouldn't make any difference.

But although the code I wrote yesterday doesn't work I'm hoping it can still lay the groundwork for something that will. What I've done is implemented that missing override, so that now there's a ReadPixels() method in SharedSurface_EGLImage that looks like this:

bool SharedSurface_EGLImage::ReadPixels(GLint x, GLint y, GLsizei width, 
    GLsizei height,
                        GLenum format, GLenum type, GLvoid* pixels) {
  const auto& gl = GLContextEGL::Cast(mDesc.gl);

  gl->fBindTexture(LOCAL_GL_TEXTURE_2D, mProdTex);
  gl->fGetTexImage(LOCAL_GL_TEXTURE_2D, 0, format, type, pixels);
  return true;
}

The idea is that this will bind the texture associated with the surface and then read the pixels from the currently bound texture. Sadly when I run it, the app crashes with a backtrace that looks like this:

Thread 37 "Compositor" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f178ee7e0 (LWP 2120)]
0x0000000000000000 in ?? ()
(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x0000007ff112e5f0 in mozilla::gl::GLContext::fGetTexImage (
    img=0x7ed81ea5d0, type=5121, format=6408, level=0, target=3553, 
    this=0x7ed819aa50)
    at ${PROJECT}/gecko-dev/gfx/gl/GLContext.h:1331
#2  mozilla::gl::SharedSurface_EGLImage::ReadPixels (this=<optimized out>, 
    x=<optimized out>, y=<optimized out>, width=<optimized out>, 
    height=<optimized out>, format=6408, type=5121, pixels=0x7ed81ea5d0)
    at ${PROJECT}/gecko-dev/gfx/gl/SharedSurfaceEGL.cpp:178
#3  0x0000007ff1101a34 in mozilla::gl::GLScreenBuffer::ReadPixels (
    this=this@entry=0x7ed4002e00, x=x@entry=540, y=y@entry=1260, 
    width=width@entry=1, 
    height=height@entry=1, format=format@entry=6408, type=type@entry=5121, 
    pixels=pixels@entry=0x7ed81ea5d0)
    at ${PROJECT}/gecko-dev/gfx/gl/GLScreenBuffer.cpp:462
#4  0x0000007ff11027e4 in mozilla::gl::GLScreenBuffer::Swap (
    this=this@entry=0x7ed4002e00, size=...)
    at ${PROJECT}/gecko-dev/gfx/gl/GLScreenBuffer.cpp:531
#5  0x0000007ff3663f48 in mozilla::gl::GLScreenBuffer::PublishFrame (size=..., 
    this=0x7ed4002e00)
    at ${PROJECT}/obj-build-mer-qt-xr/dist/include/GLScreenBuffer.h:229
#6  mozilla::embedlite::EmbedLiteCompositorBridgeParent::
    PresentOffscreenSurface (this=0x7fc4b422d0)
    at ${PROJECT}/gecko-dev/mobile/sailfishos/embedthread/
    EmbedLiteCompositorBridgeParent.cpp:191
#7  0x0000007ff367d3ac in mozilla::embedlite::nsWindow::PostRender (
    this=0x7fc4b09280, aContext=<optimized out>)
    at ${PROJECT}/gecko-dev/mobile/sailfishos/embedshared/nsWindow.cpp:248
#8  0x0000007ff2a62664 in mozilla::widget::InProcessCompositorWidget::
    PostRender (this=0x7fc4d0eb40, aContext=0x7f178ed7e8)
    at ${PROJECT}/gecko-dev/widget/InProcessCompositorWidget.cpp:60
#9  0x0000007ff128d1c8 in mozilla::layers::LayerManagerComposite::Render (
    this=this@entry=0x7ed81b7160, aInvalidRegion=..., aOpaqueRegion=...)
    at ${PROJECT}/obj-build-mer-qt-xr/dist/include/mozilla/layers/Compositor.h:
    575
#10 0x0000007ff128d644 in mozilla::layers::LayerManagerComposite::
    UpdateAndRender (this=this@entry=0x7ed81b7160)
    at ${PROJECT}/gecko-dev/gfx/layers/composite/LayerManagerComposite.cpp:657
#11 0x0000007ff128d9f4 in mozilla::layers::LayerManagerComposite::
    EndTransaction (this=this@entry=0x7ed81b7160, aTimeStamp=..., 
    aFlags=aFlags@entry=mozilla::layers::LayerManager::END_DEFAULT)
    at ${PROJECT}/gecko-dev/gfx/layers/composite/LayerManagerComposite.cpp:572
#12 0x0000007ff12cf174 in mozilla::layers::CompositorBridgeParent::
    CompositeToTarget (this=0x7fc4b422d0, aId=..., aTarget=0x0, 
    aRect=<optimized out>)
    at ${PROJECT}/obj-build-mer-qt-xr/dist/include/mozilla/RefPtr.h:313
#13 0x0000007ff3663c48 in mozilla::embedlite::EmbedLiteCompositorBridgeParent::
    CompositeToDefaultTarget (this=0x7fc4b422d0, aId=...)
    at ${PROJECT}/gecko-dev/mobile/sailfishos/embedthread/
    EmbedLiteCompositorBridgeParent.cpp:165
#14 0x0000007ff12b48d0 in mozilla::layers::CompositorVsyncScheduler::Composite (
    this=0x7fc4c9c050, aVsyncEvent=...)
    at ${PROJECT}/gecko-dev/gfx/layers/ipc/CompositorVsyncScheduler.cpp:256
#15 0x0000007ff12acd50 in mozilla::detail::RunnableMethodArguments<mozilla::
    VsyncEvent>::applyImpl<mozilla::layers::CompositorVsyncScheduler, void (
    mozilla::layers::CompositorVsyncScheduler::*)(mozilla::VsyncEvent const&), 
    StoreCopyPassByConstLRef<mozilla::VsyncEvent>, 0ul> (args=..., m=<optimized 
    out>, 
    o=<optimized out>) at ${PROJECT}/obj-build-mer-qt-xr/dist/include/
    nsThreadUtils.h:887
[...]
#27 0x0000007ff6a0289c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/
    clone.S:78
(gdb) 

Although the crash isn't ideal it does tell me one useful thing: the method is getting called, which means that the surface really is a SharedSurface_EGLImage after all. That's at least one part confirmed. The reason for the crash is less clear, given that all the inputs to the fGetTexImage() method look sensible. The reason must be because the fGetTexImage() function pointer is null. And indeed it is:

(gdb) frame 1
#1  0x0000007ff112e5f0 in mozilla::gl::GLContext::fGetTexImage (
    img=0x7ed81ea5d0, type=5121, format=6408, level=0, target=3553, 
    this=0x7ed819aa50)
    at ${PROJECT}/gecko-dev/gfx/gl/GLContext.h:1331
1331    in ${PROJECT}/gecko-dev/gfx/gl/GLContext.h
(gdb) p mSymbols.fGetTexImage
$5 = (void (*)(GLenum, GLint, GLenum, GLenum, GLvoid *)) 0x0
(gdb) 

Okay. I'm not sure what I can do about that. It could be that the reason is because there's no fGetTexImage() in the GLContext::InitImpl() method, which is used to set up all of these function pointers. It's worth noting that the call to fBindTexture() that happens right before the call to fGetTexImage() works just fine and fBindTexture() can be found in a table at the top of the GLContext.cpp file, so that adds weight to the claim. If that's the case, maybe I can add fGetTexImage in?

Unfortunately I don't have time to do that this evening, but it does at least give me a very clear plan for what I can be working on tomorrow morning.

If you'd like to read any of my other gecko diary entries, they're all available on my Gecko-dev Diary page.

As discussed yesterday, today I'm investigating GLScreenBuffer::Swap(). A bit of debugging has demonstrated that it's hit frequently during the WebView render loop and it seems pretty integral. I also notice that there's already some commented-out code in there for reading off and debug-printing some pixel colours.

My plan was to do the same: read off some pixel colours, print them to the console, see whether they're updating as they should be. So the fact there's already some code in there to do this is an encouraging sign.

Unfortunately the code that's there doesn't work. Here's a snippet of the code in question:

    // uint32_t srcPixel = ReadPixel(src);
    // uint32_t destPixel = ReadPixel(dest);
    // printf_stderr("Before: src: 0x%08x, dest: 0x%08x\n", srcPixel,
    // destPixel);

It looks like a pretty slick implementation — there's not much to it after all — so using this would be ideal. Unfortunately it won't compile. As you can see it references a method called ReadPixel(), but this method doesn't actually exist. It's not even clear to me what the method is supposed to do, so replicating it isn't an option either.

Consequently I've grabbed the code that I used earlier in the CompositorOGL::EndFrame() method, transferred it over to this Swap() method and tweaked it for the new context. The new code looks like this:

  int xpos = size.width / 2;
  int ypos = size.height / 2;
  size_t bufferSize = sizeof(char) * 4;
  uint8_t* buf = static_cast<uint8_t*>(calloc(sizeof(uint8_t), bufferSize));
  bool result = ReadPixels(xpos, ypos, 1, 1, LOCAL_GL_RGBA, 
    LOCAL_GL_UNSIGNED_BYTE, buf);
  int pos = 0;

  volatile char red = buf[pos];
  volatile char green = buf[pos + 1];
  volatile char blue = buf[pos + 2];
  volatile char alpha = buf[pos + 3];

  printf_stderr(&quot;Colour before: (%d, %d, %d, %d), %d\n&quot;, red, green, 
    blue, alpha, result);

As you can see, it grabs a pixel from the surface and prints the colour value to the console. This is the "before" version and there's also an "after" version which does the same right after a call to SharedSurface::ProdCopy() has been made which appears to make a copy of the front buffer on to the back buffer. That's to match the commented-out code that was there before.

Now, having built and run the application, I don't get any output. Debugging shows that the Swap() method is definitely being hit. But the code I added to sample the surface is never called. Stepping through the code the reason immediately becomes obvious: the section of code I added them to is wrapped in a condition:

  if (mCaps.preserve && mFront && mBack) {
    auto src = mFront->Surf();
    auto dest = mBack->Surf();
[...]
  }

I just assumed that the code that was there before was a good place to add the sampling code, but mCaps.preserve is set to false so this condition is never being met.

So I've now moved the new sampling code outside of this condition, rebuilt and installed.

When I run the new library, the code is now definitely being executed, but the sampling is showing a very uniform output:

[...]
Colour before: (0, 0, 0, 0), 0
Colour after: (0, 0, 0, 0), 0
Colour before: (0, 0, 0, 0), 0
Colour after: (0, 0, 0, 0), 0
Colour before: (0, 0, 0, 0), 0
Colour after: (0, 0, 0, 0), 0
Colour before: (0, 0, 0, 0), 0
Colour after: (0, 0, 0, 0), 0
Colour before: (0, 0, 0, 0), 0
Colour after: (0, 0, 0, 0), 0
Colour before: (0, 0, 0, 0), 0
Colour after: (0, 0, 0, 0), 0
Colour before: (0, 0, 0, 0), 0
Colour after: (0, 0, 0, 0), 0
[...]

This can only be for one of two reasons: either the sampling code is broken, or the transfer of the image to the surface is broken. To understand which is happening we can consider the extra hint that's there in the output: the last value on the line is giving the result value of the call to ReadPixels(). In other words this indicates whether or not the ReadPixels() call was successful. The value being returned is always zero, equivalent of false. So no, the answer is that the call is not being successful.

Stepping through the code shows why. Inside the GLScreenBuffer::ReadPixels() method there's a call to actual do the work that looks like this:

    return surf->ReadPixels(x, y, width, height, format, type, pixels);

There are many different types of surface with potentially different overrides for ReadPixels() that this could be calling, but in this specific case, stepping through takes us to this method:

  virtual bool ReadPixels(GLint x, GLint y, GLsizei width, GLsizei height,
                          GLenum format, GLenum type, GLvoid* pixels) {
    return false;
  }

As we can see, this implementation of the call makes no attempt to read the pixels and is guaranteed to return false. Now even though this is a SharedSurface method it doesn't follow that the surface is actually of that type. For example SharedSurface_EGLImage is a subclass of SharedSurface, but since it doesn't override this particular method, it'll use the underlying SharedSurface method instead.

The debugger is also showing it as a SharedSurface type, but I'm suspicious. But I'm also tired, so I'm going to return to this tomorrow morning. With any luck, it'll be possible to unravel what's happening here and fix it.

If you'd like to read any of my other gecko diary entries, they're all available on my Gecko-dev Diary page.