Image-based lighting

[bagnell]

Adds the ability to add image-based lighting to a model.

• Model.sphericalHarmonicCoefficients is a property that is an array of 9 Cartesian3 coefficients used to evaluate the diffuse irradiance.
• Model.specularEnvironmentMaps is a property that is a url to a KTX file that contains a cubmap of the specular contribution plus the specular convolution mipmap levels.

TODO:

• Tests
• Update CHANGES.md

NOTE: This is a PR into the hdr branch.

[lilleyse]

Procedural IBL seems to be a lot brighter in this branch compared to master. Maybe tone down the default luminance?

Also the environment map seems a lot darker than it did before. Can that also be adjusted?

[lilleyse]

It might be good to look at the NYC building while tuning the default lighting. Right now they look washed out.

I would say master is too dark. We should aim for something in the middle that has a stronger contrast between the dark and light areas.

[lilleyse]

localhost example

The lighting orientation looks wrong in the NYC tileset compared to a model. After talking offline @bagnell thinks this is because the tiles don't have a model matrix and aren't sampling the environment map/spherical harmonics in the right direction.

[lilleyse]

In this demo with two models added I see TextureCache.addTexture getting called twice. In a separate demo with 3D Tiles I only see it get called once. Maybe there is some edge case with how the demo adds models.

Demo.

[lilleyse]

Textures will get cleaned up from the cache every 120 frames. Should this be longer?

I think this is fine. The texture cache code looks good.

[bagnell]

Also the environment map seems a lot darker than it did before. Can that also be adjusted?

This is what it looked like before:

The reason is because the prototype wasn't using the BRDF LUT. You can see by commenting out this line:
https://github.com/AnalyticalGraphicsInc/cesium/pull/7172/files#diff-6f6a773d137da52954260087ddc76a95R747

[bagnell]

@lilleyse This is ready for another look. The only thing I didn't change was the model in the IBL example. I could barely see any reflection from the environment map on that helicopter model.

[lilleyse]

@bagnell Try Pawns.glb.txt which cuts down on the sample model file size.

[bagnell]

@lilleyse This is ready for another review. If you think this is ready, I can update the Sandcastle example image and update CHANGES.

[lilleyse]

Nothing else from me. 👍

[bagnell]

@lilleyse The Sandcastle image and CHANGES have been updated. This should be ready to go.

[TimvanScherpenzeel]

Hi,

I was looking over the code (great work! 👍), as I'm looking to implement something similar, and was wondering about the spherical harmonics reconstruction in: https://github.com/AnalyticalGraphicsInc/cesium/blob/master/Source/Shaders/Builtin/Functions/sphericalHarmonics.glsl

From what I understand you are using the spherical harmonics generated with cmgen. There are a couple of things to be aware of when doing so because the developers of Filament have tried to pre-compute as much as possible in order to make reconstruction in the shader as cheap as possible.

This means that reconstruction in the shader should be as simple as:

As implemented here in the Filament engine:

https://github.com/google/filament/blob/f676d3d3ea55539774c7eefd894ff156c3f9cd86/shaders/src/light_indirect.fs#L127-L147

There is mention about Fd_Lambert() being baked into the SH, not really sure what that is about but could possibly lead to some brightness issues if not taken into account.

https://github.com/google/filament/blob/2c4ecebf11bdcd0099d869197d9c6ee1ff7bcfbf/shaders/src/light_indirect.fs#L442

Another thing I noticed is that cmgen automatically mirrors your cubemap (you can prevent this by passing the --no-mirror flag).

One last thing you might not have been aware of is that in the metadata of the outputted .ktx file the spherical harmonics are embedded meaning you could possibly save some extra manual steps.

[romainguy]

We do bake the division by pi from Lambert in the SH coefficients by default. cmgen can be invoked with --sh-output=file.txt -i to output coefficients with no pre-baking. The pre-baking does keep the reconstruction slightly simpler, it's always nice to save a few instructions (which would matter more when supporting local probes or SH-based light maps):

vec3 Irradiance_SphericalHarmonics(const vec3 n) {
    return max(
          frameUniforms.iblSH[0]
#if SPHERICAL_HARMONICS_BANDS >= 2
        + frameUniforms.iblSH[1] * (n.y)
        + frameUniforms.iblSH[2] * (n.z)
        + frameUniforms.iblSH[3] * (n.x)
#endif
#if SPHERICAL_HARMONICS_BANDS >= 3
        + frameUniforms.iblSH[4] * (n.y * n.x)
        + frameUniforms.iblSH[5] * (n.y * n.z)
        + frameUniforms.iblSH[6] * (3.0 * n.z * n.z - 1.0)
        + frameUniforms.iblSH[7] * (n.z * n.x)
        + frameUniforms.iblSH[8] * (n.x * n.x - n.y * n.y)
#endif
        , 0.0);
}