Integration of own OpenGL renderer

[llfab]

Before I start: I had posted a similar question about DirectX here. But this time it is about integration with OpenGL.

We are considering to use Avalonia within our .NET 5 based framework. We have our own 3D renderer which currently has a DirectX 11 backend but will be equipped with an additional OpenGL backend. This renderer is currently integrated into WPF. Because WPF is DirectX 9 based, the only option to make our renderer work 100% correct within WPF (unfortunately), was to draw the rendered texture into a Windows Forms surface and use a WindowsFormsHost within WPF tree.
I wonder whether this could be avoided using Avalonia. It would be beneficial to be able to integrate the rendered OpenGL texture directly into an Avalonia UI scene graph node.
This is targeting Windows and Linux as as the OS platform.

[kekekeks]

There is already a PR that integrates with OpenTK. It's not finished yet since I'm currently still suffering from covid repercursions and still need to complete macOS support, but it is already usable for Windows and Linux.

While PR is integrating OpenTK in particular, it introduces a generic infrastructure for creating OpenGL contexts and using those to draw to an offscreen surface that is later used by Avalonia's renderer.

The build is available as 0.10.999-cibuild0015264-beta from our https://nuget.avaloniaui.net/repository/avalonia-all/index.json feed

[BAndysc]

Great to hear the Windows version is already working!

I've been using old OpenGL shared context on MacOS with good success, but WGL was crashing constantly on Windows, therefore it was Mac only previously.

I have tried this beta build, I have moved things to OpenTK and I can confirm it works like a charm now on Windows. And on MacOS I can still use old shared context drawing instead of OpenTK. So it is quite a working solution for me already :) thanks!

[llfab]

I like this. OpenTK seems like a nice .net interop for OpenGL!

[BAndysc]

@kekekeks I've been using 0.10.999-cibuild0015264-beta beta build with a great success in my app https://github.com/BAndysc/WoWDatabaseEditor, however it looks like this build has been removed from the Avalonia nuget?

C:\Program Files\dotnet\sdk\6.0.100\NuGet.targets(130,5): error : The feed 'AvaloniaCI [https://nuget.avaloniaui.net/repository/avalonia-all/index.json]' lists package 'Avalonia.0.10.999-cibuild0015264-beta' but multiple attempts to download the nupkg have failed. The feed is either invalid or required packages were removed while the current operation was in progress. Verify the package exists on the feed and try again. [C:\projects\wowdatabaseeditor\LoaderAvalonia\LoaderAvalonia.csproj]
C:\Program Files\dotnet\sdk\6.0.100\NuGet.targets(130,5): error :   Unable to find package 'Avalonia.0.10.999-cibuild0015264-beta'. [C:\projects\wowdatabaseeditor\LoaderAvalonia\LoaderAvalonia.csproj]

The error sounds ambiguously to me, I am not sure if it is some problem with nuget.avaloniaui.net or simply the build has been removed. If the latter, is there any option to trigger the build again so that I could use it again from the feed?

The 3D view feature is essential in this app, so I am a little bit stuck right now.

[kekekeks]

We have suffered from a database corruption on our Nexus server, it's still reindexing some packages. I'll update the PR to the latest master so it will produce a new build.

[BAndysc]

@kekekeks

We have suffered from a database corruption on our Nexus server, it's still reindexing some packages. I'll update the PR to the latest master so it will produce a new build.

Thank you a lot! As I can see the PR still builds correctly on all platforms, but it looks like there is a problem publishing to the nuget.avaloniaui.net feed https://dev.azure.com/AvaloniaUI/AvaloniaUI/_releaseProgress?releaseId=12765&_a=release-environment-logs&environmentId=14528 Right? Is it still related to problems with the Nexus server? Sorry for the trouble 😅

EDIT:
Not sure if it happened automatically or you had fixed it, but now the package is published, thank you all for your hard work!

[llfab]

Thanks for the quick reply. I have also made the teapot sample work within a new Avalonia MVVM project template. So, maybe OpenGlControlBase would make sense both for rendering and taking the user inputs like picking or camera control. What says you?

[kekekeks]

OpenGlControlBase is just a regular control with a special bitmap attached to it.
You can do everything you'd expect to do with any other control, including allowing it to be focused and getting user input.

[llfab]

So, for the typical renderer integration features like resize (make the renderer use a different resolution and do not resize the resulting texture, control updates and user input you would still propose offscreen rendering?

[robloo]

As far as I understand things, OpenGL should not be used for new cross-platform applications. There are two reasons:

1. It isn't supported on macOS.
2. Newer, "more-powerful" from a performance perspective, alternatives exist

Vulkan with MoltenVK would make more sense as you haven't written the renderer yet.

[llfab]

True. Vulkan will be the future. But when it comes to effort our guys have the feeling that an additional render backend as an alternative to the existing DX11 (using SharpDX) and WebGL should be straight forward for OpenGL whereas for Vulkan may take much longer. Also I would hope we could use SkiaSharp.

[llfab]

Thanks. There is also the idea of OpenGL Vulkan texture sharing. However, I have detected some problems with WGL. Is there a way to use OpenGL rendering on windows without UseWgl=true?

[llfab]

A main problem was an error from WGL: cannot make context current. Even though there are no threads involved.

[BAndysc]

@llfab there is a PR with OpenTK integration by @ kekekeks, which thanks to a different method, works perfectly fine on Windows, without WGL. Even though it is not yet fully ready (i.e. no macos support) I am using it without any problems so far.

To use this PR, you need to include beta build of Avalonia, which is available as 0.10.999-cibuild0016933-beta from https://nuget.avaloniaui.net/repository/avalonia-all/index.json feed

[llfab]

I have looked at that branch, seeing that also ES/Angle backend would support texture sharing. Nice.

We are investigating how to make the Khronos group sample for texture sharing work. I still do not fully understand how I can "choose" a specific OpenGL (ES) version e.g., for the teapot sample. Any ideas?

In addition: in the source I can see that for Windows there would be the Wgl or the AngleWin32Egl option. You guys seem to prefer the AngleWin32Egl one, right?

[kekekeks]

I still do not fully understand how I can "choose" a specific OpenGL (ES) version e.g., for the teapot sample.

You need to either specify a pre-existing context via OpenGlControlSettings.Context or provide a factory method via OpenGlControlSettings.ContextFactory.

Manual context creation would look like this:

        static IGlContext Create(IList<GlVersion> probeVersions)
        {
            var feature = AvaloniaLocator.Current.GetService<IPlatformOpenGlInterface>();
            if (feature == null)
                throw new PlatformNotSupportedException("GL feature isn't enabled");

            if (feature.CanShareContexts)
                return feature.CreateContext(feature.PrimaryContext, probeVersions);

            return feature.CreateOSTextureSharingCompatibleContext(null, probeVersions);
        }

[mysteryx93]

A question about this OpenGL rendering within Avalonia app. I'm looking at integrating MPV media player.

Here it recommends using OpenGL Render API to integrate.

Would that currently work in Avalonia on all platforms? Any leads or issues to be aware of?

This seems to be their only sample integration code.

[kekekeks]

Looks pretty straightforward, specify a FBO and coordinates and it draws into the surface. Should work with OpenGLControlBase.

[mysteryx93]

Oh there's an OpenGL control in Avalonia? Sweet. Any kind of documentation related to it?

[BAndysc]

[I am just the user of Avalonia] From the user point of view, usage is very straightforward, just take a look at the sample https://github.com/AvaloniaUI/Avalonia/blob/master/samples/ControlCatalog/Pages/OpenGlPage.xaml.cs

However, do note that current master OpenGl control doesn't really work well on Windows, because of how it was implemented, it requires using WGL, which tends to be very unstable on Windows.

Luckily @ kekekeks has been working on OpenTK integration, which thanks to a different method, works perfectly fine on Windows. And it is already working very good, I am using it without any problems already (sample).

To use this PR, you need to include beta build of Avalonia, which is available as 0.10.999-cibuild0016933-beta from https://nuget.avaloniaui.net/repository/avalonia-all/index.json feed

[etkramer]

I read in another thread that you guys had previously intended to support generic DXGI surfaces. This would in turn allow images from Direct3D, Vulkan, and most other APIs to be used with varying degrees of difficulty.

Is this still part of the plan?

[kekekeks]

Yes, but most likely in January.

[profix898]

Is there any update on OpenGL interop for Avalonia 11? I'm specifically looking for OpenTK integration (PR #6391 was apparently abandoned). I found an old sample project (https://github.com/SamboyCoding/OpenTKAvalonia) which does not work with Avalonia 11 due to Compositor, GPUInterop etc. changes.

[kekekeks]

Unlike OpenTK, Silk.NET is compatible with OpenGlControlBase. You can find an example (for 0.10.x, not updated) here - https://github.com/kekekeks/Avalonia-Silk.NET-Example/tree/master

We are currently not planning to spend more resources on OpenTK integration.

[profix898]

Will look into Silk.NET then. Thanks!

[markusalbrecht-procam]

Hi,
a question from my side as we are currently trying to implement OPEN ES 3.1+ (OpenGL Version 4.6) via ANGLE into AvaloniaUI in order to create a 3D cad application. The problem we encountered is that the latest ANGLE only support V3.0 of OpenES - how can we integrate the latest one as we do not want to re-program the complete shader functionality that the latest OpgenGL version provides.

We are using latest AvaloniaUI and ANGLE

BR
Markus

[maxkatz6]

We do use up to date Angle version from Oct 6. https://github.com/AvaloniaUI/angle/commits/avalonia-fork-master/

[markusalbrecht-procam]

Thnaks for info, we mayben eed to upgrade to the latest state of Avalonia - the main solution still uses Version

Which one is the latest ?

[maxkatz6]

If you are using Avalonia 11.0.5 you can use latest angle by installing nuget package

<PackageReference Include="Avalonia.Angle.Windows.Natives" Version="2.1.22087.20231008" />

[maxkatz6]

Nightly builds already use this version just in case

[markusalbrecht-procam]

Thanks for the info, we will try that ;).

[markusalbrecht-procam]

Ok Guys I now installed the latest Angle.Windows.Natives Version 2.2.xxx and still Avalonia uses the version 3.0 what do we need to modify to use the latest ?

[anose001]

@markusalbrecht-procam, go to the Program.cs in your Desktop project, and add the following two With() into your AppBuilder configuration

var angleOptions = new AngleOptions
{
    GlProfiles = new[] { new GlVersion(GlProfileType.OpenGLES, 3, 1) }
};

AppBuilder.Configure<App>()
           .UsePlatformDetect()
           .With(angleOptions)
           .With(new Win32PlatformOptions { RenderingMode = new[] { Win32RenderingMode.AngleEgl } })
           .WithInterFont()
           .LogToTrace()
           .UseReactiveUI();

I believe that ANGLE on Desktop only supports version 3.1. I played around with using

win32Options = new Win32PlatformOptions
{
      RenderingMode = new[] { Win32RenderingMode.Wgl },
      WglProfiles = new[] { var glVersion = new GlVersion(GlProfileType.OpenGL, 4, 6);}
};

but there are known issues with crashes using WGL so that didn't pan out.

Hope this helps!

[maxkatz6]

Yes, Angle is only 3.0 stable and 3.1 experimental.