[3.0] Add generator docs

docs/CHANGELOG.md

@@ -10,8 +10,10 @@ Please read [the Contributions Guide](CONTRIBUTING.md) before writing release no
 
 Silk.NET 3.0 Preview 1
 
-- Added SDL3 bindings using the 3.0 bindings style.
 - Added combined OpenGL bindings using the 3.0 bindings style.
+- Added OpenAL bindings using the 3.0 bindings style.
+- Added SDL3 bindings using the 3.0 bindings style.
+- Added Vulkan bindings using the 3.0 bindings style.
 - Improved the bindings style to be more accessible, IDE-friendly, and AOT-friendly. Learn more at **TODO ADD A LINK TO DOCUMENTATION HERE**.
 - Removed SDL2 bindings.
 

docs/for-contributors/Generator/api-specific-notes.md

@@ -0,0 +1,38 @@
+# API-Specific Notes
+
+This document's purpose is to note down any decisions or quirks that are specific to a library that Silk is generating
+bindings for.
+
+This is meant to be a living document. Please update this as new work is being done on the generator.
+
+## OpenAL
+
+Currently empty.
+
+## OpenGL
+
+Currently empty.
+
+## SDL
+
+Currently empty.
+
+## Vulkan
+
+There will be the following errors in the generation log. This is expected. These types are part of the XML
+specification, but not part of the main `vulkan.h` header.
+
+For more information: https://github.com/dotnet/Silk.NET/pull/2457#issuecomment-2910293716
+
+```
+fail: Silk.NET.SilkTouch.Mods.MixKhronosData[0] Enum "VkFullScreenExclusiveEXT" has no base type. Please add TypeMap entry to the configuration. This enum group will be skipped.
+fail: Silk.NET.SilkTouch.Mods.MixKhronosData[0] Enum "VkFaultLevel" has no base type. Please add TypeMap entry to the configuration. This enum group will be skipped.
+fail: Silk.NET.SilkTouch.Mods.MixKhronosData[0] Enum "VkFaultType" has no base type. Please add TypeMap entry to the configuration. This enum group will be skipped.
+fail: Silk.NET.SilkTouch.Mods.MixKhronosData[0] Enum "VkFaultQueryBehavior" has no base type. Please add TypeMap entry to the configuration. This enum group will be skipped.
+fail: Silk.NET.SilkTouch.Mods.MixKhronosData[0] Enum "VkPipelineMatchControl" has no base type. Please add TypeMap entry to the configuration. This enum group will be skipped.
+fail: Silk.NET.SilkTouch.Mods.MixKhronosData[0] Enum "VkSciSyncClientTypeNV" has no base type. Please add TypeMap entry to the configuration. This enum group will be skipped.
+fail: Silk.NET.SilkTouch.Mods.MixKhronosData[0] Enum "VkSciSyncPrimitiveTypeNV" has no base type. Please add TypeMap entry to the configuration. This enum group will be skipped.
+fail: Silk.NET.SilkTouch.Mods.MixKhronosData[0] Enum "VkPipelineCacheValidationVersion" has no base type. Please add TypeMap entry to the configuration. This enum group will be skipped.
+fail: Silk.NET.SilkTouch.Mods.MixKhronosData[0] Enum "VkDisplacementMicromapFormatNV" has no base type. Please add TypeMap entry to the configuration. This enum group will be skipped.
+fail: Silk.NET.SilkTouch.Mods.MixKhronosData[0] Enum "VkCompressedTriangleFormatAMDX" has no base type. Please add TypeMap entry to the configuration. This enum group will be skipped.
+```

docs/for-contributors/Generator/using-the-generator.md

@@ -0,0 +1,231 @@
+# Using the Generator
+
+**Warning: The generator is still in heavy development and will likely be subject to frequent breaking changes.**
+This will likely be the case until we are a few previews in.
+
+SilkTouch is Silk 3's bindings generator, which works by taking the output of ClangSharpPInvokeGenerator, which itself
+is its own bindings generator, and modifying the output with a set of mods. These mods apply transformations such as
+renaming identifiers, creating types such as handle structs or enums, and adding method overloads.
+
+In other words, ClangSharpPInvokeGenerator acts as the input to the SilkTouch generator and SilkTouch works to improve
+on that input. Other inputs to the SilkTouch generator will likely be available in the future to cover APIs such as
+Metal.
+
+**Also note that only C bindings are supported right now. COM will be available later.**
+
+## Generator Overview
+
+There are two main things to configure:
+
+1. Silk 3 - This is the [`generator.json`](https://github.com/dotnet/Silk.NET/blob/develop/3.0/generator.json) file.
+
+2. ClangSharpPInvokeGenerator - This is the [`eng/silktouch`](https://github.com/dotnet/Silk.NET/tree/develop/3.0/eng/silktouch) folder.
+
+Both are organized by native API.
+
+Note: For the average C API, SDL's generator configuration would be the best configuration to reference. Most options
+used for the SDL bindings should be applicable after replacing the SDL-specific paths and values to suit the C API that
+you are binding.
+
+## SilkTouch Configuration
+
+Silk defines the configuration for SilkTouch in the
+[`generator.json`](https://github.com/dotnet/Silk.NET/blob/develop/3.0/generator.json) file. This file defines the
+different bindings jobs and the list of mods to run for each job.
+
+For example, when binding to a C API:
+
+- `AddIncludes` tells `ClangScraper` where to find the system header files. You likely want to include this.
+
+- `ClangScraper` runs ClangSharpPInvokeGenerator. Including this mod on its own is equivalent to running
+  ClangSharpPInvokeGenerator directly.
+
+- The rest of the mods apply different transformations to the output of `ClangScraper`. Documentation for the other mods
+  can be found in the [Generator Mods](generator-mods.md) documentation.
+
+Aside from reading documentation, some other ways to learn about the mods are to:
+
+- Read through the tests. The tests act as examples for specific behaviors expected by each mod, with configurations,
+  inputs, and expected outputs provided for each case.
+
+- Add them one by one. Mods run in the order you define them and work off the output of the previous mod.
+
+(TODO: Not sure how to set up bindings test projects. This refers to the `TestProject` property in `generator.json`.)
+
+## ClangSharpPInvokeGenerator Configuration
+
+Due to SilkTouch using ClangSharpPInvokeGenerator as an input, ClangSharpPInvokeGenerator must also be configured.
+Silk stores its configuration for ClangSharpPInvokeGenerator in the
+[`eng/silktouch`](https://github.com/dotnet/Silk.NET/tree/develop/3.0/eng/silktouch) folder as `.rsp` files (response
+files). These response files store command line arguments to be passed into ClangSharpPInvokeGenerator.
+
+Note that these files can be stored anywhere since the SilkTouch configuration lets you configure where the SilkTouch
+generator looks for these response files.
+
+> To read more about ClangSharpPInvokeGenerator's command line arguments, a good option is to install the tool directly
+> and use `--help` to display its command line documentation.
+>
+> ```sh
+> dotnet tool install --global ClangSharpPInvokeGenerator
+> ClangSharpPInvokeGenerator --help
+> ClangSharpPInvokeGenerator --config help
+> ```
+
+Aside from simply storing the command line arguments to be passed into ClangSharpPInvokeGenerator, response files can
+also import other response files using the `@path` syntax. For example: `@../settings.rsp`.
+
+Silk commonly uses these import paths to share settings between different sets of bindings, such as the
+[common.rsp](https://github.com/dotnet/Silk.NET/blob/develop/3.0/eng/silktouch/common.rsp) file for general shared
+settings and the [remap-stdint.rsp](https://github.com/dotnet/Silk.NET/blob/develop/3.0/eng/silktouch/remap-stdint.rsp)
+file used to ensure that the `stdint.h` types behave consistently between Windows and Linux.
+
+Please note that these paths are relative to the response file specified in the generator and **not** relative to the
+response file the `@path` directive is actually defined in.
+
+For example, Silk's SDL bindings sets `ClangSharpResponseFiles` to be `eng/silktouch/sdl/**/generate.rsp`. Therefore,
+any import paths used in the response files, including transitively imported response files, must be relative to
+the matched `generate.rsp` file.
+
+### Example Response File Folder Structure
+
+This is the general structure of the `eng/silktouch` folder:
+
+```
+eng
+- silktouch
+  - opengl <-- This level contains folders per native API.
+    - glcompat <-- This level contains folders for each "profile", which represent variants of the API.
+    - glcore
+    - gles1
+    - gles2
+  - sdl
+    - SDL3
+```
+
+Profiles likely will not be relevent for most C APIs, so the examples here will keep focusing on the SDL case.
+
+The following is the folder structure used for Silk's SDL bindings. You do not have to structure it the way Silk does.
+Silk's structure focuses on keeping consistency in its response file organization, regardless of whether the API makes
+use of profiles or not.
+
+```
+eng
+- silktouch
+  - sdl
+    - SDL3
+      - generate.rsp <-- The main settings file. Used as the entrypoint. All import paths must be relative to this file.
+      - header.txt
+      - sdl-SDL.h <-- Handwritten header file that #includes the relevant headers of the library you want to bind.
+    - remap.rsp
+    - settings.rsp <-- Shared settings for all profiles.
+```
+
+### Example ClangSharpPInvokeGenerator Configuration
+
+This section will now focus on how to actually create the response files, starting with the `sdl-SDL.h`, `generate.rsp`,
+and `settings.rsp` files. The snippets below will only contain the most important sections of those files for brevity.
+
+`sdl-SDL.h`:
+```h
+#include <SDL3/SDL.h>
+#include <SDL3/SDL_main.h>
+#include <SDL3/SDL_vulkan.h>
+```
+
+`generate.rsp`:
+```rsp
+@../settings.rsp
+@../remap.rsp
+--exclude
+SDL_SetX11EventHook
+SDL_SetWindowsMessageHook
+SDL_FILE
+SDL_LINE
+--file
+sdl-SDL.h
+--methodClassName
+Sdl
+--namespace
+Silk.NET.SDL
+--output
+../../../../sources/SDL/SDL3
+--traverse
+../../../submodules/sdl/include/SDL3/SDL_assert.h
+../../../submodules/sdl/include/SDL3/SDL_atomic.h
+../../../submodules/sdl/include/SDL3/SDL_audio.h
+```
+
+`settings.rsp`:
+```rsp
+@../../common.rsp
+--define-macro
+TODO_DEFINE_MACROS=HERE
+--headerFile
+header.txt
+--include-directory
+../../../submodules/sdl/include
+--with-callconv
+*=Winapi
+--with-librarypath
+*=SDL3
+```
+
+#### Relevant Options from `generate.rsp`:
+
+- `--file` specifies the header file to use as the entrypoint. This should be the custom header you defined.
+
+- `--traverse` specifies which header files contribute towards the output.
+
+This separation is because while header files, such as system headers, may be required to compile the library, we do not
+want to include those headers as part of the final set of generated bindings.
+
+- `--output` should point to the same `Jobs.JOB_NAME.SourceProject` path you defined in `generator.json`.
+
+- `--methodClassName` specifies which C# class contains the generated methods/constants.
+
+- `--namespace` specifies the C# namespace of the generated files.
+
+- `--exclude` allows you exclude types/functions/constants from the output. These usually are APIs that are not useful,
+  do not generate correctly, or are platform-specific.
+
+#### Relevant options from `settings.rsp`:
+
+- `--headerFile` specifies the header file appended to the top of every generated file. Silk uses this to inject its
+  copyright headers.
+
+- `--include-directory` specifies the include directories to be used. This affects all headers included, such as
+  `sdl-SDL.h`.
+
+- `--with-librarypath` is the name of the native library without prefixes/suffixes. If the library name differs outside
+  of the usual `lib` prefix or `.dll`/`.so`/`.dylib` suffixes, the way to handle this is to add `UseAlternativeName` in
+  the generated bindings. An example with Vulkan can be found in
+  [`sources/Vulkan/Vulkan/Vk.cs`](https://github.com/dotnet/Silk.NET/blob/develop/3.0/sources/Vulkan/Vulkan/Vk.cs),
+  which is a manually written file.
+
+```cs
+static Vk()
+{
+    LoaderInterface.RegisterHook(Assembly.GetExecutingAssembly());
+    LoaderInterface.RegisterAlternativeName("vulkan", "vulkan-1");
+    LoaderInterface.RegisterAlternativeName("vulkan", "MoltenVK");
+}
+```
+
+### Generated Bindings Output
+
+All generated binding will be output to the `Jobs.JOB_NAME.SourceProject` path defined in `generator.json`.
+
+These generated files all have the `.gen.cs` suffix and most of them are partial type declarations.
+This means by creating a similarly named `.cs` file and using the `partial` C# keyword, you can add to the type.
+
+Do not modify the `.gen.cs` files since running the generator again will overwrite those changes.
+
+### Packing the Generated Bindings
+
+(TODO: This section needs verification.)
+
+`dotnet pack` should simply work here.
+
+If you are contributing to Silk's repository, Silk automatically packs and pushes changes made during a pull request
+to its experimental NuGet feed. More information on how to access this feed is available in the
+[Experimental Feed](../../silk.net/experimental-feed.md) documentation.

docs/silk.net/static-vs-instance-bindings.md

@@ -1,4 +1,4 @@
-# Static vs Input Bindings
+# Static vs Instance Bindings
 
 ## Overview
 

sources/SilkTouch/SilkTouch/Mods/Common/ModUtils.cs

@@ -160,17 +160,24 @@ public static string GetMethodDiscriminator(BaseParameterSyntax param) =>
         GetMethodDiscriminator(param.Modifiers, param.Type);
 
     /// <summary>
-    /// Gets the relative path for this document.
+    /// Gets the path relative to the document's project path for the specified document.
     /// </summary>
     /// <param name="doc">The document.</param>
     /// <returns>The relative path.</returns>
     public static string? RelativePath(this Document doc)
     {
-        if (
-            doc.FilePath is null
-            || doc.Project.FilePath is null
-            || Path.GetDirectoryName(doc.Project.FilePath) is not { Length: > 0 } dir
-        )
+        if (doc.FilePath is null)
+        {
+            return default;
+        }
+
+        // Handle projects with no path by simply returning the document path
+        if (doc.Project.FilePath is null)
+        {
+            return doc.FilePath;
+        }
+
+        if (Path.GetDirectoryName(doc.Project.FilePath) is not { Length: > 0 } dir)
         {
             return default;
         }

sources/SilkTouch/SilkTouch/Mods/ExtractNestedTyping.cs

@@ -17,7 +17,7 @@ namespace Silk.NET.SilkTouch.Mods;
 /// <list type="bullet">
 /// <item><description>
 /// Replacing function pointers identified by their <see cref="NativeTypeNameAttribute"/>s with delegates and
-/// <c>Pfn</c>-prefixed structures.
+/// function pointer structs.
 /// </description></item>
 /// <item><description>
 /// Moving constants into their respective enums. These constants are identified by checking for an enum with

sources/SilkTouch/SilkTouch/Mods/MarkNativeNames.cs

@@ -3,6 +3,7 @@
 
 using Microsoft.CodeAnalysis;
 using Microsoft.CodeAnalysis.CSharp.Syntax;
+using Silk.NET.SilkTouch.Clang;
 
 namespace Silk.NET.SilkTouch.Mods;
 
@@ -11,8 +12,8 @@ namespace Silk.NET.SilkTouch.Mods;
 /// </summary>
 /// <remarks>
 /// This mod is currently kept pretty dumb and just applies [NativeName] attributes to almost everything.
-/// Syntax nodes not output by ClangSharp are intentionally not processed.
-/// This mod is best placed directly after ClangScraper.
+/// Syntax nodes not output by <see cref="ClangScraper"/> are intentionally not processed.
+/// This mod is best placed directly after <see cref="ClangScraper"/>.
 /// </remarks>
 public class MarkNativeNames : IMod
 {

sources/SilkTouch/SilkTouch/Mods/TransformHandles.cs

@@ -15,16 +15,19 @@
 namespace Silk.NET.SilkTouch.Mods;
 
 /// <summary>
-/// Identifies handle types by finding pointers to empty structs or missing types.
+/// Identifies handle types by finding pointers to empty structs.
 /// In general, a handle type is a struct that wraps an underlying opaque pointer (or some other underlying value).
 /// These handle types are then transformed by making the struct wrap the underlying pointer and
 /// reducing the dimension of pointers referencing that handle type by one.
 /// </summary>
 /// <example>
 /// Given an empty struct, <c>struct VkBuffer</c>, and all usages of that struct are through a pointer,
-/// <c>VkBuffer*</c>, usages of that pointer will be replaced by <c>VkBufferHandle</c>. For a 2-dimensional pointer,
-/// <c>VkBuffer**</c>, the resulting replacement is <c>VkBufferHandle*</c>.
+/// <c>VkBuffer*</c>, usages of that pointer will be replaced by <c>VkBuffer</c>. For a 2-dimensional pointer,
+/// <c>VkBuffer**</c>, the resulting replacement is <c>VkBuffer*</c>.
 /// </example>
+/// <remarks>
+/// The `Handle` suffix is not applied until <see cref="PrettifyNames"/> executes.
+/// </remarks>
 [ModConfiguration<Config>]
 public class TransformHandles(
     IOptionsSnapshot<TransformHandles.Config> config,

tests/SilkTouch/SilkTouch/ArrayParameterTransformerTests.cs

@@ -1,12 +1,10 @@
 // Licensed to the .NET Foundation under one or more agreements.
 // The .NET Foundation licenses this file to you under the MIT license.
 
-using System;
 using System.Diagnostics.CodeAnalysis;
 using Microsoft.CodeAnalysis;
 using Microsoft.CodeAnalysis.CSharp;
 using Microsoft.CodeAnalysis.CSharp.Syntax;
-using NUnit.Framework;
 using Silk.NET.SilkTouch.Mods.Metadata;
 using Silk.NET.SilkTouch.Mods.Transformation;
 

tests/SilkTouch/SilkTouch/Caching/CacheTests.cs

@@ -1,8 +1,6 @@
 // Licensed to the .NET Foundation under one or more agreements.
 // The .NET Foundation licenses this file to you under the MIT license.
 
-using System.Collections;
-using System.Diagnostics;
 using System.IO.Compression;
 using System.Runtime.CompilerServices;
 using System.Text;