Questionable Fun with Source Generators

Posted by SarahElSaig on September 21, 2021

psst, if you don't care about the moaning skip to The Project

I had some run-ins with the old T4 template system back in Framework. That was kind of a mixed bag. On one hand you had complete control over the output so it can be whatever type you want. On the other hand, there was no built-in syntax highlight or code completion for either the target or the meta code, yoinking you all the way back as if you were coding on Notepad. Also even though you could work around it, the template was really made for creating 1 file from 1 template so if you needed multifile it resulted in some leftover garbage (eg. empty files in your Entity Framework tables directory).

Source Generators to the Rescue?

Recently I've learned that the analysis features in Roslyn let you create more than just analysers: Source Generators. From a developer experience perspective, this is pretty much a stright improvement over T4. It aleviates some of the pain points (though sadly keeps many), while only adding one limitation. That is, you can only generate source code for an IL langauage that can be compiled by Roslyn, like C#. This is perfectly fine, I don't think I ever needed T4 for anything else.

Now onto the good news:

  • The code generation logic is regular code with the usual syntax highlight and code completion.
  • One generator can create many files across the whole project that consumes it.
  • It works off the whole source tree, so all compiled files in your project, not just one designated template file.
  • Your input code is already parsed.
  • You can still use non-compilable additional files as reference if you want to, again as many as you want.

But to temper expectations:

  • The template code is still not highlighted or parsed.
    • Worse yet, you probably have to work with StringBuilder.
  • The additional files have to be marked in your project file which is not intuitive and barely documented. (though easy once you know what to do)
  • You can't edit or override code, not even your own. This is a strictly additive process. (partial classes help here)
  • The generator has to go into its own project.

Lots of Jank

This is one of those barely advertised features. Even though .Net 5 has been out for a while, the most official looking this I could find was a Microsoft Blog from the .Net 5 Preview days.

The weirdest idiosincracy is that even though it takes .Net 5, a source generator must be .Net Standard 2.0. That's right, not even the latest 2.1.

There is also a return to everyone's dear old friend, the ḏ̙̳̙̹e͚̘̼͉̯͈ͅṕ̲͎̼͙͉̦ͅe̫̠̺̯̜̦̝n͍͚͕dè̹ͅnc͓̥̟̪̼͈̘͞y̠ ̺͉̕h̫e͎̖l͍͚̗l҉͔̫̝̜̲̩̳. Right, didn't we all just miss it so much? So listen up, this is important: If you are on .Net 5 import these exact versions!

    <ItemGroup>
        <PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="3.8.0-4.final" PrivateAssets="all" />
        <PackageReference Include="Microsoft.CodeAnalysis.Analyzers" Version="3.0.0" PrivateAssets="all" />
    </ItemGroup>

Updating them to the latest non-prerelease version will break the build becuase it expects dependencies from .Net 6 preview.

If you don't add the magic <MSBuildWarningsAsErrors>CS8785</MSBuildWarningsAsErrors> to your csproj's PropertyGroup, any code gen error will be rendered as a warning instead of an error. Though your build will still fail because at this point no code is generated.

I personally wasn't able make it work with NuGet dependenceis (eg. Json.Net) inside the source generator project. It failed the build with missing file even if I added the same dependency to the consumer project. This happens from both .Net CLI and IDEs that rely on it (eg Rider, VS Code) - if it works in VS only I don't want to hear about it. In the end I didn't need it for what I was working on, but it's still disheartening. Supposedly there is a way to make it work, but it seemed too much work for what I was doing with this side project.

There is a debug mode if you add Debugger.Launch() into your ISourceGenerator.Execute method, but you've got to set it up.

The Project

I wanted to make something that might be actually useful, not just a playground project. (Also apparently the INotifyPropertyChanged has been done to death and it's pretty much the Hello World of this topic, even though I haven't seen it used in the wild yet.) So I went back to the question that put me on the path to this code gen rabbit hole: can we have cleaner constant string concatenation? So I made ConstantsGenerator, a source generator that lets your express path collections in structured form using XML and then recursively builds nested static classes with path constants inside them. This makes things much cleaner when you have those multi-level paths in ASP.Net Core. Yeah I know it sounds like a nonissue, but it's a big cleanup with larger web projects, okay?

Usage

Include this project using ProjectReference with the attributes you see below. Also Include your files into the AdditionalFiles item group:

    <ItemGroup>
        <ProjectReference Include="..\ConstantGenerator\ConstantGenerator.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="false" />
        <AdditionalFiles Include="*.constants.xml" />
    </ItemGroup>

Example

Say you want custom routes for your ASP.Net application, but also put it in a constants file so you can refer to them in your logic.

sample.routes.constants.xml

<?xml version="1.0" encoding="utf-8" ?>
<Management xmlns:g="https://github.com/DAud-IcI/constant-generator/">
    <!-- Without this, your class will be in the Constants namespace. That can work if the file names are unique. -->
    <g:Namespace Value="ConstantGenerator.Sample.Constants.Routes" />
    
    <!-- This is the default already so you can safely omit it. -->
    <g:Separator Value="/" />

    <Users />
    <Companies />
    <New>
        <User />
        <Company />
    </New>
    <Unassign>
        <User />
    </Unassign>
</Management>

The elements with the g namespace are optional configuration items, they are optional, must be direct children of the root and each must have a Value attribute.

  • Namespace: You can specifiy the generated class's namespace. If you omit it, it will use Constants.
  • Separator: The values in the constants are the full path from the root, with this string used as the separator. The default value is /.

It generates this class:

SampleRoutes.GeneratedConstant.cs

namespace ConstantGenerator.Sample.Constants.Routes
{
    public static class Management
    {
        public const string ThisRoute = "Management";

        public const string Users = "Management/Users";
        public const string Companies = "Management/Companies";

        public static class New
        {
            public const string ThisRoute = "Management/New";

            public const string User = "Management/New/User";
            public const string Company = "Management/New/Company";
        }

        public static class Unassign
        {
            public const string ThisRoute = "Management/Unassign";

            public const string User = "Management/Unassign/User";
        }
    }
}

One advantage of this generated code is cleanliness. If you did this manually you probably need auxiliary variables like this:

        public static class New
        {
            private const string NewPathBase = nameof(Management) + "/" + nameof(New);
            public const string User = NewPathBase + "/" + nameof(User);
            public const string Company = NewPathBase + "/" + nameof(Company);
        }

Don't get me wrong, nameof is great but just look at all this visual noise! We don't need that with the source generator beucase the name is generated from the same text as the value.

Whew, what a disastreously boring topic

Yeah, but disastreous is going to be on-brand here even when things actually work out. Enjoy your stay, there aren't any escape pods left on this derelict.

C# C# Abnormalcy Programming .Net