Using Roslyn and unit tests to enforce coding guidelines and more

Last year, during a few of my Roslyn talks, I was presenting a cool idea of leveraging Roslyn in unit tests to enforce a certain style in code, and in general inspect the consistency of the code in various ways.

It’s a really powerful concept, and something I wanted to blog about, but of course forgot – until I was reminded of that yesterday on Twitter.

Let’s have a look.

The idea

The idea is quite simple – since Roslyn let’s you analyze the entire solution, in a very meta way you can analyze your current solution from the unit tests in that solution.

With that, you can do pretty much anything – verify naming styles, check whitespace or spacing requirements, and even dig into the semantic model of the code (more on that later. All I use in the following samples is the *Microsoft.CodeAnalysis.Workspaces.Common* Roslyn NuGet package, as well as *xUnit* and *Should* packages for tests and assertions.

Here’s a most basic example. Let’s check if all the interfaces in the solution start with I.

So in the constructor (test set up) we navigate up all the way to the solution file. We load the solution file using Roslyn and then iterate through all of the projects, picking up all of the source files – exposing them as a list of Document objects for our tests to process.

Then in the test we simply grab all InterfaceDeclarationSyntax instances, which will represent syntax nodes that declare interfaces, and check if all of their identifies start with capital letter I. Easy and brilliant!

Real life usage

If you are interested in a real life scenario, a few months ago we had a discussion with Jason, the mastermind behind OmniSharp about coding guidelines when contributing to OmniSharp. Jason is really obsessed about two things in the PRs – having spaces instead of tabs and having using statements sorted properly. Of course if you use Visual Studio, it will convert tabs to spaces automatically (unless you explicitly change that behavior), but especially in the OmniSharp world, where developers can use a wide array of editors, tabs can easily creep into the code base.

So instead of manually checking for those two things on every PR, we came up with an idea to use Roslyn in unit tests to verify those two things. You can find the code here, but let’s also paste it here for reference.

This is actually extremely simple – just inspect syntax tree of every source file and find out if there are any SyntaxKind.WhitespaceTrivia that happen to have tabs in them. If so the assertion will fail.

The other one is:

In this case it’s not just blindly checking for the using statements to be sorted alphabetically, but also verifying if the System ones are placed before anything else. And finding all using statements, in all files, with Roslyn, is as simple as grabbing the CompilationUnitSyntax of each syntax tree representing each file (there are some edge cases but for simplicity let’s not get in there).

Enhancing with semantic analysis

So let’s go further, and for example do something that can have some semantic benefit too – instead of purely inspecting the code from the syntactic perspective.

Imagine a scenario where you work on a financial project (and in C#, no F# :) ) so immutability is a bit painful to support and enforce in your team. So say you have a marker interface ICalculationResult and you want to ensure that any type that is any sort of calculation result (and implements ICalculationResult), is immutable.

So this would be acceptable:

But not this:

Here’s how you can write a test covering this scenario. The setup remains the same as in the first part of this post – we must grab a list of all documents to iterate through.

So we iterate the documents, and find all class declarations syntax nodes – by filtering on ClassDeclarationSyntax. For each of the documents, we need to grab a semantic model, using GetSemanticModelAsync method. Now we can iterate through the class syntax nodes of a given document, and get the ITypeSymbol representing each class. This will have invaluable semantic information such as the hierarchy of implemented interfaces – so we will not just know about ICalculationResult in case our class does Foo : ICalculationResult but also if any of its parent types implements ICalculationResult as well.

At that point it becomes very easy – just get all of the property symbols for each of the filtered class symbols and find out whether the properties on the class are all readonly (no setter at all) or at least not public. Now if an irresponsible developer waltzes in and breaks your immutability, the tests will fail immediately.

Bonus

and as a bonus, I’m just gonna leave this here, it’s rather self explanatory! This test should be mandatory in every code base. #endregions

Source code for this article is on Github.

Be Sociable, Share!

  • http://daveaglick.com Dave Glick

    I absolutely love this idea. Moving style enforcement to the test harness instead of hooking into the compile pipeline and spitting out warnings/errors like StyleCop, etc. does is probably where this sort of thing belonged in the first place.

  • http://m0sa.net/ m0sa

    “This test should be mandatory in every code base.” is actually an argument for writing this as a DiagnosticAnalyzer and packaging it up as a nuget package.

  • https://cmatskas.com/ Christos Matskas

    Great piece of work demonstrating the power of Roslyn. Would be interesting to see timings on larg(ish) solutions.

  • Nahid F

    Love this post, thanks, its great idea to use Roslyn in tests as well.

  • Jason Bock

    For anyone who likes the last example….I have some code that lets you wipe out regions from your entire code base :) https://github.com/JasonBock/Deregionizer

  • Bomj Kompot

    Can’t that immutability check be done by using a piece of reflection?

    • http://www.strathweb.com/ Filip W

      yeah it can – the simple example here, absolutely. Although using semantic analysis things become much more powerful. For example this code, instead of a unit test, could be part of a Roslyn analyzer which runs in real time in Visual Studio and you can catch these types of things at a time when code is being written. On top of that, you could combine it with some extra checks – i.e. verify if documentation is present on the members, whitespace rules etc., which is something you wouldn’t be able to do with reflection.

      • Bomj Kompot

        yeah, true.

  • Bomj Kompot

    Can’t that immutability check be done by using a piece of reflection?

    • http://www.strathweb.com/ Filip W

      yeah it can – the simple example here, absolutely. Although using semantic analysis things become much more powerful. For example this code, instead of a unit test, could be part of a Roslyn analyzer which runs in real time in Visual Studio and you can catch these types of things at a time when code is being written. On top of that, you could combine it with some extra checks – i.e. verify if documentation is present on the members, whitespace rules etc., which is something you wouldn’t be able to do with reflection.

      • Bomj Kompot

        yeah, true.

  • Søren Reinke

    Thanks for the post, love the idea of using Roslyn to do coding standards test :)