Mike Lindegarde... Online

Things I'm likely to forget.

Validating API input in ASP.NET Core 1.1 with FluentValidation

Jump to Instructions

It doesn't matter if your API is nothing more than a facade for simple CRUD operations built on-top of an Active Record implementation or if your API is merely the gateway into your complex Domain Driven Design that leverages the latest and greatest CQSR/ES patterns to scale on demand:  you need to validate your input.

Writing code to validate input is quite possibly one of the most tedious task ever created (right next to doing any sort of processing that involves any file format from the medical world).  Thankfully we are far from the days of having to manually handle that task.  

When I started my latest ASP.NET Core 1.1 project I wanted a more expressive way to handle validation.  Enter FluentValidation:  a small library that does an excellent job handling input validation (high level validation before you get into the heart of your business logic).  Below I show you the three phases my validation code went through before I finally end up where I probably should have started.

Getting Started

Before getting to far into this tutorial you'll want to make sure you:

The First Pass (Also Known as the Really Bad Idea)

I knew I wanted to use FluentValidation and I know that ASP.Net has built in model validation.  What I didn't know was how to bring them together.  Eventually I got there, but it took a few passes.  The first step was a given, add the required dependencies to my project.json file:

{
  "dependencies": {
    "Microsoft.NETCore.App": {
      "version": "1.1.0",
      "type": "platform"
    },
    "Microsoft.AspNetCore.Mvc": "1.1.0",
    // other dependencies removed for brevity
    "StructureMap.Microsoft.DependencyInjection": "1.3.0",
    "FluentValidation.AspNetCore": "6.4.0-beta9",
    "Serilog": "2.3.0",
    "Serilog.Extensions.Logging": "1.3.1",
    "Serilog.Sinks.Literate": "2.0.0"
  },
  // other sections removed for brevity
}

With that in place I wrote a few validators for the input into my POST and PUT action handlers.  You can find all of the documentation for FluentValidation here.  Below is an example command and it's related validator:

public class AddRecipe
{
	public Guid Id {get; set;}
	public Guid CreatedBy {get; set;}
	public DateTime CreatedAt {get ;set;}
	public string Title {get; set;}
	public string Instructions {get; set;}
	public List<string> Ingredients {get; set;}
}

public class AddRecipeValidator : AbstractValidator<AddRecipe>
{
	private static readonly DateTime MinDate = new DateTime(2000, 1, 1);
	private static readonly DateTime MaxDate = new DateTime(2100, 1, 1);

	public AddRecipeValidator()
	{
		RuleFor(cmd => cmd.Id).NotEmpty();
		RuleFor(cmd => cmd.CreatedBy).NotEmpty().NotEqual(cmd => cmd.Id);
		RuleFor(cmd => cmd.CreatedAt).Must(BeValidDate)
			.WithMessage("'Created At' must be a valid date");
		RuleFor(cmd => cmd.Title).Length(5, 100);
		RuleFor(cmd => cmd.Ingredients).NotEmpty();
	}

	private bool BeValidDate(DateTime input)
	{
		return input.Date > MinDate && input.Date < MaxDate;
	}
}

Next, I wanted a generic way to get the validator for a given class based on the class's type.  I didn't want to have to know what the exact type of the validator implementation was.  This seemed like a good time to use StructureMap (in reality, it wasn't).  Here's how you could use StructureMap if you really really wanted to (for some reason):

In your StructureMap scanner configuraiton add the following line:

s.ConnectImplementationsToTypesClosing(typeof(AbstractValidator<>));

ConnectImplementationsToTypesClosing will allow you to get an instance of a given validator knowing only the type of the object the validator handles.  I wanted to be able to get an instance of a validator using the same syntax I would use to get any other object from my IoC container.  While I generally avoid extension methods, this seems like a good place to put the pattern to use:

public static class StructureMapUtilities
{
	public static IValidator TryGetValidatorInstance<T>(this IContainer container)
	{
		return container.TryGetInstance<AbstractValidator<T>>();
	}
}

With that in place, you can validate the input to your controller's action methods as follows:

[HttpPost]
public IActionResult Post([FromBody,Required] AddIngredient command)
{
	if(command == null)
		return BadRequest();

	ValidationResult result = 
		_container
			.TryGetValidatorInstance<AddIngredient>()?.Validate(command);

	if(result?.IsValid == false)
		return BadRequest(result);

	return CreatedAtRoute("GetIngredient", new {id = Guid.NewGuid()}, null);
}

That's, ummm.... not good.  There are a few major problems with this solution:

  • It requires injecting my IoC container into the controller (leaky abstraction, I might as well have my repositories return IQueryable while I'm at it)
  • I have to type the same few lines at the beginning of every method where I want to validate the input

The Second (Less Bad) Solution

Surface level validation is kind of a cross-cutting concern right?  Aspect Oriented Programming is a way to handle cross-cutting concerns... AOP frequently uses attributes... I can use an attribute to handle my validation.  A few degrees short of Kevin Bacon and I have a new direction: ValidateInputAttribute:

public class ValidateInputAttribute : ActionFilterAttribute
{
	public override void OnActionExecuting(ActionExecutingContext context)
	{
		if(context.ModelState.IsValid)
			return;

		context.Result = new BadRequestObjectResult(context.ModelState);
	}
}

Progress.

This solution allowed me to pull the IoC container out of my controllers.  It also simplified the code for validating input:

[HttpPost]
[ValidateInput]
public IActionResult Post([FromBody,Required] AddReview command)
{
	return CreatedAtRoute("GetReview", new {id = Guid.NewGuid()}, null);
}

Unfortunately I still had one problem:  I can't inject my logging framework of choice into an Attribute.  Well, where there's a will there's a way... but if you have to work that hard to get something to work, there's probably a better solution...

The Third and Final Attempt (for Now)

Filters allow you to execute code in the MVC Action Pipeline prior to the action being executed but after the model had been bound via Action Filters.  This seems like the perfect place to solve my problem.

Before we can handle validation errors in a Filter we first need to update our Startup class's ConfigureServices method:

public IServiceProvider ConfigureServices(IServiceCollection services)
{
	services.AddMvc()
		.AddFluentValidation(
			fv => fv.RegisterValidatorsFromAssemblyContaining<Startup>());

	return services.AddStructureMap();
}

The highlighted lines above will configure things so that FluentValidation handles validating input for you and updates the ModelState accordingly.  Note, using this approach does not require StructureMap.

Next you need to add an implementation of IActionFilter to handle validation errors:

public class ValidateInputFilter : IActionFilter
{
	#region
	private readonly ILogger _logger;
	#endregion

	#region Constructor
	public ValidateInputFilter(ILogger logger)
	{
		_logger = logger.ForContext<ValidateInputFilter>();
	}
	#endregion

	#region IActionFilter Implementation
	public void OnActionExecuting(ActionExecutingContext context)
	{
		if(context.ModelState.IsValid)
			return;

		using(LogContext.PushProperties(BuildIdentityEnrichers(context.HttpContext.User)))
		{
			_logger.Warning("Model validation failed for {@Input} with validation {@Errors}",
				context.ActionArguments,
				context.ModelState?
					.SelectMany(kvp => kvp.Value.Errors)
					.Select(e => e.ErrorMessage));
		}

		context.Result = new BadRequestObjectResult(
			from kvp in context.ModelState
			from e in kvp.Value.Errors
			let k = kvp.Key
			select new ValidationError(ValidationError.Type.Input, null, k, e.ErrorMessage));
	}

	public void OnActionExecuted(ActionExecutedContext context)
	{
		// This filter doesn't do anything post action.
	}
	#endregion
}

The above code should be pretty straight forward.  If there are no errors in the given model, simply return and do nothing.  Let the next filter do it's thing.  If there is a problem with the ModelState, log it and let the client know that a Bad Request was made (HTTP status code 400).

Note, in the current version of FluentValidation if a null object is passed into your action method context.ModelState.IsValid will return true.  Given what I read here, that's not what I expected.

Conclusion

I double my current solution is perfect and I'm almost certain it'll go through another refactoring (or two) as I continue to work on the project.  Hopefully you found something useful above.  If not, thanks for taking the time to read this article and I would appreciate any feedback you might have.

You can find a working example on GitHub: https://github.com/mlindegarde/examples-validation-api

Useful links

The links below my help answer any questions you may have:

Debugging ASP.NET Core Web APIs with Swagger

Debugging APIs

Debugging .NET based RESTful APIs isn't really that difficult.  Once have your code base successfully passing all unit tests it's just a matter of having the right tools and knowing the URLs for all of the end points you need to test.  Usually you're doing this to verify that your API works as expected (authentication / authorization, HTTP status codes, location headers, response bodies, etc...)

For a long time now I've been using an excellent Chrome App called Postman.  Postman offers a lot of great features:

  1. Slick user interface
  2. Ability to save API calls as Collections
  3. You can access your Collections from any computer (using Chrome)
  4. It supports Environments (which allow you to setup environment variables)
  5. You can share Collections and Environments
  6. Test automation

So why not just stick with Postman?  Simple, it doesn't lend itself well to exploring an API.  That's not a problem for the API developer (usually); however, it is a problem for third parties looking to leverage your API (be it another team or another company).  Swagger does an excellent job documenting your API and making it much easier for other users to explore and test.

Using Swagger with an ASP.NET Core 1.0 Web API

Like most things in the .NET world, adding Swagger boils down to adding a NuGet package to your project.  I would assume you could still use the NuGet Package Manager Console; however, we'll just add the required package to our project.json file:

dependencies": {
    "Microsoft.NETCore.App": {
      "version": "1.0.0",
      "type": "platform"
    },
    "Swashbuckle": "6.0.0-beta901"
  },

Next you'll need to add a few lines to your Startup.cs file:

public void ConfigureServices(IServiceCollection services)
{
    // Add framework services.
    services.AddApplicationInsightsTelemetry(Configuration);
    services.AddMvc();
    services.AddSwaggerGen();
}

and:

public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
{
    loggerFactory.AddConsole(Configuration.GetSection("Logging"));
    loggerFactory.AddDebug();
 
    app.UseApplicationInsightsRequestTelemetry();
    app.UseApplicationInsightsExceptionTelemetry();
 
    app.UseMvc();
    app.UseSwagger();
    app.UseSwaggerUi();
}

Now you should be able to run your app and explore your API using Swagger by appending /swagger/ui to the Web API's base URL.  It would probably be a good idea to set the your project's Launch URL to the Swagger UI's URL.  You can set by right clicking on your project, selecting Properties, and navigating to the Debug tab.

Security via OperationFilters

In most situations you're going to need to add some sort of Authorization header to your API call.  Fortunately Swashbuckle provides a relatively easy way to add new fields to the Swagger UI.

The following class will take care of adding the Authorization field to the Swagger UI:

public class AuthorizationHeaderParameterOperationFilter : IOperationFilter
{
	public void Apply(Operation operation, OperationFilterContext context)
	{
		var filterPipeline = context.ApiDescription.ActionDescriptor.FilterDescriptors;
		var isAuthorized = filterPipeline.Select(filterInfo => filterInfo.Filter).Any(filter => filter is AuthorizeFilter);
					var allowAnonymous = filterPipeline.Select(filterInfo => filterInfo.Filter).Any(filter => filter is IAllowAnonymousFilter);

		if (isAuthorized && !allowAnonymous)
		{
			if (operation.Parameters == null)
				operation.Parameters = new List<IParameter>();

			operation.Parameters.Add(new NonBodyParameter
			{                    
				Name = "Authorization",
				In = "header",
				Description = "access token",
				Required = false,
				Type = "string"
			});
		}
	}
}

With that in place you simply need to tell Swashbuckle about it in your Startup.cs:

public void ConfigureServices(IServiceCollection services)
{
    // Add framework services.
    services.AddApplicationInsightsTelemetry(Configuration);
    services.AddMvc();
    services.AddSwaggerGen();
    services.ConfigureSwaggerGen(options =>
	{
		options.SingleApiVersion(new Info
		{
			Version = "v1",
			Title = "Sample API",
			Description = "This is a sample API",
			Contact = new Contact
			{
				Name = "Mike",
				Email = "email@example.com"
			}
		});
		options.OperationFilter<AuthorizationHeaderParameterOperationFilter>();
		options.IncludeXmlComments(GetXmlCommentsPath());
		options.DescribeAllEnumsAsStrings();
	});
}

If you run your API project you should now see the Authorization field added to the "Try it out!" section of the Swagger UI for the selected end point.

That's all there is to it.  You now have a self documenting API that is both easy to explore and test using the Swagger UI.  To add even more value to the Swagger UI you should look into using the attributes and XML Documentation support that Swashbuckle offers.

Using WCF to Monitor Your Windows Services

Background...

Skip the story

Having come to age as a professional developer in an era where putting business logic in your database was considered sacrilege, I never used the database for anything more than storing data.  Using SQL Server was (is) a last resort.

A few months back I was working on a project at a company that has its roots firmly planted in a database oriented approach to development.  I get that it's impossible to rewrite a massive legacy system every time contemporary programming practices change.  However, I was surprised how quickly (and frequently) developers turned to the database or logging as a solution.

One such example, we needed a way to monitor and manage a Windows service.  Certainly logging provided a low level means of monitoring; however, it didn't provide an effective way to manage the Windows service.  One suggestion was to use a table in a database as a control mechanism.  That could work, but what about a more direct approach?

Setting up the solution

In this example we'll Create two console applications.  One will use TopShelf to start a Windows service (this post doesn't cover TopShelf).  The other will be a normal console application that'll communicate with the Windows service via WCF.  Generally I prefer to put a WPF application in the system tray; however, I'm keeping it simple for this example.

Create a blank Visual Studio 2015 solution named ServiceMonitorDemo.

The Windows service project

Add a new C# Console Application to your project named ServiceMonitorDemo.Service.  The first thing you'll need to do is to add TopShelf to the project:

Install-Package TopShelf

With that taken care of, you'll need to write two service contracts.  For this tutorial we're going to use a duplex channel.  You'll need one interface for each direction through the channel.

 First write the contract that other programs will use to communicate with the service:

using System.ServiceModel;

namespace ServiceMonitorDemo.Service.Contracts
{
    [ServiceContract(SessionMode = SessionMode.Required, CallbackContract = typeof(IDemoServiceCallbackChannel))]
    public interface IDemoServiceChannel
    {
        [OperationContract(IsOneWay = true)]
        void Connect();

        [OperationContract(IsOneWay = false)]
        bool DisplayMessage(string message);
    }
}

You'll notice this service contract is decorated with the ServiceContract attribute.  This is how you tell .NET what interface to use for the callback contract.  The callback contract is used by the service to communicate with connected clients.  You'll define the callback interface shortly.

Notice that the contract consists of two methods:

  • Connect - This method is used to add the calling client to our list of connected clients.
  • DisplayMessage - This is used as an example of bidirectional communication and to show how clients can control the service through WCF

The callback contract is pretty straight forward:

using System.ServiceModel;
using ServiceMonitorDemo.Model;

namespace ServiceMonitorDemo.Service.Contracts
{
    [ServiceContract]
    public interface IDemoServiceCallbackChannel
    {
        [OperationContract(IsOneWay = true)]
        void UpdateStatus(StatusUpdate status);

        [OperationContract(IsOneWay = true)]
        void ServiceShutdown();
    }
}

Again, our simple service contract has just two methods

  • UpdateStatus - Used by the service to push the service's status out to all connected clients.
  • ServiceShutdown - WCF does not cleanly handle shutting down things.  We need to make sure that the code takes care of opening and closing connection correctly.

With that out of the way we need to take care of writing the actual service.  For this example the service won't do anything exciting, it'll simply post a status object to all connected clients.  The code for this is pretty long, so I'm only going to post important sections here.  You can find the completed example solution on GitHub.

In order to accept connections to the service you'll need to initialize the named pipe:

_host = new ServiceHost(this);

NetNamedPipeBinding binding = new NetNamedPipeBinding();
binding.ReceiveTimeout = TimeSpan.MaxValue;

_host.AddServiceEndpoint(typeof(IDemoServiceChannel),
    binding,
    new Uri(Uri));

_host.Open();

This code simply creates a new host using the current object for the ServiceHost.  A named pipe binding is is added to the host.  Clients connect to this endpoint via the Connect method on the channel service contract defined above.  Our service implements the Connect method as follows:

public void Connect()
{
    AddCallbackChannel(OperationContext.Current.GetCallbackChannel<IDemoServiceCallbackChannel>());
}

The only other important detail here is that DemoService class is implemented as a singleton and decorated with the following attribute:

[ServiceBehavior(InstanceContextMode = InstanceContextMode.Single)]

In this case I have a single service running and I want to share data with all connected clients.  With this in mind, using InstanceContextMode.Single makes sense.  You can also use per session and per call context modes.  You can find a good overview of the differences on Code Project.

The rest of the cod for this project is pretty much boilerplate setup and tare down.

The Console Application

Fortunately this entire application is less than 100 lines long.  Again, I'll refer you to the GitHub repository to see the full implementation.  Below you'll find the most important section of the code:

        private void Connect()
        {
            while(!_isConnected)
            {
                try
                {
                    DuplexChannelFactory<IDemoServiceChannel> channelFactory = new DuplexChannelFactory<IDemoServiceChannel>(
                        new InstanceContext(this),
                        new NetNamedPipeBinding(),
                        new EndpointAddress(Uri));

                    _channel = channelFactory.CreateChannel();
                    _channel.Connect();

                    _isConnected = true;
                    Console.WriteLine("Channel connected.");
                }
                catch(Exception)
                {
                    Console.WriteLine("Failed to connect to channel.");
                }

                Thread.Sleep(1000);
            }
        }

I wouldn't recommend taking the above code and dumping into your production code.  It's designed to demonstrate how to establish a connection to the Windows service via a WCF named pipe.

As long as the console application is connected to the service, the service will continue to trigger the UpdateStatus method.  In a real world implementation UpdateStatus would most likely toggle some sort of visual status indicator on a WPF application (e.g. a red / green light).  In tutorial land displaying a message in the console works just fine.

Wrap Up

If you've cloned, forked, or downloaded a zip file of the repository you should be able to run the Windows service as a service by navigating to your binary folder for the project (Debug or Release depending on your active build configuration) in your preferred terminal / command prompt and running:

../ServiceMonitorDemo.Service install
../ServiceMonitorDemo.Service start

You can then run a few instances of the ServiceMonitorDemo.Monitor and see what happens.

GitHub repository Link

Getting Started with Visual Studio 2015, Antlr 4, C#, and a SQL Grammar

The Part You Can Skip...

Back when I had a lot of free time (high school / college) I enjoyed studying the art of compiler design.  My first first lexer and parser (LL(1)) were written from scratch.  By college I had discovered lex / yacc and flex / bison.  Unfortunately, I haven't made the time to dabble in this area in the decade or so that has lapsed since leaving school.

Fortunately I recently had a real world problem I was able to over complicate enough to warrant a domain specific language.  Of course I could have solved the problem with a switch statement, but switch statements are evil (I can't back that up with facts).  Instead I took the opportunity to play with Antlr.  Given the current state of the open source community and the abundance of Stack Overflow answers in search results I expected this to be much more straight forward than it was.

Although I'm perfectly content to develop in Java, my use case for the DSL is based in .NET.  Antlr 4 has a C# target, so I didn't expect this to be a problem.  It turns out that the documentation is either sparse or a bit out dated.  Below you'll find what I had to do to get things up and running using Antlr 4 and Visual Studio 2015...

Setting up Visual Studio 2015

You can find the outdated documentation for the C# Antlr 4 target on GitHub.  The documentation is pretty complete and still useful.  The biggest problem you'll run into is that the Antlr language support extension it references has not been updated for Visual Studio 2015.

Step 1: Create a new Class Library project

Generating the C# lexer and parser happens through the magic of NuGet packages.  All you need is a class library project, a valid grammar file (or two), and the appropriate NuGet packages.

Step 2: Add NuGet packages to your class library

Install-Package Antlr4
Install-Package Antlr4.Runtime

As of version 4.3.0, the NuGet package will not work with .NET 4.5.1 or newer.  I'm currently targeting 4.6.  It took me a moment or two to realize this problem.  As long as you're isolating your lexer and parser in it's own project (as you should be) this shouldn't be a problem.  Just edit your project to target the older version of the .NET framework.

Step 3: Add your grammar file(s)

If the Antlr extension for Visual Studio work with 2015 this would be trivial: just follow the documentation.  But... it doesn't (at least not at the time of writing this).

You must make sure the grammar file has the correct encoding.  If you don't, things won't work.  The GitHub documentation walks you through using the advanced save options in Visual Studio to set the proper encoding.  Although I found the advanced save options in Visual Studio 2015, I did not see Unicode (UTF-8 without signature) - Codepage 65001 listed as an option.  I ended up using NotePad++ and setting the encoding to UTF-8-BOM.

I was not able to edit the grammar file properties as outlined in the official documentation.  Instead I resorted to editing the csproj file.  You can do this two ways:

  1. Unload the project in Visual Studio, then open the project file in Visual Studio for editing.  Once your done, reload the project
  2. Use an external editor (like NotePad++, Atom, Sublime, VIM, whatever floats your boat) to edit the csproj file.  When you switch back to Visual Studio you'll be prompted to reload the project.

I had to add the following section to my csproj file

  <ItemGroup>
    <Compile Include="Properties\AssemblyInfo.cs" />
  </ItemGroup>
  <ItemGroup>
    <Antlr4 Include="SomeParser.g4">
      <Generator>MSBuild:Compile</Generator>
      <CustomToolNamespace>RootNamespace.Folder</CustomToolNamespace>
    </Antlr4>
  </ItemGroup>
  <ItemGroup>
    <None Include="packages.config" />
  </ItemGroup>

Make sure you remove the grammar file from any other section it may appear in.  If you have more than one g4 file (your lexer and parser are split) you will need to add an Antlr element for each file.

Finding a SQL Grammar

At this point you should be able to build your class library and generate an assembly that has your lexer, parser, base visitor, and base listener classes in it.  The next hurdle is finding an existing Antlr 4 SQL grammar.

After some digging, I found a SQL grammar file on GitHub (where else?).  Whether or not you can legally use the grammar I've linked you to is something you need to decide.  According to a Google Group discussion I found, Salesforce.com may have released their SQL grammar.  I have not looked for that.  Let me know if you find it.  You can also check the GitHub Antlr 4 grammar repository for grammar files.  As of writing there wasn't a SQL grammar file there.

Random Other Links

Here are few other random links that helped me in some way:

Helpful Beginner Hints

Make sure you order your predicates with the most specific at the top.  If you don't you'll end up in a situation where your intended predicate can't be reached because a more generic one is reached first (i.e. defining a string literal that can have spaces before you define an identifier token that can't have spaces).

Take a look at ANTLRWorks 2.  It's incredibly useful when writing your grammar files.

I strongly encourage you to read a book or two.  Blogs (like this one) are great at getting you started; however, books do a better job providing you with the full picture.

Let me know if you know something I don't.  I'd love to update this article with more useful and / or up to date information.