ASP.NET Core – adding controllers directly from integration tests

2020-06-222020-06-21 TPodolakASP.NET Core, testing

1. Introduction

From time to time it might happen, that you need to test certain parts of your ASP.NET Core configuration without hitting publicly visible business-related controllers. For instance, in my case, I wanted to make sure that my ASP.NET Core API behavior is consistent with the original API written in Nancy. As you might expect there are quite a lot of differences between both of the frameworks, so let’s focus on testing one thing, namely non-nullable reference type handling. Long story short, in order to have the same behavior in ASP.NET Core as in Nancy I had to add the following line of configuration

// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
    services.AddControllers(opts => opts.SuppressImplicitRequiredAttributeForNonNullableReferenceTypes = true)
        .AddNewtonsoftJson();
}

// This method gets called by the runtime. Use this method to add services to the container.

public void ConfigureServices(IServiceCollection services)

{

services.AddControllers(opts => opts.SuppressImplicitRequiredAttributeForNonNullableReferenceTypes = true)

.AddNewtonsoftJson();

}

This one prevents ASP.NET Core from marking non-nullable reference type properties in request as required. Having that configuration ready, I wanted to test it with an end to end test. Because there was no business-related logic yet, I needed to figure out the way of adding API controllers to my application directly from the integration tests. Here is how you can achieve that.

2. Accessing ApplicationPartManager from integration tests

ASP.NET Core is able to compose your API from different parts thanks to ApplicationPartManager. By default, you don’t use it directly but rather with IMvcBuilder AddApplicationPart extension method while setting up your application

// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
    services.AddControllers()
        .AddApplicationPart(typeof(SomeClass).Assembly)
        .AddNewtonsoftJson();
}

// This method gets called by the runtime. Use this method to add services to the container.

public void ConfigureServices(IServiceCollection services)

{

services.AddControllers()

.AddApplicationPart(typeof(SomeClass).Assembly)

.AddNewtonsoftJson();

}

However, there is no easy way of getting IMvcBuidler from the integration tests(at least I didn’t find a proper way of doing that without tampering too much with the original application pipeline) so we need to access ApplicationPartManager directly. Inspecting the source code of the framework you will find that ApplicationPartManager can be retrieved directly from IServiceCollection with following code

private static ApplicationPartManager GetApplicationPartManager(IServiceCollection services)
{
    var manager = GetServiceFromCollection<ApplicationPartManager>(services);
    if (manager == null)
    {
        manager = new ApplicationPartManager();

        var environment = GetServiceFromCollection<IWebHostEnvironment>(services);
        var entryAssemblyName = environment?.ApplicationName;
        if (string.IsNullOrEmpty(entryAssemblyName))
        {
            return manager;
        }

        manager.PopulateDefaultParts(entryAssemblyName);
    }

    return manager;
}

private static T GetServiceFromCollection<T>(IServiceCollection services)
{
    return (T)services
        .LastOrDefault(d => d.ServiceType == typeof(T))
        ?.ImplementationInstance;
}

private static ApplicationPartManager GetApplicationPartManager(IServiceCollection services)

{

var manager = GetServiceFromCollection<ApplicationPartManager>(services);

if (manager == null)

{

manager = new ApplicationPartManager();

var environment = GetServiceFromCollection<IWebHostEnvironment>(services);

var entryAssemblyName = environment?.ApplicationName;

if (string.IsNullOrEmpty(entryAssemblyName))

{

return manager;

}

manager.PopulateDefaultParts(entryAssemblyName);

}

return manager;

}

private static T GetServiceFromCollection<T>(IServiceCollection services)

{

return (T)services

.LastOrDefault(d => d.ServiceType == typeof(T))

?.ImplementationInstance;

}

Applying similar code to the WebApplicationFactory will allow us to add controllers directly from tests

[Fact]
public async Task DoesNotReturnBadRequest_WhenNonNullableTypeHasNoValue_FirstAttempt()
{
    var webApplicationFactory = new WebApplicationFactory<Startup>()
        .WithWebHostBuilder(
            builder => builder.ConfigureServices(services =>
            {
                var partManager = (ApplicationPartManager) services
                    .Last(descriptor => descriptor.ServiceType == typeof(ApplicationPartManager))
                    .ImplementationInstance;

                partManager.ApplicationParts.Add(new AssemblyPart(GetType().Assembly));
            }));

    var expectedResponse = new Request
    {
        Id = "1"
    };

    var httpClient = webApplicationFactory.CreateClient();
    var httpResponseMessage = await httpClient.GetAsync("test?id=1");

    httpResponseMessage.IsSuccessStatusCode.Should().BeTrue();
    var rawContent = await httpResponseMessage.Content.ReadAsStringAsync();
    var result = JsonConvert.DeserializeObject<Request>(rawContent);

    result.Should().BeEquivalentTo(expectedResponse);
}

[ApiController]
public class MvcOptionsTestController : ControllerBase
{
    [HttpGet("test")]
    public object Get([FromQuery] Request request)
    {
        return request;
    }
}

public class Request
{
    public string Id { get; set; }

    public string OtherId { get; set; }
}

[Fact]

public async Task DoesNotReturnBadRequest_WhenNonNullableTypeHasNoValue_FirstAttempt()

{

var webApplicationFactory = new WebApplicationFactory<Startup>()

.WithWebHostBuilder(

builder => builder.ConfigureServices(services =>

{

var partManager = (ApplicationPartManager) services

.Last(descriptor => descriptor.ServiceType == typeof(ApplicationPartManager))

.ImplementationInstance;

partManager.ApplicationParts.Add(new AssemblyPart(GetType().Assembly));

}));

var expectedResponse = new Request

{

Id = "1"

};

var httpClient = webApplicationFactory.CreateClient();

var httpResponseMessage = await httpClient.GetAsync("test?id=1");

httpResponseMessage.IsSuccessStatusCode.Should().BeTrue();

var rawContent = await httpResponseMessage.Content.ReadAsStringAsync();

var result = JsonConvert.DeserializeObject<Request>(rawContent);

result.Should().BeEquivalentTo(expectedResponse);

}

[ApiController]

public class MvcOptionsTestController : ControllerBase

{

[HttpGet("test")]

public object Get([FromQuery] Request request)

{

return request;

}

public class Request

{

public string Id { get; set; }

public string OtherId { get; set; }

}

3. Handling private controllers

The solution presented below works nice, however it has couple of drawbacks. First of all, it only discovers public non-nested controllers. Second of all, it will always add all controllers from given assembly – which potentially might affect other integration tests. In order to get rid of these drawbacks, we need to tell ApplicationPartManager to include only selected controllers. We can achieve that by adding additional IApplicationFeatureProvider<ControllerFeature> to the list of FeatureProviders.

public class ExternalControllersFeatureProvider : IApplicationFeatureProvider<ControllerFeature>
{
    private readonly Type[] _controllers;

    public ExternalControllersFeatureProvider(params Type[] controllers)
    {
        _controllers = controllers ?? Array.Empty<Type>();
    }

    public void PopulateFeature(IEnumerable<ApplicationPart> parts, ControllerFeature feature)
    {
        foreach (var controller in _controllers)
        {
            feature.Controllers.Add(controller.GetTypeInfo());
        }
    }
}

[Fact]
public async Task DoesNotReturnBadRequest_WhenNonNullableTypeHasNoValue()
{
    var webApplicationFactory = new WebApplicationFactory<Startup>()
        .WithWebHostBuilder(
            builder => builder.ConfigureServices(services =>
            {
                var partManager = (ApplicationPartManager) services
                    .Last(descriptor => descriptor.ServiceType == typeof(ApplicationPartManager))
                    .ImplementationInstance;

                partManager.FeatureProviders.Add(new ExternalControllersFeatureProvider(typeof(MvcOptionsTestController)));
            }));
// rest of the code omitted for brevity
}

public class ExternalControllersFeatureProvider : IApplicationFeatureProvider<ControllerFeature>

{

private readonly Type[] _controllers;

public ExternalControllersFeatureProvider(params Type[] controllers)

{

_controllers = controllers ?? Array.Empty<Type>();

}

public void PopulateFeature(IEnumerable<ApplicationPart> parts, ControllerFeature feature)

{

foreach (var controller in _controllers)

{

feature.Controllers.Add(controller.GetTypeInfo());

}

[Fact]

public async Task DoesNotReturnBadRequest_WhenNonNullableTypeHasNoValue()

{

var webApplicationFactory = new WebApplicationFactory<Startup>()

.WithWebHostBuilder(

builder => builder.ConfigureServices(services =>

{

var partManager = (ApplicationPartManager) services

.Last(descriptor => descriptor.ServiceType == typeof(ApplicationPartManager))

.ImplementationInstance;

partManager.FeatureProviders.Add(new ExternalControllersFeatureProvider(typeof(MvcOptionsTestController)));

}));

// rest of the code omitted for brevity

}

Putting it all together and applying some refactoring we will end up with following code

internal static class WebHostBuilderExtensions
{
    public static IWebHostBuilder WithAdditionalControllers(this IWebHostBuilder builder, params Type[] controllers)
    {
        return builder.ConfigureTestServices(
            services =>
            {
                var partManager = GetApplicationPartManager(services);

                partManager.FeatureProviders.Add(new ExternalControllersFeatureProvider(controllers));
            });
    }

    private static ApplicationPartManager GetApplicationPartManager(IServiceCollection services)
    {
        var partManager = (ApplicationPartManager)services
            .Last(descriptor => descriptor.ServiceType == typeof(ApplicationPartManager))
            .ImplementationInstance;
        return partManager;
    }

    private class ExternalControllersFeatureProvider : IApplicationFeatureProvider<ControllerFeature>
    {
        private readonly Type[] _controllers;

        public ExternalControllersFeatureProvider(params Type[] controllers)
        {
            _controllers = controllers ?? Array.Empty<Type>();
        }

        public void PopulateFeature(IEnumerable<ApplicationPart> parts, ControllerFeature feature)
        {
            foreach (var controller in _controllers)
            {
                feature.Controllers.Add(controller.GetTypeInfo());
            }
        }
    }
}

public class MvcOptionsConfigurationTests
{
    [Fact]
    public async Task DoesNotReturnBadRequest_WhenNonNullableTypeHasNoValue()
    {
        var webApplicationFactory = new WebApplicationFactory<Startup>()
            .WithWebHostBuilder(
                builder => builder.WithAdditionalControllers(typeof(MvcOptionsTestController)));

        var httpClient = webApplicationFactory.CreateClient();
        var expectedResponse = new Request
        {
            Id = "1"
        };

        var httpResponseMessage = await httpClient.GetAsync("test?id=1");

        httpResponseMessage.IsSuccessStatusCode.Should().BeTrue();
        var rawContent = await httpResponseMessage.Content.ReadAsStringAsync();
        var result = JsonConvert.DeserializeObject<Request>(rawContent);

        result.Should().BeEquivalentTo(expectedResponse);
    }

    [ApiController]
    private class MvcOptionsTestController: ControllerBase
    {
        [HttpGet("test")]
        public object Get([FromQuery] Request request)
        {
            return request;
        }
    }

    private class Request
    {
        public string Id { get; set; }

        public string OtherId { get; set; }
    }
}

internal static class WebHostBuilderExtensions

{

public static IWebHostBuilder WithAdditionalControllers(this IWebHostBuilder builder, params Type[] controllers)

{

return builder.ConfigureTestServices(

services =>

{

var partManager = GetApplicationPartManager(services);

partManager.FeatureProviders.Add(new ExternalControllersFeatureProvider(controllers));

});

}

private static ApplicationPartManager GetApplicationPartManager(IServiceCollection services)

{

var partManager = (ApplicationPartManager)services

.Last(descriptor => descriptor.ServiceType == typeof(ApplicationPartManager))

.ImplementationInstance;

return partManager;

}

private class ExternalControllersFeatureProvider : IApplicationFeatureProvider<ControllerFeature>

{

private readonly Type[] _controllers;

public ExternalControllersFeatureProvider(params Type[] controllers)

{

_controllers = controllers ?? Array.Empty<Type>();

}

public void PopulateFeature(IEnumerable<ApplicationPart> parts, ControllerFeature feature)

{

foreach (var controller in _controllers)

{

feature.Controllers.Add(controller.GetTypeInfo());

}

public class MvcOptionsConfigurationTests

{

[Fact]

public async Task DoesNotReturnBadRequest_WhenNonNullableTypeHasNoValue()

{

var webApplicationFactory = new WebApplicationFactory<Startup>()

.WithWebHostBuilder(

builder => builder.WithAdditionalControllers(typeof(MvcOptionsTestController)));

var httpClient = webApplicationFactory.CreateClient();

var expectedResponse = new Request

{

Id = "1"

};

var httpResponseMessage = await httpClient.GetAsync("test?id=1");

httpResponseMessage.IsSuccessStatusCode.Should().BeTrue();

var rawContent = await httpResponseMessage.Content.ReadAsStringAsync();

var result = JsonConvert.DeserializeObject<Request>(rawContent);

result.Should().BeEquivalentTo(expectedResponse);

}

[ApiController]

private class MvcOptionsTestController: ControllerBase

{

[HttpGet("test")]

public object Get([FromQuery] Request request)

{

return request;

}

private class Request

{

public string Id { get; set; }

public string OtherId { get; set; }

}

Source code for this post can be found here

Running ASP.NET Core together with Nancy

2020-04-202020-04-19 TPodolakASP.NET Core, Nancy

1. Introduction

Our current API runs on Nancy which in my opinion is past its prime. Recent news from the GitHub issue tracker seems to confirm that thesis, that is why we started looking for a migration path from Nancy based API to ASP.NET Core. Because the codebase is quite large we didn’t want to do a Big Bang Rewrite but instead of that, we wanted to gradually replace old API with a new one, so both of them can coexist next to each other.

2. Legacy API

Before I jump into implementation, here is a sample Nancy API with two endpoints – products and variants

public class Startup
{
    // This method gets called by the runtime. Use this method to add services to the container.
    public void ConfigureServices(IServiceCollection services)
    {
        // nancy has its own container
    }

    // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
    public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
    {
        app.UseOwin(owin => owin.UseNancy());
    }
}

public class Startup

{

// This method gets called by the runtime. Use this method to add services to the container.

public void ConfigureServices(IServiceCollection services)

{

// nancy has its own container

}

// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)

{

app.UseOwin(owin => owin.UseNancy());

}

public sealed class ProductsNancyModule : NancyModule
{
    public ProductsNancyModule()
    {
        Get("products", args => Response.AsJson(new List<Product>
        {
            new Product(Guid.NewGuid().ToString(), "First product from nancy"),
            new Product(Guid.NewGuid().ToString(), "Second product from nancy")
        }));
    }
}

public sealed class ProductsNancyModule : NancyModule

{

public ProductsNancyModule()

{

Get("products", args => Response.AsJson(new List<Product>

{

new Product(Guid.NewGuid().ToString(), "First product from nancy"),

new Product(Guid.NewGuid().ToString(), "Second product from nancy")

}));

}

public sealed class VariantsNancyModule : NancyModule
{
    public VariantsNancyModule()
    {
        Get("variants", args => Response.AsJson(new List<Variant>
        {
            new Variant(Guid.NewGuid().ToString(), Guid.NewGuid().ToString(), "first variant from nancy"),
            new Variant(Guid.NewGuid().ToString(), Guid.NewGuid().ToString(), "second variant from nancy")
        }));
    }
}

public sealed class VariantsNancyModule : NancyModule

{

public VariantsNancyModule()

{

Get("variants", args => Response.AsJson(new List<Variant>

{

new Variant(Guid.NewGuid().ToString(), Guid.NewGuid().ToString(), "first variant from nancy"),

new Variant(Guid.NewGuid().ToString(), Guid.NewGuid().ToString(), "second variant from nancy")

}));

}

The goal is to replace products endpoint with ASP.NET Core implementation while variants endpoint should still be served by Nancy

3. Combining Nancy with ASP.NET Core

The solution is based on branching the pipeline feature which is available starting from ASP.NET Core 2.1. Long story short – it is possible to configure different pipelines for different route paths thanks to

public static IApplicationBuilder Map(this IApplicationBuilder app, PathString pathMatch, Action<IApplicationBuilder> configuration)

1	public static IApplicationBuilder Map(this IApplicationBuilder app, PathString pathMatch, Action<IApplicationBuilder> configuration)

method. Having that in mind we can Map products path to run through ASP.NET Core pipeline whereas the rest would go through Nancy.

// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
    // register mvc services
    services.AddControllers();
    services.AddMvc(options => options.EnableEndpointRouting = false);
}

// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
    // run everything with products prefix through mvc
    app.Map("/products", appBuilder =>
    {
        appBuilder.UseMvc();
    });

    // run the rest through Nancy
    app.Map(string.Empty, appBuilder =>
    {
        appBuilder.UseOwin(options => options.UseNancy());
    });
}

// This method gets called by the runtime. Use this method to add services to the container.

public void ConfigureServices(IServiceCollection services)

{

// register mvc services

services.AddControllers();

services.AddMvc(options => options.EnableEndpointRouting = false);

}

// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)

{

// run everything with products prefix through mvc

app.Map("/products", appBuilder =>

{

appBuilder.UseMvc();

});

// run the rest through Nancy

app.Map(string.Empty, appBuilder =>

{

appBuilder.UseOwin(options => options.UseNancy());

});

}

[ApiController]
public class ProductsController : ControllerBase
{
    [HttpGet("/products")]
    public List<Product> Get()
    {
        return new List<Product>
        {
            new Product(Guid.NewGuid().ToString(), "First product from asp net core"),
            new Product(Guid.NewGuid().ToString(), "Second product from asp net core")
        };
    }
}

[ApiController]

public class ProductsController : ControllerBase

{

[HttpGet("/products")]

public List<Product> Get()

{

return new List<Product>

{

new Product(Guid.NewGuid().ToString(), "First product from asp net core"),

new Product(Guid.NewGuid().ToString(), "Second product from asp net core")

};

}

At this point we are almost there, however running request through Map pipeline will remove the path prefix, meaning that our controller would be accessible under / path instead of products. In order to bypass this limitation we have to restore original prefix with RewriteMiddleware. Once we put it all together, now we are able to replace multiple Nancy endpoints with ASP.NET Core ones using following piece of code

// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
    var aspNetCorePaths = new PathString[] {"/products"};
    foreach (var path in aspNetCorePaths)
    {
        app.Map(path, appBuilder =>
        {
            // we still want to access the resource under old url, so we need to add the branch path back to the route
            var rewriteOptions = new RewriteOptions().AddRewrite(
                "(?<Path>.*)",
                $"{path}/${{Path}}",
                skipRemainingRules: true);
            appBuilder.UseRewriter(rewriteOptions);
            appBuilder.UseMvc();
        });
    }

    app.Map(string.Empty, appBuilder =>
    {
        appBuilder.UseOwin(options => options.UseNancy());
    });
}

// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)

{

var aspNetCorePaths = new PathString[] {"/products"};

foreach (var path in aspNetCorePaths)

{

app.Map(path, appBuilder =>

{

// we still want to access the resource under old url, so we need to add the branch path back to the route

var rewriteOptions = new RewriteOptions().AddRewrite(

"(?<Path>.*)",

$"{path}/${{Path}}",

skipRemainingRules: true);

appBuilder.UseRewriter(rewriteOptions);

appBuilder.UseMvc();

});

}

app.Map(string.Empty, appBuilder =>

{

appBuilder.UseOwin(options => options.UseNancy());

});

}

Source code for this post can be found here

ASP.NET Core – custom ILogger<T> and Nlog callsite layout renderer issue

2018-03-012018-02-28 TPodolakASP.NET Core, Logging2 Comments

1. Introduction

In my current project, we use Microsoft.Extensions.Logging.ILogger<T> default implementation as our logging mechanism. This basically comes with the ASP.NET Core 2.0 package and it was working fine so far. However recently we decided to switch the default implementation with a custom one, due to the fact that we wanted to have some additional logic applied for all of the logging providers.

2. Implementation

The custom implementation itself is very similar to the original one, we came up with something like that

public class Logger<T> : ILogger<T>
{
    private readonly ILogger _logger;

    public Logger(ILoggerFactory loggerFactory)
    {
        _logger = loggerFactory.CreateLogger<T>();
    }

    public void Log<TState>(LogLevel, EventId, TState state, Exception, Func<TState, Exception, string> formatter)
    {
        string Formatter(TState innserState, Exception innerException)
        {
            // additional logic goes here, in my case that was extracting additional information from custom exceptions
            var message = formatter(innserState, innerException) ?? string.Empty;
            return message + " additional stuff in here";
        }

        _logger.Log(logLevel, eventId, state, exception, Formatter);
    }

    public bool IsEnabled(LogLevel logLevel)
    {
        return _logger.IsEnabled(logLevel);
    }

    public IDisposable BeginScope<TState>(TState state)
    {
        return _logger.BeginScope(state);
    }
}

public class Logger<T> : ILogger<T>

{

private readonly ILogger _logger;

public Logger(ILoggerFactory loggerFactory)

{

_logger = loggerFactory.CreateLogger<T>();

}

public void Log<TState>(LogLevel, EventId, TState state, Exception, Func<TState, Exception, string> formatter)

{

string Formatter(TState innserState, Exception innerException)

{

// additional logic goes here, in my case that was extracting additional information from custom exceptions

var message = formatter(innserState, innerException) ?? string.Empty;

return message + " additional stuff in here";

}

_logger.Log(logLevel, eventId, state, exception, Formatter);

}

public bool IsEnabled(LogLevel logLevel)

{

return _logger.IsEnabled(logLevel);

}

public IDisposable BeginScope<TState>(TState state)

{

return _logger.BeginScope(state);

}

and we replaced the default logger with this code

services.Replace(new ServiceDescriptor(typeof(ILogger<>), typeof(Logger.Logger<>), ServiceLifetime.Singleton));

1	services.Replace(new ServiceDescriptor(typeof(ILogger<>), typeof(Logger.Logger<>), ServiceLifetime.Singleton));

As you can see we basically wanted to add information from custom exceptions to the message. Due to the fact that this is added in here, we don’t have to modify all of our existing logging providers.

3. NLog call site renderer issue

Once we run the application all of the providers were working fine and were producing expected output apart from NLog provider. Originally NLog provider for the following code

[Route("api/[controller]")]
public class ValuesController : Controller
{
    private readonly ILogger<ValuesController> _logger;

    public ValuesController(ILogger<ValuesController> logger)
    {
        _logger = logger;
    }

    [HttpGet]
    public IEnumerable<string> Get()
    {
        _logger.LogTrace("Get method called trace");
        _logger.LogWarning("Get method called warning");
        _logger.LogError("Get method called error");
        _logger.LogInformation("Get method called information");

        return new string[] {"value1", "value2"};
    }
}

[Route("api/[controller]")]

public class ValuesController : Controller

{

private readonly ILogger<ValuesController> _logger;

public ValuesController(ILogger<ValuesController> logger)

{

_logger = logger;

}

[HttpGet]

public IEnumerable<string> Get()

{

_logger.LogTrace("Get method called trace");

_logger.LogWarning("Get method called warning");

_logger.LogError("Get method called error");

_logger.LogInformation("Get method called information");

return new string[] {"value1", "value2"};

}

was producing this output

TimeStamp=2018-02-28 20:14:37.2711 Callsite=AspNetCoreCustomLoggerAndNLogCallsiteIssue.Controllers.ValuesController.Get Level=TRACE Message=Get method called trace 
TimeStamp=2018-02-28 20:14:37.3261 Callsite=AspNetCoreCustomLoggerAndNLogCallsiteIssue.Controllers.ValuesController.Get Level=WARN Message=Get method called warning 
TimeStamp=2018-02-28 20:14:37.3261 Callsite=AspNetCoreCustomLoggerAndNLogCallsiteIssue.Controllers.ValuesController.Get Level=ERROR Message=Get method called error 
TimeStamp=2018-02-28 20:14:37.3261 Callsite=AspNetCoreCustomLoggerAndNLogCallsiteIssue.Controllers.ValuesController.Get Level=INFO Message=Get method called information

TimeStamp=2018-02-28 20:14:37.2711 Callsite=AspNetCoreCustomLoggerAndNLogCallsiteIssue.Controllers.ValuesController.Get Level=TRACE Message=Get method called trace

TimeStamp=2018-02-28 20:14:37.3261 Callsite=AspNetCoreCustomLoggerAndNLogCallsiteIssue.Controllers.ValuesController.Get Level=WARN Message=Get method called warning

TimeStamp=2018-02-28 20:14:37.3261 Callsite=AspNetCoreCustomLoggerAndNLogCallsiteIssue.Controllers.ValuesController.Get Level=ERROR Message=Get method called error

TimeStamp=2018-02-28 20:14:37.3261 Callsite=AspNetCoreCustomLoggerAndNLogCallsiteIssue.Controllers.ValuesController.Get Level=INFO Message=Get method called information

However, after introducing new implementation of ILogger<T>, it started appending wrong caller member name

TimeStamp=2018-02-28 20:17:27.4044 Callsite=AspNetCoreCustomLoggerAndNLogCallsiteIssue.Logger`1.Log Level=TRACE Message=Get method called trace additional stuff in here 
TimeStamp=2018-02-28 20:17:27.5117 Callsite=AspNetCoreCustomLoggerAndNLogCallsiteIssue.Logger`1.Log Level=WARN Message=Get method called warning additional stuff in here 
TimeStamp=2018-02-28 20:17:27.5117 Callsite=AspNetCoreCustomLoggerAndNLogCallsiteIssue.Logger`1.Log Level=ERROR Message=Get method called error additional stuff in here 
TimeStamp=2018-02-28 20:17:27.5202 Callsite=AspNetCoreCustomLoggerAndNLogCallsiteIssue.Logger`1.Log Level=INFO Message=Get method called information additional stuff in here

TimeStamp=2018-02-28 20:17:27.4044 Callsite=AspNetCoreCustomLoggerAndNLogCallsiteIssue.Logger`1.Log Level=TRACE Message=Get method called trace additional stuff in here

TimeStamp=2018-02-28 20:17:27.5117 Callsite=AspNetCoreCustomLoggerAndNLogCallsiteIssue.Logger`1.Log Level=WARN Message=Get method called warning additional stuff in here

TimeStamp=2018-02-28 20:17:27.5117 Callsite=AspNetCoreCustomLoggerAndNLogCallsiteIssue.Logger`1.Log Level=ERROR Message=Get method called error additional stuff in here

TimeStamp=2018-02-28 20:17:27.5202 Callsite=AspNetCoreCustomLoggerAndNLogCallsiteIssue.Logger`1.Log Level=INFO Message=Get method called information additional stuff in here

After some digging, it turned out that the issue lies in the (indirect)configuration of NLog’s callsite layout renderer. Long story short, when you add NLog provider to your logger providers, under the hood it adds Microsoft.Extensions.Logging.Abstractions assembly(among others) to something called hidden assembly collection. Once the NLog tries to figure out what is the caller member name, it crawls the stack trace to find a method name. If the method in current frame comes from a hidden assembly, logger skips it and continues iteration. As we changed the Logger<T> to our custom one, it is no longer part of Microsoft.Extensions.Logging.Abstractions assembly, so the NLog reports Log method from our custom logger as a caller member name.

4. The fix

In order to restore previous behavior you have to add custom logger assembly to the hidden assembly collection. You can do it via

LogManager.AddHiddenAssembly(typeof(Logger.Logger<>).Assembly);

1	LogManager.AddHiddenAssembly(typeof(Logger.Logger<>).Assembly);

Of course, if the logger is located in the same assembly as rest of your code, this might not be the perfect solution – as this will impact the way the callsite is rendered for other parts of your code in this assembly. In that case, you might consider moving the logging piece to separate assembly or resign from using callsite layout renderer at all. We decided to replace it with logger layout renderer as callsite layout renderer, either way, wouldn’t be used in production environments.

Source code for this post can be found here

ASP.NET Core – make sure IOptions<T> is initialized

2018-01-292018-02-28 TPodolakASP.NET Core1 Comment

1. Introduction

As most people know IOptions<T> is a convenient way of handling configuration options in your application. Even though I’ve been using it for quite some time, last week I was unpleasantly surprised by a production bug caused by wrong usage of this mechanism.

2. Problem

Let’s say we have a simple controller which depends on two instances of IOptions<T>

[Route("api/[controller]")]
public class ValuesController : Controller
{
    private readonly IOptions<PaymentOptions> _paymentOptions;
    private readonly IOptions<NotificationOptions> _notificationOptions;

    public ValuesController(IOptions<PaymentOptions> paymentOptions, IOptions<NotificationOptions> notificationOptions)
    {
        _paymentOptions = paymentOptions ?? throw new ArgumentNullException(nameof(paymentOptions));
        _notificationOptions = notificationOptions ?? throw new ArgumentNullException(nameof(notificationOptions));
    }

    // GET api/values
    [HttpGet]
    public IEnumerable<string> Get()
    {
        var notificationOptions = _notificationOptions.Value;
        var paymentOptions = _paymentOptions.Value;
            
        return new string[] {"value1", "value2"};
    }
}

[Route("api/[controller]")]

public class ValuesController : Controller

{

private readonly IOptions<PaymentOptions> _paymentOptions;

private readonly IOptions<NotificationOptions> _notificationOptions;

public ValuesController(IOptions<PaymentOptions> paymentOptions, IOptions<NotificationOptions> notificationOptions)

{

_paymentOptions = paymentOptions ?? throw new ArgumentNullException(nameof(paymentOptions));

_notificationOptions = notificationOptions ?? throw new ArgumentNullException(nameof(notificationOptions));

}

// GET api/values

[HttpGet]

public IEnumerable<string> Get()

{

var notificationOptions = _notificationOptions.Value;

var paymentOptions = _paymentOptions.Value;

return new string[] {"value1", "value2"};

}

Then we add only IOptions in Startup class – IOptions<PaymentOptoins>

public class Startup
{
    // Rest of the code omitted for brevity
    // This method gets called by the runtime. Use this method to add services to the container.
    public void ConfigureServices(IServiceCollection services)
    {
        services.AddOptions();
        services.AddMvc();
        services.Configure<PaymentOptions>(Configuration.GetSection("PaymentOptions"));
    }
      // Rest of the code omitted for brevity
}

public class Startup

{

// Rest of the code omitted for brevity

// This method gets called by the runtime. Use this method to add services to the container.

public void ConfigureServices(IServiceCollection services)

{

services.AddOptions();

services.AddMvc();

services.Configure<PaymentOptions>(Configuration.GetSection("PaymentOptions"));

}

// Rest of the code omitted for brevity

}

Now, when we run the app and hit one of the endpoints in ValuesController I was expecting to get an exception, as I didn’t register IOptions<NotificationOptions>. Surprisingly for me, the application was running normally and IOptions<NotificationOptions> was created with the new instance of NotificationOptions class(with all the properties defaulted).

In order to find out why it behaves like that we have to take a look how services responsible for handling IOptions<T> are registered in the container.

public static IServiceCollection AddOptions(this IServiceCollection services)
{
    if (services == null)
        throw new ArgumentNullException(nameof (services));

    services.TryAdd(ServiceDescriptor.Singleton(typeof (IOptions<>), typeof (OptionsManager<>)));
    services.TryAdd(ServiceDescriptor.Scoped(typeof (IOptionsSnapshot<>), typeof (OptionsManager<>)));
    services.TryAdd(ServiceDescriptor.Singleton(typeof (IOptionsMonitor<>), typeof (OptionsMonitor<>)));
    services.TryAdd(ServiceDescriptor.Transient(typeof (IOptionsFactory<>), typeof (OptionsFactory<>)));
    services.TryAdd(ServiceDescriptor.Singleton(typeof (IOptionsMonitorCache<>), typeof (OptionsCache<>)));
    return services;
}

public static IServiceCollection AddOptions(this IServiceCollection services)

{

if (services == null)

throw new ArgumentNullException(nameof (services));

services.TryAdd(ServiceDescriptor.Singleton(typeof (IOptions<>), typeof (OptionsManager<>)));

services.TryAdd(ServiceDescriptor.Scoped(typeof (IOptionsSnapshot<>), typeof (OptionsManager<>)));

services.TryAdd(ServiceDescriptor.Singleton(typeof (IOptionsMonitor<>), typeof (OptionsMonitor<>)));

services.TryAdd(ServiceDescriptor.Transient(typeof (IOptionsFactory<>), typeof (OptionsFactory<>)));

services.TryAdd(ServiceDescriptor.Singleton(typeof (IOptionsMonitorCache<>), typeof (OptionsCache<>)));

return services;

}

As you can see AddOptions method registers open generics for (among others) IOptions<> which means that you don’t have to register specific types of IOptions<T>, however, you are responsible for its configuration with

services.Configure<T>(Configuration.GetSection("Section"));

1	services.Configure<T>(Configuration.GetSection("Section"));

I was always convinced that services.Configure registers and configure IOptions<T> but apparently, I was wrong.

3. Ensuring initialization

As these kind of errors are rather difficult to catch(at least in case of our application) I wanted to minimize the risk of doing the same bug again. The idea was to make sure that at least one IConfigureOptions<T> or IPostConfigureOptions<T> service is registered in the container (as these are the services which are responsible for setting the values of T class). My first approach was to replace default OptionsFactory<T> with a custom one

public class ValidatableOptionsFactory<TOptions> : OptionsFactory<TOptions> where TOptions : class, new()
{
    private static readonly string NamespacePrefix = typeof(Program).Namespace.Split('.').First();

    public ValidatableOptionsFactory(IEnumerable<IConfigureOptions<TOptions>> setups, IEnumerable<IPostConfigureOptions<TOptions>> postConfigures)
            : base(setups, postConfigures)
    {
        if (typeof(TOptions).Namespace.StartsWith(NamespacePrefix) && setups.Any() == false && postConfigures.Any() == false)
        {
            throw new InvalidOperationException($"No configuration options or post configuration options was found to configure {typeof(TOptions)}");
        }
    }
}

public class Startup
{      
    // This method gets called by the runtime. Use this method to add services to the container.
    public void ConfigureServices(IServiceCollection services)
    {
        services.AddTransient(typeof(IOptionsFactory<>), typeof(ValidatableOptionsFactory<>));
        services.AddOptions();
    }
}

public class ValidatableOptionsFactory<TOptions> : OptionsFactory<TOptions> where TOptions : class, new()

{

private static readonly string NamespacePrefix = typeof(Program).Namespace.Split('.').First();

public ValidatableOptionsFactory(IEnumerable<IConfigureOptions<TOptions>> setups, IEnumerable<IPostConfigureOptions<TOptions>> postConfigures)

: base(setups, postConfigures)

{

if (typeof(TOptions).Namespace.StartsWith(NamespacePrefix) && setups.Any() == false && postConfigures.Any() == false)

{

throw new InvalidOperationException($"No configuration options or post configuration options was found to configure {typeof(TOptions)}");

}

public class Startup

{

// This method gets called by the runtime. Use this method to add services to the container.

public void ConfigureServices(IServiceCollection services)

{

services.AddTransient(typeof(IOptionsFactory<>), typeof(ValidatableOptionsFactory<>));

services.AddOptions();

}

This method works fine (although probably needs smarter TOptions filter)

however I got to conclusion that there is no point to do validation in a runtime, and the integration test might be a better choice. I came up with following code (I am assuming that all configuration options are stored in the same assembly, as this is convention in my project)

public class OptionsTests
{
    private readonly Lazy<IWebHost> _webHost = new Lazy<IWebHost>(Program.BuildWebHost(null));

    // this test will fail as not every option in assembly is configured
    [Fact]
    public void AllConfigurationOptions_HasConfigurationServicesDefined()
    {
        var optionsType = typeof(IOptions<>);
        var constructorParameterTypes = typeof(Program).Assembly.GetExportedTypes()
                .SelectMany(type => type.GetConstructors().SelectMany(ctor => ctor.GetParameters()))
                .Select(parameter => parameter.ParameterType);

        var optionValueTypes = constructorParameterTypes.Where(parameterType => IsAssignableToGenericType(parameterType, optionsType))
                .Select(options => options.GetGenericArguments().Single())
                .Distinct();
            
        var postConfigureOptionsType = typeof(IPostConfigureOptions<>);
        var configureOptionsType = typeof(IConfigureOptions<>);

        var postConfiguration = optionValueTypes.Select(type => new
        {
            optionsType = type,
            configureOptionsType = _webHost.Value.Services.GetServices(configureOptionsType.MakeGenericType(type)),
            postConfigureServices = _webHost.Value.Services.GetServices(postConfigureOptionsType.MakeGenericType(type))
        });

        var emptyConfigurations =
                postConfiguration.Where(item => item.configureOptionsType.Any() == false && item.postConfigureServices.Any() == false)
                    .Select(item => item.optionsType);

        emptyConfigurations.Should().BeEmpty("All options should be configured");
    }

    // https://stackoverflow.com/questions/74616/how-to-detect-if-type-is-another-generic-type/1075059#1075059
    public static bool IsAssignableToGenericType(Type givenType, Type genericType)
    {
        var interfaceTypes = givenType.GetInterfaces();

        if (interfaceTypes.Any(it => it.IsGenericType && it.GetGenericTypeDefinition() == genericType))
        {
            return true;
        }

        if (givenType.IsGenericType && givenType.GetGenericTypeDefinition() == genericType)
            return true;

        var baseType = givenType.BaseType;
        if (baseType == null) return false;

        return IsAssignableToGenericType(baseType, genericType);
    }
}

public class OptionsTests

{

private readonly Lazy<IWebHost> _webHost = new Lazy<IWebHost>(Program.BuildWebHost(null));

// this test will fail as not every option in assembly is configured

[Fact]

public void AllConfigurationOptions_HasConfigurationServicesDefined()

{

var optionsType = typeof(IOptions<>);

var constructorParameterTypes = typeof(Program).Assembly.GetExportedTypes()

.SelectMany(type => type.GetConstructors().SelectMany(ctor => ctor.GetParameters()))

.Select(parameter => parameter.ParameterType);

var optionValueTypes = constructorParameterTypes.Where(parameterType => IsAssignableToGenericType(parameterType, optionsType))

.Select(options => options.GetGenericArguments().Single())

.Distinct();

var postConfigureOptionsType = typeof(IPostConfigureOptions<>);

var configureOptionsType = typeof(IConfigureOptions<>);

var postConfiguration = optionValueTypes.Select(type => new

{

optionsType = type,

configureOptionsType = _webHost.Value.Services.GetServices(configureOptionsType.MakeGenericType(type)),

postConfigureServices = _webHost.Value.Services.GetServices(postConfigureOptionsType.MakeGenericType(type))

});

var emptyConfigurations =

postConfiguration.Where(item => item.configureOptionsType.Any() == false && item.postConfigureServices.Any() == false)

.Select(item => item.optionsType);

emptyConfigurations.Should().BeEmpty("All options should be configured");

}

// https://stackoverflow.com/questions/74616/how-to-detect-if-type-is-another-generic-type/1075059#1075059

public static bool IsAssignableToGenericType(Type givenType, Type genericType)

{

var interfaceTypes = givenType.GetInterfaces();

if (interfaceTypes.Any(it => it.IsGenericType && it.GetGenericTypeDefinition() == genericType))

{

return true;

}

if (givenType.IsGenericType && givenType.GetGenericTypeDefinition() == genericType)

return true;

var baseType = givenType.BaseType;

if (baseType == null) return false;

return IsAssignableToGenericType(baseType, genericType);

}

Running the test shows that NotificationOptions was not configured

Source code for this post can be found here

ASP.NET Core – passing command-line arguments to Startup class

2018-01-162018-01-16 TPodolakASP.NET Core1 Comment

1. Introduction

I usually pass configuration to my web API via JSON files, but recently I needed to pass them also via command-line arguments. As the solution for ASP.NET Core 2.x is quite different than for ASP.NET Core 1.x I decided to describe both methods in here.

2. ASP.NET Core 2.x

For the newest version of ASP.NET Core, the solution is pretty straightforward as IConfiguration is injected by default to the Startup class. All we have to do is to add CommandLineConfigurationSource to IConfigurationBuilder when configuring IWebHost. CommandLineConfigurationSource is part of the Microsoft.Extensions.Configuration.CommandLine package and can be easily used by AddCommandLine extension method

public class Program
{
    public static void Main(string[] args)
    {
        BuildWebHost(args).Run();
    }
	
    public static IWebHost BuildWebHost(string[] args)
    {
        var webHost = new WebHostBuilder()
	     .UseKestrel()
	     .UseContentRoot(Directory.GetCurrentDirectory())
	     .ConfigureAppConfiguration((hostingContext, config) =>
	     {
	         var env = hostingContext.HostingEnvironment;
	         config.AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
	               .AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true, reloadOnChange: true);
	         config.AddEnvironmentVariables();
	         config.AddCommandLine(args);
	     }).UseStartup<Startup>()
	       .Build();
        return webHost;
    }
}

public class Program

{

public static void Main(string[] args)

{

BuildWebHost(args).Run();

}

public static IWebHost BuildWebHost(string[] args)

{

var webHost = new WebHostBuilder()

.UseKestrel()

.UseContentRoot(Directory.GetCurrentDirectory())

.ConfigureAppConfiguration((hostingContext, config) =>

{

var env = hostingContext.HostingEnvironment;

config.AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)

.AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true, reloadOnChange: true);

config.AddEnvironmentVariables();

config.AddCommandLine(args);

}).UseStartup<Startup>()

.Build();

return webHost;

}

Now we can access our command-line arguments just like any other configuration

public class Startup
{
    private readonly IConfiguration _configuration;

    public Startup(IConfiguration configuration)
    {
        _configuration = configuration;
    }

    // This method gets called by the runtime. Use this method to add services to the container.
    public void ConfigureServices(IServiceCollection services)
    {
        services.AddMvc();
    }

    // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
    public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory, IApplicationLifetime applicationLifetime)
    {
        loggerFactory.AddConsole();
        loggerFactory.AddDebug();
        applicationLifetime.ApplicationStarted.Register(() => loggerFactory.CreateLogger<Startup>().LogInformation("CommandLine argument {0}: ", _configuration["usecustomschema"]));
        if (env.IsDevelopment())
        {
            app.UseDeveloperExceptionPage();
        }

        app.UseMvc();
    }
}

public class Startup

{

private readonly IConfiguration _configuration;

public Startup(IConfiguration configuration)

{

_configuration = configuration;

}

// This method gets called by the runtime. Use this method to add services to the container.

public void ConfigureServices(IServiceCollection services)

{

services.AddMvc();

}

// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.

public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory, IApplicationLifetime applicationLifetime)

{

loggerFactory.AddConsole();

loggerFactory.AddDebug();

applicationLifetime.ApplicationStarted.Register(() => loggerFactory.CreateLogger<Startup>().LogInformation("CommandLine argument {0}: ", _configuration["usecustomschema"]));

if (env.IsDevelopment())

{

app.UseDeveloperExceptionPage();

}

app.UseMvc();

}

3. ASP.NET Core 1.x

The solution for ASP.NET Core 1.x requires a bit more work but it is possible to achieve the same result. We still will be using CommandLineConfigurationSource from Microsoft.Extensions.Configuration.CommandLine package, however this time we have to manually create IConfiguration object and register it in the container before the Startup class is created. This is possible because the WebHostBuilder exposes ConfigureServices method.

public class Program
{
    public static void Main(string[] args)
    {
        var host = new WebHostBuilder()
             .UseKestrel()
             .UseContentRoot(Directory.GetCurrentDirectory())
             UseIISIntegration()
             .ConfigureServices(serviceCollection =>
                 serviceCollection.AddSingleton<IConfiguration>(new ConfigurationBuilder().AddCommandLine(args).Build()))
             .UseStartup<Startup>()
             .Build();

          host.Run();
    }
}

public class Program

{

public static void Main(string[] args)

{

var host = new WebHostBuilder()

.UseKestrel()

.UseContentRoot(Directory.GetCurrentDirectory())

UseIISIntegration()

.ConfigureServices(serviceCollection =>

serviceCollection.AddSingleton<IConfiguration>(new ConfigurationBuilder().AddCommandLine(args).Build()))

.UseStartup<Startup>()

.Build();

host.Run();

}

Having our service registered now we can easily access IConfiguration object in our application

public class Startup
{
    // This method gets called by the runtime. Use this method to add services to the container.
    public void ConfigureServices(IServiceCollection services)
    {
        // Add framework services.
        services.AddMvc();
    }

    // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
    public void Configure(IApplicationBuilder app, IHostingEnvironment hostingEnvironment, IConfiguration configuration, ILoggerFactory loggerFactory, IApplicationLifetime applicationLifetime)
    {
        loggerFactory.AddConsole();
        loggerFactory.AddDebug();
        applicationLifetime.ApplicationStarted.Register(() =>
                loggerFactory.CreateLogger<Startup>().LogInformation("CommandLine argument {0}: ", configuration["usecustomschema"]));

        if (hostingEnvironment.IsDevelopment())
        {
            app.UseDeveloperExceptionPage();
        }

        app.UseMvc();
    }
}

public class Startup

{

// This method gets called by the runtime. Use this method to add services to the container.

public void ConfigureServices(IServiceCollection services)

{

// Add framework services.

services.AddMvc();

}

// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.

public void Configure(IApplicationBuilder app, IHostingEnvironment hostingEnvironment, IConfiguration configuration, ILoggerFactory loggerFactory, IApplicationLifetime applicationLifetime)

{

loggerFactory.AddConsole();

loggerFactory.AddDebug();

applicationLifetime.ApplicationStarted.Register(() =>

loggerFactory.CreateLogger<Startup>().LogInformation("CommandLine argument {0}: ", configuration["usecustomschema"]));

if (hostingEnvironment.IsDevelopment())

{

app.UseDeveloperExceptionPage();

}

app.UseMvc();

}

Source code for this post can be found here

.NET BLOG

.NET, C# blog

ASP.NET Core

ASP.NET Core – adding controllers directly from integration tests

1. Introduction

2. Accessing ApplicationPartManager from integration tests

3. Handling private controllers

Running ASP.NET Core together with Nancy

1. Introduction

2. Legacy API

3. Combining Nancy with ASP.NET Core

ASP.NET Core – custom ILogger<T> and Nlog callsite layout renderer issue

1. Introduction

2. Implementation

3. NLog call site renderer issue

4. The fix

ASP.NET Core – make sure IOptions<T> is initialized

1. Introduction

2. Problem

3. Ensuring initialization

ASP.NET Core – passing command-line arguments to Startup class

1. Introduction

2. ASP.NET Core 2.x

3. ASP.NET Core 1.x