ASP.NET Core – adding controllers directly from integration tests

2020-06-222020-06-21 TPodolakASP.NET Core, testing

1. Introduction

From time to time it might happen, that you need to test certain parts of your ASP.NET Core configuration without hitting publicly visible business-related controllers. For instance, in my case, I wanted to make sure that my ASP.NET Core API behavior is consistent with the original API written in Nancy. As you might expect there are quite a lot of differences between both of the frameworks, so let’s focus on testing one thing, namely non-nullable reference type handling. Long story short, in order to have the same behavior in ASP.NET Core as in Nancy I had to add the following line of configuration

// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
    services.AddControllers(opts => opts.SuppressImplicitRequiredAttributeForNonNullableReferenceTypes = true)
        .AddNewtonsoftJson();
}

// This method gets called by the runtime. Use this method to add services to the container.

public void ConfigureServices(IServiceCollection services)

{

services.AddControllers(opts => opts.SuppressImplicitRequiredAttributeForNonNullableReferenceTypes = true)

.AddNewtonsoftJson();

}

This one prevents ASP.NET Core from marking non-nullable reference type properties in request as required. Having that configuration ready, I wanted to test it with an end to end test. Because there was no business-related logic yet, I needed to figure out the way of adding API controllers to my application directly from the integration tests. Here is how you can achieve that.

2. Accessing ApplicationPartManager from integration tests

ASP.NET Core is able to compose your API from different parts thanks to ApplicationPartManager. By default, you don’t use it directly but rather with IMvcBuilder AddApplicationPart extension method while setting up your application

// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
    services.AddControllers()
        .AddApplicationPart(typeof(SomeClass).Assembly)
        .AddNewtonsoftJson();
}

// This method gets called by the runtime. Use this method to add services to the container.

public void ConfigureServices(IServiceCollection services)

{

services.AddControllers()

.AddApplicationPart(typeof(SomeClass).Assembly)

.AddNewtonsoftJson();

}

However, there is no easy way of getting IMvcBuidler from the integration tests(at least I didn’t find a proper way of doing that without tampering too much with the original application pipeline) so we need to access ApplicationPartManager directly. Inspecting the source code of the framework you will find that ApplicationPartManager can be retrieved directly from IServiceCollection with following code

private static ApplicationPartManager GetApplicationPartManager(IServiceCollection services)
{
    var manager = GetServiceFromCollection<ApplicationPartManager>(services);
    if (manager == null)
    {
        manager = new ApplicationPartManager();

        var environment = GetServiceFromCollection<IWebHostEnvironment>(services);
        var entryAssemblyName = environment?.ApplicationName;
        if (string.IsNullOrEmpty(entryAssemblyName))
        {
            return manager;
        }

        manager.PopulateDefaultParts(entryAssemblyName);
    }

    return manager;
}

private static T GetServiceFromCollection<T>(IServiceCollection services)
{
    return (T)services
        .LastOrDefault(d => d.ServiceType == typeof(T))
        ?.ImplementationInstance;
}

private static ApplicationPartManager GetApplicationPartManager(IServiceCollection services)

{

var manager = GetServiceFromCollection<ApplicationPartManager>(services);

if (manager == null)

{

manager = new ApplicationPartManager();

var environment = GetServiceFromCollection<IWebHostEnvironment>(services);

var entryAssemblyName = environment?.ApplicationName;

if (string.IsNullOrEmpty(entryAssemblyName))

{

return manager;

}

manager.PopulateDefaultParts(entryAssemblyName);

}

return manager;

}

private static T GetServiceFromCollection<T>(IServiceCollection services)

{

return (T)services

.LastOrDefault(d => d.ServiceType == typeof(T))

?.ImplementationInstance;

}

Applying similar code to the WebApplicationFactory will allow us to add controllers directly from tests

[Fact]
public async Task DoesNotReturnBadRequest_WhenNonNullableTypeHasNoValue_FirstAttempt()
{
    var webApplicationFactory = new WebApplicationFactory<Startup>()
        .WithWebHostBuilder(
            builder => builder.ConfigureServices(services =>
            {
                var partManager = (ApplicationPartManager) services
                    .Last(descriptor => descriptor.ServiceType == typeof(ApplicationPartManager))
                    .ImplementationInstance;

                partManager.ApplicationParts.Add(new AssemblyPart(GetType().Assembly));
            }));

    var expectedResponse = new Request
    {
        Id = "1"
    };

    var httpClient = webApplicationFactory.CreateClient();
    var httpResponseMessage = await httpClient.GetAsync("test?id=1");

    httpResponseMessage.IsSuccessStatusCode.Should().BeTrue();
    var rawContent = await httpResponseMessage.Content.ReadAsStringAsync();
    var result = JsonConvert.DeserializeObject<Request>(rawContent);

    result.Should().BeEquivalentTo(expectedResponse);
}

[ApiController]
public class MvcOptionsTestController : ControllerBase
{
    [HttpGet("test")]
    public object Get([FromQuery] Request request)
    {
        return request;
    }
}

public class Request
{
    public string Id { get; set; }

    public string OtherId { get; set; }
}

[Fact]

public async Task DoesNotReturnBadRequest_WhenNonNullableTypeHasNoValue_FirstAttempt()

{

var webApplicationFactory = new WebApplicationFactory<Startup>()

.WithWebHostBuilder(

builder => builder.ConfigureServices(services =>

{

var partManager = (ApplicationPartManager) services

.Last(descriptor => descriptor.ServiceType == typeof(ApplicationPartManager))

.ImplementationInstance;

partManager.ApplicationParts.Add(new AssemblyPart(GetType().Assembly));

}));

var expectedResponse = new Request

{

Id = "1"

};

var httpClient = webApplicationFactory.CreateClient();

var httpResponseMessage = await httpClient.GetAsync("test?id=1");

httpResponseMessage.IsSuccessStatusCode.Should().BeTrue();

var rawContent = await httpResponseMessage.Content.ReadAsStringAsync();

var result = JsonConvert.DeserializeObject<Request>(rawContent);

result.Should().BeEquivalentTo(expectedResponse);

}

[ApiController]

public class MvcOptionsTestController : ControllerBase

{

[HttpGet("test")]

public object Get([FromQuery] Request request)

{

return request;

}

public class Request

{

public string Id { get; set; }

public string OtherId { get; set; }

}

3. Handling private controllers

The solution presented below works nice, however it has couple of drawbacks. First of all, it only discovers public non-nested controllers. Second of all, it will always add all controllers from given assembly – which potentially might affect other integration tests. In order to get rid of these drawbacks, we need to tell ApplicationPartManager to include only selected controllers. We can achieve that by adding additional IApplicationFeatureProvider<ControllerFeature> to the list of FeatureProviders.

public class ExternalControllersFeatureProvider : IApplicationFeatureProvider<ControllerFeature>
{
    private readonly Type[] _controllers;

    public ExternalControllersFeatureProvider(params Type[] controllers)
    {
        _controllers = controllers ?? Array.Empty<Type>();
    }

    public void PopulateFeature(IEnumerable<ApplicationPart> parts, ControllerFeature feature)
    {
        foreach (var controller in _controllers)
        {
            feature.Controllers.Add(controller.GetTypeInfo());
        }
    }
}

[Fact]
public async Task DoesNotReturnBadRequest_WhenNonNullableTypeHasNoValue()
{
    var webApplicationFactory = new WebApplicationFactory<Startup>()
        .WithWebHostBuilder(
            builder => builder.ConfigureServices(services =>
            {
                var partManager = (ApplicationPartManager) services
                    .Last(descriptor => descriptor.ServiceType == typeof(ApplicationPartManager))
                    .ImplementationInstance;

                partManager.FeatureProviders.Add(new ExternalControllersFeatureProvider(typeof(MvcOptionsTestController)));
            }));
// rest of the code omitted for brevity
}

public class ExternalControllersFeatureProvider : IApplicationFeatureProvider<ControllerFeature>

{

private readonly Type[] _controllers;

public ExternalControllersFeatureProvider(params Type[] controllers)

{

_controllers = controllers ?? Array.Empty<Type>();

}

public void PopulateFeature(IEnumerable<ApplicationPart> parts, ControllerFeature feature)

{

foreach (var controller in _controllers)

{

feature.Controllers.Add(controller.GetTypeInfo());

}

[Fact]

public async Task DoesNotReturnBadRequest_WhenNonNullableTypeHasNoValue()

{

var webApplicationFactory = new WebApplicationFactory<Startup>()

.WithWebHostBuilder(

builder => builder.ConfigureServices(services =>

{

var partManager = (ApplicationPartManager) services

.Last(descriptor => descriptor.ServiceType == typeof(ApplicationPartManager))

.ImplementationInstance;

partManager.FeatureProviders.Add(new ExternalControllersFeatureProvider(typeof(MvcOptionsTestController)));

}));

// rest of the code omitted for brevity

}

Putting it all together and applying some refactoring we will end up with following code

internal static class WebHostBuilderExtensions
{
    public static IWebHostBuilder WithAdditionalControllers(this IWebHostBuilder builder, params Type[] controllers)
    {
        return builder.ConfigureTestServices(
            services =>
            {
                var partManager = GetApplicationPartManager(services);

                partManager.FeatureProviders.Add(new ExternalControllersFeatureProvider(controllers));
            });
    }

    private static ApplicationPartManager GetApplicationPartManager(IServiceCollection services)
    {
        var partManager = (ApplicationPartManager)services
            .Last(descriptor => descriptor.ServiceType == typeof(ApplicationPartManager))
            .ImplementationInstance;
        return partManager;
    }

    private class ExternalControllersFeatureProvider : IApplicationFeatureProvider<ControllerFeature>
    {
        private readonly Type[] _controllers;

        public ExternalControllersFeatureProvider(params Type[] controllers)
        {
            _controllers = controllers ?? Array.Empty<Type>();
        }

        public void PopulateFeature(IEnumerable<ApplicationPart> parts, ControllerFeature feature)
        {
            foreach (var controller in _controllers)
            {
                feature.Controllers.Add(controller.GetTypeInfo());
            }
        }
    }
}

public class MvcOptionsConfigurationTests
{
    [Fact]
    public async Task DoesNotReturnBadRequest_WhenNonNullableTypeHasNoValue()
    {
        var webApplicationFactory = new WebApplicationFactory<Startup>()
            .WithWebHostBuilder(
                builder => builder.WithAdditionalControllers(typeof(MvcOptionsTestController)));

        var httpClient = webApplicationFactory.CreateClient();
        var expectedResponse = new Request
        {
            Id = "1"
        };

        var httpResponseMessage = await httpClient.GetAsync("test?id=1");

        httpResponseMessage.IsSuccessStatusCode.Should().BeTrue();
        var rawContent = await httpResponseMessage.Content.ReadAsStringAsync();
        var result = JsonConvert.DeserializeObject<Request>(rawContent);

        result.Should().BeEquivalentTo(expectedResponse);
    }

    [ApiController]
    private class MvcOptionsTestController: ControllerBase
    {
        [HttpGet("test")]
        public object Get([FromQuery] Request request)
        {
            return request;
        }
    }

    private class Request
    {
        public string Id { get; set; }

        public string OtherId { get; set; }
    }
}

internal static class WebHostBuilderExtensions

{

public static IWebHostBuilder WithAdditionalControllers(this IWebHostBuilder builder, params Type[] controllers)

{

return builder.ConfigureTestServices(

services =>

{

var partManager = GetApplicationPartManager(services);

partManager.FeatureProviders.Add(new ExternalControllersFeatureProvider(controllers));

});

}

private static ApplicationPartManager GetApplicationPartManager(IServiceCollection services)

{

var partManager = (ApplicationPartManager)services

.Last(descriptor => descriptor.ServiceType == typeof(ApplicationPartManager))

.ImplementationInstance;

return partManager;

}

private class ExternalControllersFeatureProvider : IApplicationFeatureProvider<ControllerFeature>

{

private readonly Type[] _controllers;

public ExternalControllersFeatureProvider(params Type[] controllers)

{

_controllers = controllers ?? Array.Empty<Type>();

}

public void PopulateFeature(IEnumerable<ApplicationPart> parts, ControllerFeature feature)

{

foreach (var controller in _controllers)

{

feature.Controllers.Add(controller.GetTypeInfo());

}

public class MvcOptionsConfigurationTests

{

[Fact]

public async Task DoesNotReturnBadRequest_WhenNonNullableTypeHasNoValue()

{

var webApplicationFactory = new WebApplicationFactory<Startup>()

.WithWebHostBuilder(

builder => builder.WithAdditionalControllers(typeof(MvcOptionsTestController)));

var httpClient = webApplicationFactory.CreateClient();

var expectedResponse = new Request

{

Id = "1"

};

var httpResponseMessage = await httpClient.GetAsync("test?id=1");

httpResponseMessage.IsSuccessStatusCode.Should().BeTrue();

var rawContent = await httpResponseMessage.Content.ReadAsStringAsync();

var result = JsonConvert.DeserializeObject<Request>(rawContent);

result.Should().BeEquivalentTo(expectedResponse);

}

[ApiController]

private class MvcOptionsTestController: ControllerBase

{

[HttpGet("test")]

public object Get([FromQuery] Request request)

{

return request;

}

private class Request

{

public string Id { get; set; }

public string OtherId { get; set; }

}

Source code for this post can be found here

Running ASP.NET Core together with Nancy

2020-04-202020-04-19 TPodolakASP.NET Core, Nancy

1. Introduction

Our current API runs on Nancy which in my opinion is past its prime. Recent news from the GitHub issue tracker seems to confirm that thesis, that is why we started looking for a migration path from Nancy based API to ASP.NET Core. Because the codebase is quite large we didn’t want to do a Big Bang Rewrite but instead of that, we wanted to gradually replace old API with a new one, so both of them can coexist next to each other.

2. Legacy API

Before I jump into implementation, here is a sample Nancy API with two endpoints – products and variants

public class Startup
{
    // This method gets called by the runtime. Use this method to add services to the container.
    public void ConfigureServices(IServiceCollection services)
    {
        // nancy has its own container
    }

    // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
    public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
    {
        app.UseOwin(owin => owin.UseNancy());
    }
}

public class Startup

{

// This method gets called by the runtime. Use this method to add services to the container.

public void ConfigureServices(IServiceCollection services)

{

// nancy has its own container

}

// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)

{

app.UseOwin(owin => owin.UseNancy());

}

public sealed class ProductsNancyModule : NancyModule
{
    public ProductsNancyModule()
    {
        Get("products", args => Response.AsJson(new List<Product>
        {
            new Product(Guid.NewGuid().ToString(), "First product from nancy"),
            new Product(Guid.NewGuid().ToString(), "Second product from nancy")
        }));
    }
}

public sealed class ProductsNancyModule : NancyModule

{

public ProductsNancyModule()

{

Get("products", args => Response.AsJson(new List<Product>

{

new Product(Guid.NewGuid().ToString(), "First product from nancy"),

new Product(Guid.NewGuid().ToString(), "Second product from nancy")

}));

}

public sealed class VariantsNancyModule : NancyModule
{
    public VariantsNancyModule()
    {
        Get("variants", args => Response.AsJson(new List<Variant>
        {
            new Variant(Guid.NewGuid().ToString(), Guid.NewGuid().ToString(), "first variant from nancy"),
            new Variant(Guid.NewGuid().ToString(), Guid.NewGuid().ToString(), "second variant from nancy")
        }));
    }
}

public sealed class VariantsNancyModule : NancyModule

{

public VariantsNancyModule()

{

Get("variants", args => Response.AsJson(new List<Variant>

{

new Variant(Guid.NewGuid().ToString(), Guid.NewGuid().ToString(), "first variant from nancy"),

new Variant(Guid.NewGuid().ToString(), Guid.NewGuid().ToString(), "second variant from nancy")

}));

}

The goal is to replace products endpoint with ASP.NET Core implementation while variants endpoint should still be served by Nancy

3. Combining Nancy with ASP.NET Core

The solution is based on branching the pipeline feature which is available starting from ASP.NET Core 2.1. Long story short – it is possible to configure different pipelines for different route paths thanks to

public static IApplicationBuilder Map(this IApplicationBuilder app, PathString pathMatch, Action<IApplicationBuilder> configuration)

1	public static IApplicationBuilder Map(this IApplicationBuilder app, PathString pathMatch, Action<IApplicationBuilder> configuration)

method. Having that in mind we can Map products path to run through ASP.NET Core pipeline whereas the rest would go through Nancy.

// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
    // register mvc services
    services.AddControllers();
    services.AddMvc(options => options.EnableEndpointRouting = false);
}

// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
    // run everything with products prefix through mvc
    app.Map("/products", appBuilder =>
    {
        appBuilder.UseMvc();
    });

    // run the rest through Nancy
    app.Map(string.Empty, appBuilder =>
    {
        appBuilder.UseOwin(options => options.UseNancy());
    });
}

// This method gets called by the runtime. Use this method to add services to the container.

public void ConfigureServices(IServiceCollection services)

{

// register mvc services

services.AddControllers();

services.AddMvc(options => options.EnableEndpointRouting = false);

}

// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)

{

// run everything with products prefix through mvc

app.Map("/products", appBuilder =>

{

appBuilder.UseMvc();

});

// run the rest through Nancy

app.Map(string.Empty, appBuilder =>

{

appBuilder.UseOwin(options => options.UseNancy());

});

}

[ApiController]
public class ProductsController : ControllerBase
{
    [HttpGet("/products")]
    public List<Product> Get()
    {
        return new List<Product>
        {
            new Product(Guid.NewGuid().ToString(), "First product from asp net core"),
            new Product(Guid.NewGuid().ToString(), "Second product from asp net core")
        };
    }
}

[ApiController]

public class ProductsController : ControllerBase

{

[HttpGet("/products")]

public List<Product> Get()

{

return new List<Product>

{

new Product(Guid.NewGuid().ToString(), "First product from asp net core"),

new Product(Guid.NewGuid().ToString(), "Second product from asp net core")

};

}

At this point we are almost there, however running request through Map pipeline will remove the path prefix, meaning that our controller would be accessible under / path instead of products. In order to bypass this limitation we have to restore original prefix with RewriteMiddleware. Once we put it all together, now we are able to replace multiple Nancy endpoints with ASP.NET Core ones using following piece of code

// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
    var aspNetCorePaths = new PathString[] {"/products"};
    foreach (var path in aspNetCorePaths)
    {
        app.Map(path, appBuilder =>
        {
            // we still want to access the resource under old url, so we need to add the branch path back to the route
            var rewriteOptions = new RewriteOptions().AddRewrite(
                "(?<Path>.*)",
                $"{path}/${{Path}}",
                skipRemainingRules: true);
            appBuilder.UseRewriter(rewriteOptions);
            appBuilder.UseMvc();
        });
    }

    app.Map(string.Empty, appBuilder =>
    {
        appBuilder.UseOwin(options => options.UseNancy());
    });
}

// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)

{

var aspNetCorePaths = new PathString[] {"/products"};

foreach (var path in aspNetCorePaths)

{

app.Map(path, appBuilder =>

{

// we still want to access the resource under old url, so we need to add the branch path back to the route

var rewriteOptions = new RewriteOptions().AddRewrite(

"(?<Path>.*)",

$"{path}/${{Path}}",

skipRemainingRules: true);

appBuilder.UseRewriter(rewriteOptions);

appBuilder.UseMvc();

});

}

app.Map(string.Empty, appBuilder =>

{

appBuilder.UseOwin(options => options.UseNancy());

});

}

Source code for this post can be found here

Introduce substitute refactoring comes to NSubstitute.Analyzers

2020-04-092020-04-12 TPodolakNSubstitute.Analyzers

I am glad to announce the newest version of NSubstitute.Analyzers comes with “introduce substitute” refactoring, which allows you to automatically create substitutes for constructor arguments as fields or local variables. As an image is worth a thousand words here is a quick demo of C# version

and VisualBasic one

.NET Core – missing currency symbol in docker alpine image

2019-11-18 TPodolakdocker

During the process of moving a Scala-based API to .NET Core, we encountered an interesting localization issue when running our code in a docker container based on an alpine image. The code itself was doing a currency formatting based on some culture. It looked more or less as below

[Route("api/[controller]")]
[ApiController]
public class PriceController : ControllerBase
{
    [HttpGet]
    public ActionResult<object> Get(string cultureCode = "de-DE")
    {
        var cultureInfo = CultureInfo.CreateSpecificCulture(cultureCode);
        var price = 10m;
        
        return new
        {
            price,
            formattedPrice = price.ToString("C", cultureInfo),
            currencySymbol = cultureInfo.NumberFormat.CurrencySymbol
        };
    }
}

[Route("api/[controller]")]

[ApiController]

public class PriceController : ControllerBase

{

[HttpGet]

public ActionResult<object> Get(string cultureCode = "de-DE")

{

var cultureInfo = CultureInfo.CreateSpecificCulture(cultureCode);

var price = 10m;

return new

{

price,

formattedPrice = price.ToString("C", cultureInfo),

currencySymbol = cultureInfo.NumberFormat.CurrencySymbol

};

}

We also had some integration tests for that piece of logic

public class PriceTests : IClassFixture<WebApplicationFactory<Startup>>
{
    private readonly WebApplicationFactory<Startup> _factory;

    public PriceTests(WebApplicationFactory<Startup> factory)
    {
        _factory = factory;
    }
        
    [Fact]
    public async Task GetPrice_ReturnsCorrectCurrency()
    {
        var httpClient = _factory.CreateClient();
        var httpResponseMessage = await httpClient.GetAsync("api/Price");
            
        httpResponseMessage.EnsureSuccessStatusCode();
        var content = await httpResponseMessage.Content.ReadAsStringAsync();
        var jObject = JObject.Parse(content);

        jObject["currencySymbol"].ToObject<string>().Should().Be("€");
    }
}

public class PriceTests : IClassFixture<WebApplicationFactory<Startup>>

{

private readonly WebApplicationFactory<Startup> _factory;

public PriceTests(WebApplicationFactory<Startup> factory)

{

_factory = factory;

}

[Fact]

public async Task GetPrice_ReturnsCorrectCurrency()

{

var httpClient = _factory.CreateClient();

var httpResponseMessage = await httpClient.GetAsync("api/Price");

httpResponseMessage.EnsureSuccessStatusCode();

var content = await httpResponseMessage.Content.ReadAsStringAsync();

var jObject = JObject.Parse(content);

jObject["currencySymbol"].ToObject<string>().Should().Be("€");

}

The tests were run during a CI build in a container with an image defined as below

# Builder image
FROM mcr.microsoft.com/dotnet/core/sdk:2.1.700-alpine AS builder
WORKDIR /sln

COPY ./*.sln ./

# Copy the main source project files
COPY src/*/*.csproj ./
RUN for file in $(ls *.csproj); do mkdir -p src/${file%.*}/ && mv $file src/${file%.*}/; done

# Copy the test project files
COPY test/*/*.csproj ./
RUN for file in $(ls *.csproj); do mkdir -p test/${file%.*}/ && mv $file test/${file%.*}/; done

RUN dotnet restore

# Copy across the rest of the source files
COPY ./test ./test
COPY ./src ./src

RUN dotnet build -c Release

RUN dotnet test "./test/AlpineMissingCurrencySymbol.Tests/AlpineMissingCurrencySymbol.Tests.csproj" \
    -c Release --no-build --no-restore

RUN dotnet publish "./src/AlpineMissingCurrencySymbol/AlpineMissingCurrencySymbol.csproj" \
    -c Release -o "../../dist" --no-restore

# App image
FROM mcr.microsoft.com/dotnet/core/aspnet:2.1.11-alpine
WORKDIR /app
ENTRYPOINT ["dotnet", "AlpineMissingCurrencySymbol.dll"]
COPY --from=builder /sln/dist .

# Builder image

FROM mcr.microsoft.com/dotnet/core/sdk:2.1.700-alpine AS builder

WORKDIR /sln

COPY ./*.sln ./

# Copy the main source project files

COPY src/*/*.csproj ./

RUN for file in $(ls *.csproj); do mkdir -p src/${file%.*}/ && mv $file src/${file%.*}/; done

# Copy the test project files

COPY test/*/*.csproj ./

RUN for file in $(ls *.csproj); do mkdir -p test/${file%.*}/ && mv $file test/${file%.*}/; done

RUN dotnet restore

# Copy across the rest of the source files

COPY ./test ./test

COPY ./src ./src

RUN dotnet build -c Release

RUN dotnet test "./test/AlpineMissingCurrencySymbol.Tests/AlpineMissingCurrencySymbol.Tests.csproj" \

-c Release --no-build --no-restore

RUN dotnet publish "./src/AlpineMissingCurrencySymbol/AlpineMissingCurrencySymbol.csproj" \

-c Release -o "../../dist" --no-restore

# App image

FROM mcr.microsoft.com/dotnet/core/aspnet:2.1.11-alpine

WORKDIR /app

ENTRYPOINT ["dotnet", "AlpineMissingCurrencySymbol.dll"]

COPY --from=builder /sln/dist .

At this point, we were sure that everything works fine, as the tests were green and everything was also working correctly on our local machines. However, after deployment to a testing environment, we started getting invalid currency symbols

curl localhost:8080/api/price 
{ 
  "price": 10, 
  "formattedPrice": "¤10.00", 
  "currencySymbol": "¤" 
}

curl localhost:8080/api/price

{

"price": 10,

"formattedPrice": "¤10.00",

"currencySymbol": "¤"

}

As you can see the response contains ¤ (invariant currency symbol) instead of expected €. It took us some time to figure this out but finally, it turned out that aspnet:2.1.11-alpine image(the one we used for running the application) contrary to the SDK image(used for building and running tests) is missing icu-libs package. In default conditions, the application should throw the following exception during the startup

FailFast:
Couldn't find a valid ICU package installed on the system. Set the configuration flag System.Globalization.Invariant to true if you want to run with no globalization support.

   at System.Environment.FailFast(System.String)
   at System.Globalization.GlobalizationMode.GetGlobalizationInvariantMode()
   at System.Globalization.GlobalizationMode..cctor()
   at System.Globalization.CultureData.CreateCultureWithInvariantData()
   at System.Globalization.CultureData.get_Invariant()
   at System.Globalization.CultureInfo..cctor()
   at System.StringComparer..cctor()
   at System.AppDomain.InitializeCompatibilityFlags()
   at System.AppDomain.Setup(System.Object)

FailFast:

Couldn't find a valid ICU package installed on the system. Set the configuration flag System.Globalization.Invariant to true if you want to run with no globalization support.

at System.Environment.FailFast(System.String)

at System.Globalization.GlobalizationMode.GetGlobalizationInvariantMode()

at System.Globalization.GlobalizationMode..cctor()

at System.Globalization.CultureData.CreateCultureWithInvariantData()

at System.Globalization.CultureData.get_Invariant()

at System.Globalization.CultureInfo..cctor()

at System.StringComparer..cctor()

at System.AppDomain.InitializeCompatibilityFlags()

at System.AppDomain.Setup(System.Object)

However, aspnet:2.1.11-alpine image has the DOTNET_SYSTEM_GLOBALIZATION_INVARIANT flag set to true by default, so the missing package was not validated during a startup. After all, in order to fix the issue, we had to install the icu-libs package and also set the DOTNET_SYSTEM_GLOBALIZATION_INVARIANT back to false. This was done by these two lines in Dockerfile

ENV DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=false 
RUN apk add --no-cache icu-libs

1 2	ENV DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=false RUN apk add --no-cache icu-libs

Once the lines were added the application started working as expected

curl localhost:8080/api/price 
{ 
  "price": 10, 
  "formattedPrice": "10,00 €", 
  "currencySymbol": "€" 
}

curl localhost:8080/api/price

{

"price": 10,

"formattedPrice": "10,00 €",

"currencySymbol": "€"

}

Source code for this post can be found here

MongoDB.Driver – class-based server side projection

2019-10-222019-10-20 TPodolakMongoDB, NoSql

1. Introduction

When working with NoSQL databases, your documents might be quite heavy and in some cases, you would like to get only a slice of original data. For instance, let’s assume we have an Account document which among other things contains a list of transactions

public class Account : IAccountDefinition
{
    public ObjectId Id { get; set; }

    public string Name { get; set; }

    public List<Transaction> Transactions { get; set; }
}

public class Transaction
{
    public ObjectId Id { get; set; }
        
    public decimal Amount { get; set; }
        
    public DateTime CreatedAt { get; set; }
        
    public DateTime ModifiedAt { get; set; }
}

public class Account : IAccountDefinition

{

public ObjectId Id { get; set; }

public string Name { get; set; }

public List<Transaction> Transactions { get; set; }

}

public class Transaction

{

public ObjectId Id { get; set; }

public decimal Amount { get; set; }

public DateTime CreatedAt { get; set; }

public DateTime ModifiedAt { get; set; }

}

As there might be hundreds of transaction in the account object, you might want to occasionally work on a subset of original data, say for instance AccountSlim object to improve the performance

public class AccountSlim : IAccountDefinition
{
    public ObjectId Id { get; set; }
        
        public string Name { get; set; }
    }

public class AccountSlim : IAccountDefinition

{

public ObjectId Id { get; set; }

public string Name { get; set; }

}

2.Exploring existing options

MongoDB.Driver has a couple of ways of defining projections so as you can operate on a slimmer object instead of a default one. For instance

List<AccountSlim> accountSlims = await accountsCollection.Find(defaultAccountFilterDefinition)
                .Project(Builders<Account>.Projection.As<AccountSlim>())
                .ToListAsync();

List<AccountSlim> accountSlims = await accountsCollection.Find(defaultAccountFilterDefinition)

.Project(Builders<Account>.Projection.As<AccountSlim>())

.ToListAsync();

IAsyncCursor<AccountSlim> asyncCursor = await accountsCollection.FindAsync(defaultAccountFilterDefinition,
                new FindOptions<Account, AccountSlim>());

1 2	IAsyncCursor<AccountSlim> asyncCursor = await accountsCollection.FindAsync(defaultAccountFilterDefinition, new FindOptions<Account, AccountSlim>());

Unfortunately, those are a client-side projection. This means that the entire object is returned from the database and we just serialized it to a different class. You can check it on your own by examining request and result commands sent to Mongo

var clientSettings = MongoClientSettings.FromUrl(new MongoUrl("mongodb://localhost:27017")); 
clientSettings.ClusterConfigurator += builder => 
{ 
    builder.Subscribe<CommandStartedEvent>(OnCommandStarted); 
    builder.Subscribe<CommandSucceededEvent>(OnCommandSucceeded); 
}; 
var client = new MongoClient(clientSettings); 

private static void OnCommandSucceeded(CommandSucceededEvent @event) 
{ 
    if (@event.CommandName == FindCommandName) 
    { 
        Console.WriteLine("Returned document:"); 
        PrintDocument(@event.Reply); 
    } 
} 
 
private static void OnCommandStarted(CommandStartedEvent @event) 
{ 
    if (@event.CommandName == FindCommandName) 
    { 
        Console.WriteLine("Requested document:"); 
        PrintDocument(@event.Command); 
    } 
}

var clientSettings = MongoClientSettings.FromUrl(new MongoUrl("mongodb://localhost:27017"));

clientSettings.ClusterConfigurator += builder =>

{

builder.Subscribe<CommandStartedEvent>(OnCommandStarted);

builder.Subscribe<CommandSucceededEvent>(OnCommandSucceeded);

};

var client = new MongoClient(clientSettings);

private static void OnCommandSucceeded(CommandSucceededEvent @event)

{

if (@event.CommandName == FindCommandName)

{

Console.WriteLine("Returned document:");

PrintDocument(@event.Reply);

}

private static void OnCommandStarted(CommandStartedEvent @event)

{

if (@event.CommandName == FindCommandName)

{

Console.WriteLine("Requested document:");

PrintDocument(@event.Command);

}

In both cases requested query doesn’t contain “projection” section, so entire document is returned

// query 
{ 

  "find" : "Accounts", 

  "filter" : { 

    "_id" : ObjectId("5dac6ab1ea342a254e1b7748") 

  }, 

  "$db" : "Accounting", 

  "lsid" : { 

    "id" : CSUUID("c6b40361-c47f-4fe6-99bd-6a9ec1702576") 

  } 

} 

// returned document 

{ 

  "cursor" : { 

    "firstBatch" : [{ 

        "_id" : ObjectId("5dac6ab1ea342a254e1b7748"), 

        "AccountName" : "WRKYDWKKJY", 

        "Transactions" : [{ 

            "_id" : ObjectId("5dac6ab1ea342a254e1b7749"), 

            "Amount" : "413648280", 

            "CreatedAt" : ISODate("2019-10-20T14:09:53.635Z"), 

            "ModifiedAt" : ISODate("2019-10-20T14:09:53.635Z") 

          }] 

      }], 

    "id" : NumberLong(0), 

    "ns" : "Accounting.Accounts" 

  }, 

  "ok" : 1.0 

}

// query

{

"find" : "Accounts",

"filter" : {

"_id" : ObjectId("5dac6ab1ea342a254e1b7748")

"$db" : "Accounting",

"lsid" : {

"id" : CSUUID("c6b40361-c47f-4fe6-99bd-6a9ec1702576")

}

// returned document

{

"cursor" : {

"firstBatch" : [{

"_id" : ObjectId("5dac6ab1ea342a254e1b7748"),

"AccountName" : "WRKYDWKKJY",

"Transactions" : [{

"_id" : ObjectId("5dac6ab1ea342a254e1b7749"),

"Amount" : "413648280",

"CreatedAt" : ISODate("2019-10-20T14:09:53.635Z"),

"ModifiedAt" : ISODate("2019-10-20T14:09:53.635Z")

}]

}],

"id" : NumberLong(0),

"ns" : "Accounting.Accounts"

"ok" : 1.0

}

Of course, there is a possibility to manually create a server-side projection, for instance

var accountSlims = await accountsCollection.Find(defaultAccountFilterDefinition)
                .Project(new ObjectProjectionDefinition<Account, AccountSlim>(new { id = 1, name = 1 }))
                .ToListAsync();

var accountSlims = await accountsCollection.Find(defaultAccountFilterDefinition)

.Project(new ObjectProjectionDefinition<Account, AccountSlim>(new { id = 1, name = 1 }))

.ToListAsync();

or with a strongly typed version

var bsonDocument = Builders<Account>.Projection.Exclude(x => x.Transactions)
                .Render(accountsCollection.DocumentSerializer, accountsCollection.Settings.SerializerRegistry);

List<AccountSlim> accountSlim = await accountsCollection.Find(defaultAccountFilterDefinition)
                .Project(new BsonDocumentProjectionDefinition<Account, AccountSlim>(bsonDocument))
                .ToListAsync();

var bsonDocument = Builders<Account>.Projection.Exclude(x => x.Transactions)

.Render(accountsCollection.DocumentSerializer, accountsCollection.Settings.SerializerRegistry);

List<AccountSlim> accountSlim = await accountsCollection.Find(defaultAccountFilterDefinition)

.Project(new BsonDocumentProjectionDefinition<Account, AccountSlim>(bsonDocument))

.ToListAsync();

However, in my opinion, this is error-prone and it would be better to generate server-side projection automatically based on properties in a slim object. As I didn’t find anything like that in the official driver, here is my approach for handling this

3. Class-based server-side projection

In order to create a custom projection, all we have to do is to extend ProjectionDefinition<TSource, TResult> class and provide a RenderedProjectionDefinition with all properties which are in both “heavy” and “slim” object

using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.Linq;
using System.Reflection;
using MongoDB.Bson.Serialization;
using MongoDB.Driver;

namespace MongoDBServerSideProjection.Projections
{
    public class ServerSideProjectionDefinition<TSource, TResult> : ProjectionDefinition<TSource, TResult>
    {
        private readonly ProjectionDefinition<TSource> _projectionDefinition;

        public ServerSideProjectionDefinition()
        {
            _projectionDefinition = BuildProjectionDefinition();
        }

        public override RenderedProjectionDefinition<TResult> Render(IBsonSerializer<TSource> sourceSerializer, IBsonSerializerRegistry serializerRegistry)
        {
            var bsonDocument = _projectionDefinition.Render(sourceSerializer, serializerRegistry);

            return new RenderedProjectionDefinition<TResult>(bsonDocument, serializerRegistry.GetSerializer<TResult>());
        }

        private static ProjectionDefinition<TSource> BuildProjectionDefinition()
        {
            var sourceProperties = ProjectionPropertyCache<TSource>.PropertyNames;
            var resultProperties = ProjectionPropertyCache<TResult>.PropertyNames;

            var projectionDefinitionBuilder = Builders<TSource>.Projection;

            var projectionDefinitions = resultProperties.Intersect(sourceProperties).Select(
                name => projectionDefinitionBuilder.Include(new StringFieldDefinition<TSource>(name)));

            return projectionDefinitionBuilder.Combine(projectionDefinitions);
        }

        private class ProjectionPropertyCache<T>
        {
            static ProjectionPropertyCache()
            {
                PropertyNames = GetProperties();
            }

            public static IReadOnlyCollection<string> PropertyNames { get; }

            private static ReadOnlyCollection<string> GetProperties()
            {
                return typeof(T).GetProperties(BindingFlags.Public | BindingFlags.Instance)
                    .Where(prop => !prop.IsSpecialName).Select(prop => prop.Name).ToList().AsReadOnly();
            }
        }
    }
}

using System.Collections.Generic;

using System.Collections.ObjectModel;

using System.Linq;

using System.Reflection;

using MongoDB.Bson.Serialization;

using MongoDB.Driver;

namespace MongoDBServerSideProjection.Projections

{

public class ServerSideProjectionDefinition<TSource, TResult> : ProjectionDefinition<TSource, TResult>

{

private readonly ProjectionDefinition<TSource> _projectionDefinition;

public ServerSideProjectionDefinition()

{

_projectionDefinition = BuildProjectionDefinition();

}

public override RenderedProjectionDefinition<TResult> Render(IBsonSerializer<TSource> sourceSerializer, IBsonSerializerRegistry serializerRegistry)

{

var bsonDocument = _projectionDefinition.Render(sourceSerializer, serializerRegistry);

return new RenderedProjectionDefinition<TResult>(bsonDocument, serializerRegistry.GetSerializer<TResult>());

}

private static ProjectionDefinition<TSource> BuildProjectionDefinition()

{

var sourceProperties = ProjectionPropertyCache<TSource>.PropertyNames;

var resultProperties = ProjectionPropertyCache<TResult>.PropertyNames;

var projectionDefinitionBuilder = Builders<TSource>.Projection;

var projectionDefinitions = resultProperties.Intersect(sourceProperties).Select(

name => projectionDefinitionBuilder.Include(new StringFieldDefinition<TSource>(name)));

return projectionDefinitionBuilder.Combine(projectionDefinitions);

}

private class ProjectionPropertyCache<T>

{

static ProjectionPropertyCache()

{

PropertyNames = GetProperties();

}

public static IReadOnlyCollection<string> PropertyNames { get; }

private static ReadOnlyCollection<string> GetProperties()

{

return typeof(T).GetProperties(BindingFlags.Public | BindingFlags.Instance)

.Where(prop => !prop.IsSpecialName).Select(prop => prop.Name).ToList().AsReadOnly();

}

As you can see we use MongoDB.Driver build-in projections to render our custom projection consisting of necessary properties. Note, that as we are using StringFieldDefinition instead of defining Bson document manually, the projection will take into account potential class mappings or attribute mappings applied to your object

BsonClassMap.RegisterClassMap<Account>(cm =>
            {
                cm.AutoMap();
                cm.MapProperty(account => account.Name).SetElementName("AccountName");
            });

BsonClassMap.RegisterClassMap<Account>(cm =>

{

cm.AutoMap();

cm.MapProperty(account => account.Name).SetElementName("AccountName");

});

Having the projection ready we can make it a bit easier to use by introducing some extensions. The first one looks as follows

using MongoDB.Driver;
using MongoDBServerSideProjection.Projections;

namespace MongoDBServerSideProjection.Extensions
{
    public static class ProjectionDefinitionBuilderExtensions
    {
        public static ServerSideProjectionDefinition<TSource, TResult> ServerSide<TSource, TResult>(this ProjectionDefinitionBuilder<TSource> builder)
        {
            return new ServerSideProjectionDefinition<TSource, TResult>();
        }
    }
}

using MongoDB.Driver;

using MongoDBServerSideProjection.Projections;

namespace MongoDBServerSideProjection.Extensions

{

public static class ProjectionDefinitionBuilderExtensions

{

public static ServerSideProjectionDefinition<TSource, TResult> ServerSide<TSource, TResult>(this ProjectionDefinitionBuilder<TSource> builder)

{

return new ServerSideProjectionDefinition<TSource, TResult>();

}

which allows you to use this projection similarly like others – so by accessing Builders class

var projection = Builders<Account>.Projection.ServerSide<Account, AccountSlim>();
List<AccountSlim> slims = await accountsCollection.Find(defaultAccountFilterDefinition)
                .Project(projection)
                .ToListAsync();

var projection = Builders<Account>.Projection.ServerSide<Account, AccountSlim>();

List<AccountSlim> slims = await accountsCollection.Find(defaultAccountFilterDefinition)

.Project(projection)

.ToListAsync();

The second method will extend IFindFluent<TDocument, TProjection> interface

using MongoDB.Driver;

namespace MongoDBServerSideProjection.Extensions
{
    public static class FindFluentExtensions
    {
        public static IFindFluent<TSource, TResult> ProjectTo<TSource, TResult>(this IFindFluent<TSource, TSource> findFluent)
        {
            var serverSideProjectionDefinition = Builders<TSource>.Projection.ServerSide<TSource, TResult>();
            return findFluent.Project(serverSideProjectionDefinition);
        }
    }
}

using MongoDB.Driver;

namespace MongoDBServerSideProjection.Extensions

{

public static class FindFluentExtensions

{

public static IFindFluent<TSource, TResult> ProjectTo<TSource, TResult>(this IFindFluent<TSource, TSource> findFluent)

{

var serverSideProjectionDefinition = Builders<TSource>.Projection.ServerSide<TSource, TResult>();

return findFluent.Project(serverSideProjectionDefinition);

}

and thanks to it we will end up with an even better syntax

List<AccountSlim> list = await accountsCollection.Find(defaultAccountFilterDefinition)
                .ProjectTo<Account, AccountSlim>()
                .ToListAsync();

List<AccountSlim> list = await accountsCollection.Find(defaultAccountFilterDefinition)

.ProjectTo<Account, AccountSlim>()

.ToListAsync();

One way or another we will end up with proper projection definition which will result in a smaller document returned from the database

// query
{ 

  "find" : "Accounts", 

  "filter" : { 

    "_id" : ObjectId("5dac846eea29403cb83f99bb") 

  }, 

  "projection" : { 

    "_id" : 1, 

    "AccountName" : 1 

  }, 

  "$db" : "Accounting", 

  "lsid" : { 

    "id" : CSUUID("cf5d5a88-2598-4110-9295-6417d1a5f7dd") 

  } 

} 

// returned document

{ 

  "cursor" : { 

    "firstBatch" : [{ 

        "_id" : ObjectId("5dac846eea29403cb83f99bb"), 

        "AccountName" : "CBBRRWQVKJ" 

      }], 

    "id" : NumberLong(0), 

    "ns" : "Accounting.Accounts" 

  }, 

  "ok" : 1.0 
}

// query

{

"find" : "Accounts",

"filter" : {

"_id" : ObjectId("5dac846eea29403cb83f99bb")

"projection" : {

"_id" : 1,

"AccountName" : 1

"$db" : "Accounting",

"lsid" : {

"id" : CSUUID("cf5d5a88-2598-4110-9295-6417d1a5f7dd")

}

// returned document

{

"cursor" : {

"firstBatch" : [{

"_id" : ObjectId("5dac846eea29403cb83f99bb"),

"AccountName" : "CBBRRWQVKJ"

}],

"id" : NumberLong(0),

"ns" : "Accounting.Accounts"

"ok" : 1.0

}

Source code for this post can be found here

.NET BLOG

.NET, C# blog

ASP.NET Core – adding controllers directly from integration tests

1. Introduction

2. Accessing ApplicationPartManager from integration tests

3. Handling private controllers

Running ASP.NET Core together with Nancy

1. Introduction

2. Legacy API

3. Combining Nancy with ASP.NET Core

Introduce substitute refactoring comes to NSubstitute.Analyzers

.NET Core – missing currency symbol in docker alpine image

MongoDB.Driver – class-based server side projection

1. Introduction

2.Exploring existing options

3. Class-based server-side projection