ASP.NET Core MemoryCache – GetOrCreate calls factory method multiple times

2017-12-13 TPodolakASP.NET Core

Recently I’ve been trying to locate a performance issue in our application. Stress tests have shown an excessive usage of memory combined with too many external server requests. As usual in such cases, I’ve run the profiler and after a bit of searching, I’ve noticed that issue is connected with our caching mechanism. The problematic part was a factory delegate (responsible for populating cache, if the cache for given key is empty) which was called multiple times when concurrent requests were sent. That was a bit surprising to me as we basically were using a very naïve wrapper over ASP.NET Core MemoryCache.

public class CacheService : ICacheService
{
    private readonly IMemoryCache _memoryCache;

    public CacheService(IMemoryCache memoryCache)
    {
        _memoryCache = memoryCache ?? throw new ArgumentNullException(nameof(memoryCache));
    }

    public T GetOrAdd<T>(string cacheKey, Func<T> factory, DateTime absoluteExpiration)
    {
        return _memoryCache.GetOrCreate(cacheKey, entry =>
        {
            entry.AbsoluteExpiration = absoluteExpiration;
            return factory();
        });
    }
}

public class CacheService : ICacheService

{

private readonly IMemoryCache _memoryCache;

public CacheService(IMemoryCache memoryCache)

{

_memoryCache = memoryCache ?? throw new ArgumentNullException(nameof(memoryCache));

}

public T GetOrAdd<T>(string cacheKey, Func<T> factory, DateTime absoluteExpiration)

{

return _memoryCache.GetOrCreate(cacheKey, entry =>

{

entry.AbsoluteExpiration = absoluteExpiration;

return factory();

});

}

However simple test proved that this was an actual issue

[Fact]
public void GetOrAdd_CallsFactoryMethodOnce()
{
    var factoryMock = Substitute.For<Func<string>>();
    var optionsMock = Substitute.For<IOptions<MemoryCacheOptions>>();
    optionsMock.Value.Returns(callInfo => new MemoryCacheOptions());
    var memoryCache = new MemoryCache(optionsMock);

    var subject = new CacheService(memoryCache);

    var threads = Enumerable.Range(0, 10).Select(_ => new Thread(() => subject.GetOrAdd("key", factoryMock, DateTime.MaxValue))).ToList();
            
    threads.ForEach(thread => thread.Start());
    threads.ForEach(thread => thread.Join());

    factoryMock.Received(1)();
}

[Fact]

public void GetOrAdd_CallsFactoryMethodOnce()

{

var factoryMock = Substitute.For<Func<string>>();

var optionsMock = Substitute.For<IOptions<MemoryCacheOptions>>();

optionsMock.Value.Returns(callInfo => new MemoryCacheOptions());

var memoryCache = new MemoryCache(optionsMock);

var subject = new CacheService(memoryCache);

var threads = Enumerable.Range(0, 10).Select(_ => new Thread(() => subject.GetOrAdd("key", factoryMock, DateTime.MaxValue))).ToList();

threads.ForEach(thread => thread.Start());

threads.ForEach(thread => thread.Join());

factoryMock.Received(1)();

}

After doing a bit of a digging it turned out that this was by-design behavior, which is documented in the very end of official documentation . However, as this behavior wasn’t the one we were expecting, we decided to reduce amount of factory calls by applying custom logic. We had couple of ideas when it comes to implementation, including:

Exclusive lock over factory method
Lock per key
Lock per type

After discussion we decided to go with third option, mainly because we didn’t want to over-complicate the design, plus we wanted to allow concurrent factory method calls for different types of objects. Once CacheService was updated with following code

public class CacheService: ICacheService
{
    private readonly IMemoryCache _memoryCache;

    public CacheService(IMemoryCache memoryCache)
    {
        _memoryCache = memoryCache ?? throw new ArgumentNullException(nameof(memoryCache));
    }

    public T GetOrAdd<T>(string cacheKey, Func<T> factory, DateTime absoluteExpiration)
    {
        // locks get and set internally
        if (_memoryCache.TryGetValue<T>(cacheKey, out var result))
        {
            return result;
        }

        lock (TypeLock<T>.Lock)
        {
            if (_memoryCache.TryGetValue(cacheKey, out result))
            {
                return result;
            }

            result = factory();
            _memoryCache.Set(cacheKey, result, absoluteExpiration);

            return result;
         }
    }    

    private static class TypeLock<T>
    {
        public static object Lock { get; } = new object();
    }
}

public class CacheService: ICacheService

{

private readonly IMemoryCache _memoryCache;

public CacheService(IMemoryCache memoryCache)

{

_memoryCache = memoryCache ?? throw new ArgumentNullException(nameof(memoryCache));

}

public T GetOrAdd<T>(string cacheKey, Func<T> factory, DateTime absoluteExpiration)

{

// locks get and set internally

if (_memoryCache.TryGetValue<T>(cacheKey, out var result))

{

return result;

}

lock (TypeLock<T>.Lock)

{

if (_memoryCache.TryGetValue(cacheKey, out result))

{

return result;

}

result = factory();

_memoryCache.Set(cacheKey, result, absoluteExpiration);

return result;

}

private static class TypeLock<T>

{

public static object Lock { get; } = new object();

}

we stoped experiencing excessive memory usage.

Source code for this post can be found here

Finding typos with ReSharper Command Line Tools, ReSpeller and Cake

2017-10-27 TPodolakcake, Resharper, ReSpeller

1. Introduction

I tend to do a lot of typos when I write a code and I mean a lot. This is quite annoying for me so I decided to somehow automate the process of finding the spelling errors during the build. My first thought was to use some kind of Roslyn analyzer, however, I failed to find any working one. This is why I decided to give a try to ReSharper Command Line Tools (also known as CLIT) combined with ReSpeller plugin. For those who don’t know, ReSharper Command Line Tools is set of tools which allow you to use ReSharper inspections without Visual Studio

2. First attempt

This was supposed to be a rather straightforward task because according to documentation command line inspections support extensions out of the box. The example presented in the help page was supposed to be the perfect one for me

InspectCode.exe --project=Documents -o="OutputFile.xml" --no-swea -x=EtherealCode.ReSpeller "SolutionFile.sln"

1	InspectCode.exe --project=Documents -o="OutputFile.xml" --no-swea -x=EtherealCode.ReSpeller "SolutionFile.sln"

Unfortunately, it turned out that extensions support is broken for some time and it wasn’t working for me either (tested on Command Line Tools version 2017.2).

3. The workaround

Even though extension support is not working as expected I was able to make it work with some workaround. The workaround itself is a combination of helpful tips taken from this YouTrack issue. Long story short, we have to manually install the plugin and then force ReSharper to use it. Assuming that you have command line tools installed this can be achieved in following way. First of all we have to get our hands on ReSpeller plugin. We can install it using NuGet but before you do it, remember to add additional feed to nuget.config file.

nuget sources add -name JetBrains-Feed -source https://resharper-plugins.jetbrains.com/api/v2/curated-feeds/Wave_v9.0/

1	nuget sources add -name JetBrains-Feed -source https://resharper-plugins.jetbrains.com/api/v2/curated-feeds/Wave_v9.0/

The feed url can be found in the ReSharper’s option page and for version 2017.2 it is

Having our NuGet sources updated we can retrieve the package using following command

nuget install "EtherealCode.ReSpeller" -Version "4.6.9.2"

1	nuget install "EtherealCode.ReSpeller" -Version "4.6.9.2"

The next step is to copy ReSpeller’s DLLs from following folders

etherealcode.respeller.4.6.9.2\EtherealCode.ReSpeller\lib\net45\
etherealcode.respeller.4.6.9.2\Extended.Wpf.Toolkit\lib\net40\
etherealcode.respeller.4.6.9.2\NHunspell\lib\net\

etherealcode.respeller.4.6.9.2\EtherealCode.ReSpeller\lib\net45\

etherealcode.respeller.4.6.9.2\Extended.Wpf.Toolkit\lib\net40\

etherealcode.respeller.4.6.9.2\NHunspell\lib\net\

directly to CLIT tools folder – JetBrains.ReSharper.CommandLineTools\tools. What is more, ReSpeller comes together with dictionary files which also have to be copied to the tools folder. This time, however, we have to copy entire etherealcode.respeller.4.6.9.2\EtherealCode.ReSpeller\lib\net45\dic directory, not only its content. The very last thing is to make ReSharper aware of the plugin, in order to do that we have to find EtherealCode.ReSpeller.JetMetadata.sstg file. The file itself doesn’t come together with the package but it is generated by ReSharper during plugin installation. So the easiest way of getting that file is to install ReSpeller as a ReSharper plugin in Visual Studio and then copy the sstg file to tools folder. After all that work now if you run

InspectCode.exe --project=Documents -o="OutputFile.xml" --no-swea -x=EtherealCode.ReSpeller "SolutionFile.sln"

1	InspectCode.exe --project=Documents -o="OutputFile.xml" --no-swea -x=EtherealCode.ReSpeller "SolutionFile.sln"

(note lack of -x switch) you will see spellcheck inspections in output file.

4. Cake integration

Doing all that stuff manually is error-prone and time-consuming and of course, I wanted to have the spellcheck up and running on the CI server, that is why I’ve also automated all above steps with Cake build script. The core concept is to use ReSpeller and ReSharper Command Line Tools as so-called “Cake tools” to run the inspections and Cake.Issues and Cake.Issues.InspectCode plugins for analyzing the output file.
Adding tools and add-ins is straightforward we just have to add following lines on top of the build file.

#tool "nuget:?package=EtherealCode.ReSpeller&version=4.6.9.2"
#tool "nuget:?package=JetBrains.ReSharper.CommandLineTools&version=2017.2.2"
#addin "Cake.Issues"
#addin "Cake.Issues.InspectCode"

#tool "nuget:?package=EtherealCode.ReSpeller&version=4.6.9.2"

#tool "nuget:?package=JetBrains.ReSharper.CommandLineTools&version=2017.2.2"

#addin "Cake.Issues"

#addin "Cake.Issues.InspectCode"

We also have to add nuget.config file to our build directory in order to point Cake to ReSharper feed (you could also modify the global nuget.config but I don’t want a build process to change the behavior of entire system)

  <configuration>
    <packageSources>
      <add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
      <add key="resharper-plugins" value="https://resharper-plugins.jetbrains.com/api/v2/curated-feeds/Wave_v9.0/" protocolVersion="2" />
    </packageSources>
</configuration>

</packageSources>

</configuration>

Before calling InspectCode.exe we have to make sure that all of the files and folders exist

var destinationPath = "tools/jetbrains.resharper.commandlinetools.2017.2.2/JetBrains.ReSharper.CommandLineTools/tools";
Setup(context =>
{
    if(!DirectoryExists(".artifacts"))
    {
        CreateDirectory(".artifacts");
    }

    if(!DirectoryExists(destinationPath + "/dic"))
    {
        CreateDirectory(destinationPath + "/dic");
    }
});
	
Task("Prepare-Respeller")
.Does(()=>
{
    // works for pinned version of ReSpeller
    var destinationPath = "tools/jetbrains.resharper.commandlinetools.2017.2.2/JetBrains.ReSharper.CommandLineTools/tools";
    CopyFiles("tools/etherealcode.respeller.4.6.9.2/EtherealCode.ReSpeller/lib/net45/*.dll", destinationPath);
    CopyFiles("tools/etherealcode.respeller.4.6.9.2/Extended.Wpf.Toolkit/lib/net40/*.dll", destinationPath);
    CopyFiles("tools/etherealcode.respeller.4.6.9.2/NHunspell/lib/net/*.dll", destinationPath);
    CopyFiles("../Spelling/*.dic",  destinationPath + "/dic");
    CopyFiles("../Spelling/*.aff",  destinationPath + "/dic"); // using shared dictionaries not the ReSpeller ones
    CopyFiles("EtherealCode.ReSpeller.JetMetadata.sstg", destinationPath);
});

var destinationPath = "tools/jetbrains.resharper.commandlinetools.2017.2.2/JetBrains.ReSharper.CommandLineTools/tools";

Setup(context =>

{

if(!DirectoryExists(".artifacts"))

{

CreateDirectory(".artifacts");

}

if(!DirectoryExists(destinationPath + "/dic"))

{

CreateDirectory(destinationPath + "/dic");

}

});

Task("Prepare-Respeller")

.Does(()=>

{

// works for pinned version of ReSpeller

var destinationPath = "tools/jetbrains.resharper.commandlinetools.2017.2.2/JetBrains.ReSharper.CommandLineTools/tools";

CopyFiles("tools/etherealcode.respeller.4.6.9.2/EtherealCode.ReSpeller/lib/net45/*.dll", destinationPath);

CopyFiles("tools/etherealcode.respeller.4.6.9.2/Extended.Wpf.Toolkit/lib/net40/*.dll", destinationPath);

CopyFiles("tools/etherealcode.respeller.4.6.9.2/NHunspell/lib/net/*.dll", destinationPath);

CopyFiles("../Spelling/*.dic", destinationPath + "/dic");

CopyFiles("../Spelling/*.aff", destinationPath + "/dic"); // using shared dictionaries not the ReSpeller ones

CopyFiles("EtherealCode.ReSpeller.JetMetadata.sstg", destinationPath);

});

Having all of the items in place now we are ready to run the actual spellcheck and verify the results

Task("Run-SpellCheck")
.IsDependentOn("Prepare-Respeller")
.Does(() =>
{
    var processArgumentBuilder = new ProcessArgumentBuilder().AppendSwitch("--profile","=","../Spelling/Resharper.ReSpeller.DotSettings")
                                          .AppendSwitch("--caches-home","=","tools")
                                          .AppendSwitch("-o","=","../.artifacts/inspectcode.xml")
                                          .Append("../ResharperCommandLineInspections.sln");

    var processSettings = new ProcessSettings 
    {
        Arguments = processArgumentBuilder,
        Silent = true,
        RedirectStandardOutput = true
    };

    IEnumerable<string> redirectedOutput, redirectedErrors;

    var exitCode = StartProcess(Context.Tools.Resolve("inspectcode.exe"), processSettings, out redirectedOutput, out redirectedErrors);
    if(exitCode !=0)
    {
        throw new CakeException($"InspectCode exited with unexpected error code: {exitCode}");
    }

    var issues = ReadIssues(InspectCodeIssuesFromFilePath("../.artifacts/inspectcode.xml"), "../");
    var spellingIssues = issues.Where(issue => issue.Rule.Contains("Typo")).ToList();

    if(spellingIssues.Any())
    {
        var errorMessage = spellingIssues.Aggregate(new StringBuilder(), (stringBuilder, issue) => stringBuilder.AppendFormat("FileName: {0} Line: {1} Message: {2}{3}", issue.AffectedFileRelativePath, issue.Line, issue.Message, Environment.NewLine));
        throw new CakeException($"{spellingIssues.Count} spelling errors detected: {Environment.NewLine}{errorMessage}please fix them or add missing words to the dictionary.");
    }
});

Task("Run-SpellCheck")

.IsDependentOn("Prepare-Respeller")

.Does(() =>

{

var processArgumentBuilder = new ProcessArgumentBuilder().AppendSwitch("--profile","=","../Spelling/Resharper.ReSpeller.DotSettings")

.AppendSwitch("--caches-home","=","tools")

.AppendSwitch("-o","=","../.artifacts/inspectcode.xml")

.Append("../ResharperCommandLineInspections.sln");

var processSettings = new ProcessSettings

{

Arguments = processArgumentBuilder,

Silent = true,

RedirectStandardOutput = true

};

IEnumerable<string> redirectedOutput, redirectedErrors;

var exitCode = StartProcess(Context.Tools.Resolve("inspectcode.exe"), processSettings, out redirectedOutput, out redirectedErrors);

if(exitCode !=0)

{

throw new CakeException($"InspectCode exited with unexpected error code: {exitCode}");

}

var issues = ReadIssues(InspectCodeIssuesFromFilePath("../.artifacts/inspectcode.xml"), "../");

var spellingIssues = issues.Where(issue => issue.Rule.Contains("Typo")).ToList();

if(spellingIssues.Any())

{

var errorMessage = spellingIssues.Aggregate(new StringBuilder(), (stringBuilder, issue) => stringBuilder.AppendFormat("FileName: {0} Line: {1} Message: {2}{3}", issue.AffectedFileRelativePath, issue.Line, issue.Message, Environment.NewLine));

throw new CakeException($"{spellingIssues.Count} spelling errors detected: {Environment.NewLine}{errorMessage}please fix them or add missing words to the dictionary.");

}

});

As you can see I start InspectCode.exe process with a bunch of parameters. Once the process returns successful exit code I can go ahead and process the results thanks to ReadIssues and InspectCodeIssuesFromFilePath methods provided by the add-ins. Because of the fact that ReSharper can produce a ton of inspections I only care about the ones which are typos. Simple and very naïve LINQ query can do it for me

var spellingIssues = issues.Where(issue => issue.Rule.Contains("Typo")).ToList();

1	var spellingIssues = issues.Where(issue => issue.Rule.Contains("Typo")).ToList();

If any typos were found I just throw exception with a listing of all the issues. Note that I aggregate all the issues into one big message because if you have lots of typos (just like I do) rendering them can take some time

The source code for slightly enhanced version of that script can be found here

ASP.NET Core 1.x – curios case of missing IOptions<T> array item

2017-10-172017-10-17 TPodolakASP.NET Core

1. Introduction

Last week while I was working on one of the features for my current project, I noticed quite surprising behavior of IOptions<T> object. Long story short, for some reasons there were missing array elements in my parsed application settings class.

2. Problem

Just like everyone else, we store our application settings in JSON file. The settings are pretty straightforward, just a bunch of properties with some nested objects

{
    "PaymentMethods": [
        {
            "Name": "MasterCard",
            "ExcludedCurrencies": []
        },
        {
            "Name": "Visa",
            "ExcludedCurrencies": null
        },
        {
            "Name": "Apple Pay"
        },
        {
            "Name": "PayPal",
            "ExcludedCurrencies": [
                "MAD"
            ]
        }
    ]
}

{

"PaymentMethods": [

{

"Name": "MasterCard",

"ExcludedCurrencies": []

{

"Name": "Visa",

"ExcludedCurrencies": null

{

"Name": "Apple Pay"

{

"Name": "PayPal",

"ExcludedCurrencies": [

"MAD"

]

}

]

}

which are bound to the AppSetting class thanks to these couple of lines of code

// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{            
    services.Configure<AppSettings>(Configuration);
    // Add framework services.
    services.AddMvc();
}

// This method gets called by the runtime. Use this method to add services to the container.

public void ConfigureServices(IServiceCollection services)

{

services.Configure<AppSettings>(Configuration);

// Add framework services.

services.AddMvc();

}

where AppSettings class looks as follows

public class AppSettings
{
    public List<PaymentMethod> PaymentMethods { get; set; }
}

public class PaymentMethod
{
    public string Name { get; set; }

    public List<string> ExcludedCurrencies { get; set; }
}

public class AppSettings

{

public List<PaymentMethod> PaymentMethods { get; set; }

}

public class PaymentMethod

{

public string Name { get; set; }

public List<string> ExcludedCurrencies { get; set; }

}

As you can see nothing too fancy in here, standard ASP.NET Core approach for handling configuration files. However, if you access the PaymentMethods property of IOptions<AppSettings>

[Route("api/[controller]")]
public class ValuesController : Controller
{
    private readonly IOptions<AppSettings> _appSettings;

    public ValuesController(IOptions<AppSettings> appSettings)
    {
        _appSettings = appSettings;
    }
        
    // GET api/values
    [HttpGet]
    public List<PaymentMethod> Get()
    {
        return _appSettings.Value.PaymentMethods;
    }
}

[Route("api/[controller]")]

public class ValuesController : Controller

{

private readonly IOptions<AppSettings> _appSettings;

public ValuesController(IOptions<AppSettings> appSettings)

{

_appSettings = appSettings;

}

// GET api/values

[HttpGet]

public List<PaymentMethod> Get()

{

return _appSettings.Value.PaymentMethods;

}

you will see that instead of four elements we ended up having 3 items

as the “Visa” element is missing.

3. Solution

If you look closely at the appsettings.json file you will notice that a “Visa” item configuration is a bit different than the ones which are parsed just fine. Namely null is assigned to excludedCurrencies property, whereas in other items an empty array or no property at all is used. So the first fix is just to use the same approach as in the other items

{
    "Name": "Visa",
    "ExcludedCurrencies": []
}

{

"Name": "Visa",

"ExcludedCurrencies": []

}

This approach works fine, however, it is not a silver bullet as someone still might use null and introduce a bug. Another fix might be upgrading to ASP.NET Core 2.0 as the problem doesn’t seem to exist in there. Of course, this is rather a huge change so not everyone will be willing to do it. The last option is a manual installation of Microsoft.Extensions.Configuration.Binder 2.0 along with Microsoft.Extensions.Configuration.Json 2.0 package. However, this approach requires you to retarget your application to netcoreapp2.0. One way or another every solution in here works fine, so once you apply the fix you will see all items in the array
Source code for this post can be found here

.NET Core – calculating code coverage with OpenCover (on Windows)

2017-10-122017-10-12 TPodolakOpenCover4 Comments

1. Introduction.

I’ve been struggling for some time to make OpenCover work with .NET Core projects, so I’ve decided to document a working example. This is basically an aggregation of hints and answers I found on the Web (especially in here)

2. Working example.

If you run an OpenCover with the same arguments you were running it for a classic framework, you will end up with something like this
As you can see the coverage was not calculated and OpenCover is complaining about a missing PDBs. In order to fix that we have to do two things. First of all, we have to build the project with a full PDBs

dotnet build "PathToYourSolution.sln" /p:DebugType=Full

1	dotnet build "PathToYourSolution.sln" /p:DebugType=Full

And then run OpenCover with and -oldStyle switch

OpenCover.Console.exe -target:"C:/Program Files/dotnet/dotnet.exe" -targetargs:"test \"PathToTestProject.csproj\" --configuration Debug --no-build" -filter:"+[*]* -[*.Tests*]*" -oldStyle -register:user -output:"OpenCover.xml"

1	OpenCover.Console.exe -target:"C:/Program Files/dotnet/dotnet.exe" -targetargs:"test \"PathToTestProject.csproj\" --configuration Debug --no-build" -filter:"+[] -[.Tests]*" -oldStyle -register:user -output:"OpenCover.xml"

From now on, OpenCover recognizes PDB files and the code coverage is calculated correctly

3. Cake script.

As we use a Cake as our build automation tool I’ve also prepared a sample script for generating a code coverage. The core part of the script looks as follows

// rest of the code omitted for brevity

Task("Build")
    .IsDependentOn("Restore-NuGet-Packages")
    .Does(() =>
{
    DotNetCoreBuild(paths.Files.Solution.ToString(), new DotNetCoreBuildSettings
    {
        Configuration = parameters.Configuration,
        ArgumentCustomization = arg => arg.AppendSwitch("/p:DebugType","=","Full")
    });
});

Task("Run-Tests")
    .IsDependentOn("Build")
    .Does(() =>
{
    var success = true;
    var openCoverSettings = new OpenCoverSettings
    {
        OldStyle = true,
        MergeOutput = true
    }
    .WithFilter("+[*]* -[*.Tests*]*");

    foreach(var project in paths.Files.TestProjects)
    {
        try 
        {
            var projectFile = MakeAbsolute(project).ToString();
            var dotNetTestSettings = new DotNetCoreTestSettings
            {
                Configuration = parameters.Configuration,
                NoBuild = true
            };

            OpenCover(context => context.DotNetCoreTest(projectFile, dotNetTestSettings), paths.Files.TestCoverageOutput, openCoverSettings);
        }
        catch(Exception ex)
        {
            success = false;
            Error("There was an error while running the tests", ex);
        }
    }

    ReportGenerator(paths.Files.TestCoverageOutput, paths.Directories.TestResults);

    if(success == false)
    {
        throw new CakeException("There was an error while running the tests");
    }

});

// rest of the code omitted for brevity

Task("Build")

.IsDependentOn("Restore-NuGet-Packages")

.Does(() =>

{

DotNetCoreBuild(paths.Files.Solution.ToString(), new DotNetCoreBuildSettings

{

Configuration = parameters.Configuration,

ArgumentCustomization = arg => arg.AppendSwitch("/p:DebugType","=","Full")

});

Task("Run-Tests")

.IsDependentOn("Build")

.Does(() =>

{

var success = true;

var openCoverSettings = new OpenCoverSettings

{

OldStyle = true,

MergeOutput = true

}

.WithFilter("+[*]* -[*.Tests*]*");

foreach(var project in paths.Files.TestProjects)

{

try

{

var projectFile = MakeAbsolute(project).ToString();

var dotNetTestSettings = new DotNetCoreTestSettings

{

Configuration = parameters.Configuration,

NoBuild = true

};

OpenCover(context => context.DotNetCoreTest(projectFile, dotNetTestSettings), paths.Files.TestCoverageOutput, openCoverSettings);

}

catch(Exception ex)

{

success = false;

Error("There was an error while running the tests", ex);

}

ReportGenerator(paths.Files.TestCoverageOutput, paths.Directories.TestResults);

if(success == false)

{

throw new CakeException("There was an error while running the tests");

}

});

Notice that script is able to merge code coverage results from multiple test projects thanks to MergeOutput flag passed to OpenCover.

Full sources of that script (along with a dotnet vstest alternative approach) can be found here.

ASP.NET Core – request serialization issues after migration from classic WebApi

2017-08-212017-08-21 TPodolakASP.NET Core2 Comments

We’ve been migrating a classic WebApi application into the ASP.NET Core API for quite a while now, and last week we decided to deploy it on the test environment and try to run a website against it. We were pretty confident about the quality of a migration because all of the integration tests from old API were green when running against a new one. We were expecting maybe some business logic bugs but from our perspective, the infrastructure part was good to go. Unfortunately, when the QA’s run the selenium test suite they quickly found a major issue in our codebase. They basically were getting HTTP 500 errors indicating a NullReferenceException. That was pretty surprising to us and after a bit of a digging we found following lines in the logs

TimeStamp=2017-08-20 00:43:49.4383 TraceId=0HL77AF0N305P Level=DEBUG Message=JSON input formatter threw an exception. Newtonsoft.Json.JsonSerializationException: Error converting value {null} to type 'System.Int32'. Path 'PassengerNumber', line 2, position 26. ---> System.InvalidCastException: Null object cannot be converted to a value type.
   at System.Convert.ChangeType(Object value, Type conversionType, IFormatProvider provider)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.EnsureType(JsonReader reader, Object value, CultureInfo culture, JsonContract contract, Type targetType)
   --- End of inner exception stack trace ---
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.EnsureType(JsonReader reader, Object value, CultureInfo culture, JsonContract contract, Type targetType)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)

TimeStamp=2017-08-20 00:43:49.4383 TraceId=0HL77AF0N305P Level=DEBUG Message=JSON input formatter threw an exception. Newtonsoft.Json.JsonSerializationException: Error converting value {null} to type 'System.Int32'. Path 'PassengerNumber', line 2, position 26. ---> System.InvalidCastException: Null object cannot be converted to a value type.

at System.Convert.ChangeType(Object value, Type conversionType, IFormatProvider provider)

at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.EnsureType(JsonReader reader, Object value, CultureInfo culture, JsonContract contract, Type targetType)

--- End of inner exception stack trace ---

at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.EnsureType(JsonReader reader, Object value, CultureInfo culture, JsonContract contract, Type targetType)

at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)

at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)

at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)

this was resulting in a null request in controller’s action
As you can see JSON serializer was not able to convert null into the int. We double checked the requests coming from front-end and this is how one of them looked like

{
   "PassengerNumber": null,
   "PassengerType": "2"
}

{

"PassengerNumber": null,

"PassengerType": "2"

}

whereas corresponding request class on the server side was written in following way

public class CreatePassengerRequest
{
    public int PassengerNumber { get; set; }
        
    public int PassengerType { get; set; }
}

public class CreatePassengerRequest

{

public int PassengerNumber { get; set; }

public int PassengerType { get; set; }

}

Clearly front-end shouldn’t be sending a null as a PassengerNumber however the same request was serialized just fine when running against classic WebApi
Fortunately the fix for that was pretty straightforward. All we had to do was to configure JSON serializer to ignore null values.

// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
    services.AddMvc().AddJsonOptions(options =>
    {
        options.SerializerSettings.NullValueHandling = NullValueHandling.Ignore;
    });
}

// This method gets called by the runtime. Use this method to add services to the container.

public void ConfigureServices(IServiceCollection services)

{

services.AddMvc().AddJsonOptions(options =>

{

options.SerializerSettings.NullValueHandling = NullValueHandling.Ignore;

});

}

From now on we were able to properly serialize the request
Source code for this post can be found here

.NET BLOG

.NET, C# blog

ASP.NET Core MemoryCache – GetOrCreate calls factory method multiple times

Finding typos with ReSharper Command Line Tools, ReSpeller and Cake

1. Introduction

2. First attempt

3. The workaround

4. Cake integration

ASP.NET Core 1.x – curios case of missing IOptions<T> array item

1. Introduction

2. Problem

3. Solution

.NET Core – calculating code coverage with OpenCover (on Windows)

1. Introduction.

2. Working example.

3. Cake script.

ASP.NET Core – request serialization issues after migration from classic WebApi