{"id":1267,"date":"2017-12-13T23:57:20","date_gmt":"2017-12-13T21:57:20","guid":{"rendered":"http:\/\/tpodolak.com\/blog\/?p=1267"},"modified":"2017-12-13T23:57:20","modified_gmt":"2017-12-13T21:57:20","slug":"asp-net-core-memorycache-getorcreate-calls-factory-method-multiple-times","status":"publish","type":"post","link":"https:\/\/tpodolak.com\/blog\/2017\/12\/13\/asp-net-core-memorycache-getorcreate-calls-factory-method-multiple-times\/","title":{"rendered":"ASP.NET Core MemoryCache &#8211; GetOrCreate calls factory method multiple times"},"content":{"rendered":"<p>Recently I&#8217;ve been trying to locate a performance issue in our application. Stress tests have shown an excessive usage of memory combined with too many external server requests. As usual in such cases, I&#8217;ve run the profiler and after a bit of searching, I&#8217;ve noticed that issue is connected with our caching mechanism. The problematic part was a factory delegate (responsible for populating cache, if the cache for given key is empty) which was called multiple times when concurrent requests were sent. That was a bit surprising to me as we basically were using a very na\u00efve wrapper over <i>ASP.NET Core MemoryCache<\/i>. <\/p>\n<pre lang=\"csharp\">\r\npublic class CacheService : ICacheService\r\n{\r\n    private readonly IMemoryCache _memoryCache;\r\n\r\n    public CacheService(IMemoryCache memoryCache)\r\n    {\r\n        _memoryCache = memoryCache ?? throw new ArgumentNullException(nameof(memoryCache));\r\n    }\r\n\r\n    public T GetOrAdd<T>(string cacheKey, Func<T> factory, DateTime absoluteExpiration)\r\n    {\r\n        return _memoryCache.GetOrCreate(cacheKey, entry =>\r\n        {\r\n            entry.AbsoluteExpiration = absoluteExpiration;\r\n            return factory();\r\n        });\r\n    }\r\n}\r\n<\/pre>\n<p>However simple test proved that this was an actual issue<\/p>\n<pre lang=\"csharp\">\r\n[Fact]\r\npublic void GetOrAdd_CallsFactoryMethodOnce()\r\n{\r\n    var factoryMock = Substitute.For<Func<string>>();\r\n    var optionsMock = Substitute.For<IOptions<MemoryCacheOptions>>();\r\n    optionsMock.Value.Returns(callInfo => new MemoryCacheOptions());\r\n    var memoryCache = new MemoryCache(optionsMock);\r\n\r\n    var subject = new CacheService(memoryCache);\r\n\r\n    var threads = Enumerable.Range(0, 10).Select(_ => new Thread(() => subject.GetOrAdd(\"key\", factoryMock, DateTime.MaxValue))).ToList();\r\n            \r\n    threads.ForEach(thread => thread.Start());\r\n    threads.ForEach(thread => thread.Join());\r\n\r\n    factoryMock.Received(1)();\r\n}\r\n<\/pre>\n<p><a href=\"\/\/tpodolak.com\/blog\/wp-content\/uploads\/2017\/12\/asp-net-core-memorycache-getorcreate-calls-factory-method-multiple-times\/FailedTest.png\"><img loading=\"lazy\" decoding=\"async\" src=\"\/\/tpodolak.com\/blog\/wp-content\/uploads\/2017\/12\/asp-net-core-memorycache-getorcreate-calls-factory-method-multiple-times\/FailedTest.png\" alt=\"\" width=\"1174\" height=\"602\" class=\"aligncenter size-full wp-image-1268\" srcset=\"https:\/\/tpodolak.com\/blog\/wp-content\/uploads\/2017\/12\/asp-net-core-memorycache-getorcreate-calls-factory-method-multiple-times\/FailedTest.png 1174w, https:\/\/tpodolak.com\/blog\/wp-content\/uploads\/2017\/12\/asp-net-core-memorycache-getorcreate-calls-factory-method-multiple-times\/FailedTest-150x77.png 150w, https:\/\/tpodolak.com\/blog\/wp-content\/uploads\/2017\/12\/asp-net-core-memorycache-getorcreate-calls-factory-method-multiple-times\/FailedTest-300x154.png 300w, https:\/\/tpodolak.com\/blog\/wp-content\/uploads\/2017\/12\/asp-net-core-memorycache-getorcreate-calls-factory-method-multiple-times\/FailedTest-1024x525.png 1024w\" sizes=\"auto, (max-width: 1174px) 100vw, 1174px\" \/><\/a><br \/>\nAfter doing a bit of a digging it turned out that this was by-design behavior, which is documented in the very end of official <a href=\"ttps:\/\/docs.microsoft.com\/en-us\/aspnet\/core\/performance\/caching\/memory\">documentation <\/a>. However, as this behavior wasn&#8217;t the one we were expecting, we decided to reduce amount of factory calls by applying custom logic. We had couple of ideas when it comes to implementation, including:<\/p>\n<ul>\n<li>Exclusive lock over factory method<\/li>\n<li>Lock per key<\/li>\n<li>Lock per type<\/li>\n<\/ul>\n<p>After discussion we decided to go with third option, mainly because we didn&#8217;t want to over-complicate the design, plus we wanted to allow concurrent factory method calls for different types of objects. Once <i>CacheService<\/i> was updated with following code<\/p>\n<pre lang=\"csharp\">\r\npublic class CacheService: ICacheService\r\n{\r\n    private readonly IMemoryCache _memoryCache;\r\n\r\n    public CacheService(IMemoryCache memoryCache)\r\n    {\r\n        _memoryCache = memoryCache ?? throw new ArgumentNullException(nameof(memoryCache));\r\n    }\r\n\r\n    public T GetOrAdd<T>(string cacheKey, Func<T> factory, DateTime absoluteExpiration)\r\n    {\r\n        \/\/ locks get and set internally\r\n        if (_memoryCache.TryGetValue<T>(cacheKey, out var result))\r\n        {\r\n            return result;\r\n        }\r\n\r\n        lock (TypeLock<T>.Lock)\r\n        {\r\n            if (_memoryCache.TryGetValue(cacheKey, out result))\r\n            {\r\n                return result;\r\n            }\r\n\r\n            result = factory();\r\n            _memoryCache.Set(cacheKey, result, absoluteExpiration);\r\n\r\n            return result;\r\n         }\r\n    }    \r\n\r\n    private static class TypeLock<T>\r\n    {\r\n        public static object Lock { get; } = new object();\r\n    }\r\n}\r\n<\/pre>\n<p>we stoped experiencing excessive memory usage.<\/p>\n<p>Source code for this post can be found <a href=\"https:\/\/github.com\/tpodolak\/Blog\/tree\/master\/AspNetCoreMemoryCacheIsPopulatedMultipleTimes\">here<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Recently I&#8217;ve been trying to locate a performance issue in our application. Stress tests have shown an excessive usage of memory combined with too many external server requests. As usual in such cases, I&#8217;ve run the profiler and after a bit of searching, I&#8217;ve noticed that issue is connected with our caching mechanism. The problematic [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[317],"tags":[318],"class_list":["post-1267","post","type-post","status-publish","format-standard","hentry","category-asp-net-core","tag-asp-net-core"],"_links":{"self":[{"href":"https:\/\/tpodolak.com\/blog\/wp-json\/wp\/v2\/posts\/1267","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/tpodolak.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/tpodolak.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/tpodolak.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/tpodolak.com\/blog\/wp-json\/wp\/v2\/comments?post=1267"}],"version-history":[{"count":3,"href":"https:\/\/tpodolak.com\/blog\/wp-json\/wp\/v2\/posts\/1267\/revisions"}],"predecessor-version":[{"id":1272,"href":"https:\/\/tpodolak.com\/blog\/wp-json\/wp\/v2\/posts\/1267\/revisions\/1272"}],"wp:attachment":[{"href":"https:\/\/tpodolak.com\/blog\/wp-json\/wp\/v2\/media?parent=1267"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/tpodolak.com\/blog\/wp-json\/wp\/v2\/categories?post=1267"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/tpodolak.com\/blog\/wp-json\/wp\/v2\/tags?post=1267"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}