The Setup
I was running painfully low on disk space on my C drive and was working with a Visual Studio solution containing 12 projects. A compilation / build got hung up on something, I cancelled the build tried again. That build worked.
So far just business as usual. However, I noticed that while I was successfully authenticating in the application I was getting bounced as not authenticating by several of the projects, though I was listed as authenticated in other projects. After far too much poking around in the code I discovered that most of the OWIN nuget packages where mysteriously gone and this was causing the problem. Strangely the system was not able to authenticate my login but was able to determine I was not authenticated. I got no actual authentication messages during any of this. This happened in four of the 12 projects. I was rebuilding and cleaning the solution many times throughout all of this.
I nuget installed the proper packages in all four of the projects and three of them worked perfectly. The fourth defied all logic and explanation. After much yelling, screaming and gnashing of teeth I was diagnosing with the below code
<code> [AllowAnonymous] public ActionResult DiagnoseAuthentication() { var authDetails = new { TempFileLocation = System.Web.HttpRuntime.CodegenDir, CoookeValue = Request.Cookies[".AspNet.ApplicationCookie"]?.Value ?? "No Cookie",// IsAuthenticated = User.Identity.IsAuthenticated, UserName = User.Identity.Name, AuthenticationType = User.Identity.AuthenticationType, // Extract claims safely without circular references Claims = ((ClaimsIdentity)User.Identity).Claims.Select(c => new { Type = c.Type, Value = c.Value }).ToList(), // Get roles in a safe manner Roles = ((ClaimsIdentity)User.Identity).Claims .Where(c => c.Type == ClaimTypes.Role) .Select(c => c.Value) .ToList(), // Additional environment information Environment = new { MachineName = Environment.MachineName, OSVersion = Environment.OSVersion.ToString(), Is64BitProcess = Environment.Is64BitProcess } }; return Json(authDetails, JsonRequestBehavior.AllowGet); } [AllowAnonymous] public object CompareAuthenticationSettings() { return new { // Machine Key Details MachineKeyValidation = ConfigurationManager.AppSettings["MachineKeyValidation"], // Authentication Cookie Settings CookieName = FormsAuthentication.FormsCookieName, CookiePath = FormsAuthentication.FormsCookiePath, // Application Pool // Identity Configuration IdentityConnectionString = ConfigurationManager.ConnectionStrings["IdentityConnection"]?.ConnectionString, // Security Stamp Validation SecurityStampValidationInterval = GetSecurityStampInterval() }; } private TimeSpan GetSecurityStampInterval() { // Retrieve security stamp validation interval // This is typically configured in Startup.cs or similar return TimeSpan.FromMinutes(30); // Example default }</code>
Which told me that I was simply not logged in, even though the same code listed me as logged into the other projects.
I then started investigating IIS to see if something was amiss there. I had not changed anything in IIS for over a year so this was unlikely, but one never knows.
A quick look around the general settings found nothing amiss, so I made a copy of the application in the main site to experiment with and to my great surprise that exact copy worked perfectly. The original application contined to not work but {ApplicationName}-Test worked just fine.
I then deleted the original IIS site and recreated the entire site, even creating a new app pool. Somehow there was the same error. I was perplexed and was running out of places to look for solutions. I had rebooted, cache deleted, cleaned solutions and rebuilt solutions many, many times over all this, but I decieded to delete as many caches as possible. I did all that, and added in the
TempFileLocation = System.Web.HttpRuntime.CodegenDir,
line above – just to make sure.
Lo and behold, I saw this

Somehow there were two copies of the compiled code in the relevant temp directory. I’ve never actually seen that with a recent version of IIS/Visual Studio. IIS was picking up the older directory which had the problems and never using the repaired directory. Cleaning the solution and rebuilding should have taken care of the older one but in this case it did not.
I guess the moral of the story is, if there is ever a problem with low disk space make sure you delete the caches manually. I’m writing this up as lengthy post so I can find it again