I had an interesting customer problem today where Kerberos was being attempted for a service principal name (SPN) which simply didn’t exist in Active Directory. This was causing the applications (Exchange) involved to fail as they couldn’t authenticate to one another. The client machine involved was logging numerous errors similar to the following indicating that it was presenting a service ticket encrypted by another machine to the server in question.
Log Name: System
Date: 12/6/2010 2:03:11 PM
Event ID: 4
The Kerberos client received a KRB_AP_ERR_MODIFIED error from the server server01$. The target name used was HTTP/webmail.customer.com. This indicates that the target server failed to decrypt the ticket provided by the client. This can occur when the target server principal name (SPN) is registered on an account other than the account the target service is using. Please ensure that the target SPN is registered on, and only registered on, the account used by the server. This error can also happen when the target service is using a different password for the target service account than what the Kerberos Key Distribution Center (KDC) has for the target service account. Please ensure that the service on the server and the KDC are both updated to use the current password. If the server name is not fully qualified, and the target domain (CUSTOMER.COM) is different from the client domain (CUSTOMER.COM), check if there are identically named server accounts in these two domains, or use the fully-qualified name to identify the server.
The tricky part about this was that Kerberos shouldn’t have been being used at all but instead NTLM. With SPNEGO enabled on the IIS virtual directory, once Kerberos is deemed possible, we can’t fall back to NTLM. My first step was to search Active Directory for an object which had the http/webmail.customer.com SPN adfind:
adfind -f "servicePrincipalName=http/webmail.customer.com" -gcb
I came up empty, though. This makes things even more interesting since there’s no way Kerberos could be attempted if there’s no user or machine account in the directory with a matching SPN. My next step was to do a search for any accounts which had the HOST/webmail.customer.com SPN registered:
adfind -f "servicePrincipalName=HOST/webmail.customer.com" -gcb
This time I came up with a computer account called WEBMAIL. The server had long since been retired, but, the orphaned computer account was still in AD. The root cause of the issue here is a function in Active Directory which matches the HOST service component to several dozen other services. There are numerous services (like RPC or NetLogon) which every single Windows machine has, and it would be a huge waste of space to store SPNs for those services on every computer account in the directory. To mitigate this, Active Directory has an attribute called spnMappings on the CN=Directory Service,CN=Windows NT,CN=Services,CN=Configuration,DC=… object in the Configuration NC. One of the services in this list is “HTTP” which was causing the KDC to issue a service ticket encrypted with this old machine account’s secret for http/webmail.customer.com.
Each time the KDC receives a request for a service ticket, it first does an explicit search to see if there is an account with the requested SPN in the directory. If there are no matches, the KDC then checks to see if the service component (HTTP in this case) is listed in the spnMappings attribute. If this is the case, then a search of the directory is performed to find a user or computer matching HOST/some-name.domain.com. If there’s a match, then a service ticket is issued. Below is a copy of a slide from a presentation I gave at TEC this year which summarizes this functionality.