Monday, August 20, 2012

The case of the gethostbyname() exception

While analyzing a malicious bot in OllyDbg (1.10) on my Windows XP SP3 Virtual Machine, I came across an odd exception (0x000006B0) which always occured trying to step over the Windows API function "gethostbyname()". Every time OllyDbg ended up in kernel32.dll after calling RtlRaiseException() (ntdll.dll). Because a search on Google doesn't gave me any answers I decided to find the cause on my own and hopefully solve the problem.

Figure 1: gethostbyname() exception

At first I thought it has something to do with the network configuration of my Virtual Machine, so I changed it from "NAT" to "Bridged Networking". Again the same exception occured. Next I thought it maybe was caused by OllyDbg version 1.10. Thus I downloaded the brand new OllyDbg 2.01 beta 2 and tried again to step over the function, but again without any luck. The following idea was that maybe the virtaul machine (VirtualBox) somehow causes the exception. Therefore I tried it on my real Windows XP SP3 system on my old Notebook, but again the same error. Another idea was it had something to do with the bot sample I was analyzing, so I downloaded a simple host resolver tool which imported the function gethostbyname(). Again no success, the same exception was thrown. My last idea was to try it on my Windows 7 VM (x86). Finally it worked and no exception occured!

So I wanted to know why this exception only occured on all of my Windows XP SP3 systems and started to debug gethostbyname(). My idea was to manually track down to the last function call which throws the exception and thus maybe understand the problem. So after an hour of walking through the endless subroutines and Windows API functions of gethostbyname() I finally found the cause...

The call chain of gethostbyname() exception (0x000006B0) respectively the ZwConnectPort() error is a follows:
CALL gethostbyname (sample.exe)
    |-- CALL 71A15578 (ws2_32.dll)
    |-- CALL WSALookupServiceBeginA
    |-- CALL WSALookupServiceBeginW
    |-- CALL 71A136A6
    |-- CALL 71A12E71
    |-- CALL 71A14DAC
    |-- CALL 71A14E27
    |-- CALL EAX (mswsock.NSPStartup)
        |-- CALL 719BC787 (mswsock.dll)
        |-- CALL 719BA031
        |-- CALL 719BA20E
            |-- CALL 76EE7D9C (dnsapi.dll)
            |-- CALL 76EE7DF6
            |-- CALL 76EE7E51
            |-- CALL <JMP.&RPCRT4.NdrClientCall2>
                |-- CALL NdrGetBuffer (RPCRT4.dll)
                |-- CALL I_RpcGetBuffer
                |-- CALL I_RpcGetBufferWithObject
                |-- CALL DWORD PTR DS:[EAX+34]
                |-- CALL 77E5BAF2
                |-- CALL 77E5BBC8
                |-- CALL 77E68795
                |-- CALL 77E68DFA
                |-- CALL DWORD PTR DS:[<&ntdll.NtConnectPort>] (ntdll.ZwConnectPort) -> Exception: C0000034 (STATUS_OBJECT_NAME_NOT_FOUND)

As you can see the call to ZwConnectPort failed with the error "STATUS_OBJECT_NAME_NOT_FOUND". Obviously that system object doesn't exist, so the call to ZwConnectPort failed and all the subsequent code also (-> Exception: 0x000006B0). By looking some lines above I saw a couple of wcslen() function calls with "\RPC Control\" as parameter and on stack I saw the string "\RPC Control\DNSResolver". This indicated that the object "DNSResolver" in the folder "RPC Control" was the missing object. I started WinObjEx and looked at the folder "RPC Control" and there wasn't any object named "DNSResolver". A quick cross check on my Windows 7 VM showed me the DNSResolver object was there. After a search on Google I found the corresponding program which creates this object. It is the windows Service "DNS-Client" (DNS Resolver Cache Service) which on all of my Windows XP systems was set to "Demand" and the service wasn't running. But why? What immediately came to my mind is in the past I always "hardened" the Windows Services on my XP systems with a Batch script (
After looking through the code of the Batch file I found the line responsible for setting the service start on demand:

config DNSCache start= demand

So after manually starting the service and again trying to debug gethostbyname(), the call (step over) worked correctly and no exception occured! The case was solved.

But wait, there was something I didn't understand. Why did the host resolver tool anyway completed its task and get the correct IP address?

Figure 2: Resolved host of

After another search in Google I found the answer. It is the way Windows tries to resolve a host name and the sequence of tasks it takes (

1) Local host name
Host name resolution begins when a user uses a Windows Sockets application and specifies the host name assigned to the destination host. Windows checks whether the host name matches the local host name. If the host name is the same as the local host name, the host name is resolved to an IP address that is assigned to the local host, and the name resolution process stops.
2) DNS client resolver cache (+Hosts file)
If the host name is not the same as the local host name, Windows searches the DNS client resolver cache for an entry containing the host name. The DNS client resolver cache is a RAM-based table that contains both the entries in the Hosts file and the host names that Windows has tried to resolve through DNS.
3) DNS server
When a user uses a Windows Sockets application and specifies an FQDN for the destination host and the FQDN does not match the local host name or any entries in the DNS client resolver cache, the DNS client component of TCP/IP for Windows XP and Windows Server 2003 constructs and sends a DNS Name Query Request message to the DNS server. The DNS server determines whether a mapping for the name to an IP address is stored either locally or on another DNS server. Whether or not a mapping is found, the DNS server sends back a DNS Name Query Response message to the DNS client.

This made complete sense, since the host name wasn't the local hosts name and the DNS client resolver cache service wasn't running. The remaining alternative (because NetBIOS was also disabled) was a DNS Query Request message to the DNS server (Router). The following screenshot shows the request made by gethostbyname():

Figure 3: DNS Name Request and Response

So we learned if the Windows Service "DNS-Client" isn't running, Windows can resolve the host by various other ways. But be warned if you debug an application and want to step over the function gethostbyname(), it will end up in an odd exception. ;-)

1 comment: