64位windows下32位进程一例卡死分析_ntusermessagecall

1 故障现象

win7 64 os下，360浏览器不定期出现卡死。

2 dmp分析

使用任务管理器给360se进程下dump。链接: https://pan.baidu.com/s/1Hd-T0T6WEcufjkFKDA6MHg?pwd=z4u5 提取码: z4u5。
windbg打开之，提示

For analysis of this file, run !analyze -v
wow64win!NtUserMessageCall+0xa:
00000000`749bfdaa c3              ret
1
2
3

2.1 wow64exts插件转换到32位进程空间

这是一个32位进程的dmp，欲参看32位进程模式下的数据，须执行
!wow64exts.sw
其实，如果是64位os下给32位进程下dump，可以用C:\windows\syswow64\taskmgr.exe来抓dmp，这样就不用转换了。

2.2 查看Critical Section死锁

输入!locks看看有没有互相死锁的Critical Section变量。

0:000:x86> !locks

Scanned 9 critical sections
1
2
3

结果是没有。

2.3 怀疑是0号线程陷入内核不返回

~*k浏览一下每个线程。

.  0  Id: 1f9c.7d4 Suspend: 0 Teb: fffdb000 Unfrozen
 # ChildEBP RetAddr      
00 004fea9c 74ae74bb     user32!NtUserMessageCall+0x15
01 004feb28 74ae6a8c     user32!RealDefWindowProcWorker+0x73
02 004feb48 72fe0b64     user32!RealDefWindowProcW+0x4a
03 004feba4 72fe0b96     uxtheme!_ThemeDefWindowProc+0x197
04 004febc0 74ae729a     uxtheme!ThemeDefWindowProcW+0x18
05 004fec08 666cba83     user32!DefWindowProcW+0x68
WARNING: Stack unwind information not available. Following frames may be wrong.
06 004fec60 63940a93     chrome!IsSandboxedProcess+0x440ff3
07 004feca0 6393fe82     chrome!ChromeMain+0x1dd5c3
08 004fed1c 637bd924     chrome!ChromeMain+0x1dc9b2
09 004fed44 637bd877     chrome!ChromeMain+0x5a454
0a 004fed84 74ae62fa     chrome!ChromeMain+0x5a3a7
0b 004fedb0 74ae7316     user32!InternalCallWinProc+0x23
0c 004fee28 74ae77c4     user32!UserCallWinProcCheckWow+0xd8
0d 004fee88 74ae788a     user32!DispatchMessageWorker+0x3b5
0e 004fee98 63ae3ff1     user32!DispatchMessageW+0xf
0f 004fef38 65dc4979     chrome!ChromeMain+0x380b21
10 004fefc0 63810c84     chrome!CreateQCMServiceForHiya+0x49b89
11 004ff000 637a274a     chrome!ChromeMain+0xad7b4
12 004ff024 637a02b3     chrome!ChromeMain+0x3f27a
13 004ff060 6379f5bf     chrome!ChromeMain+0x3cde3
14 004ff0b0 63ae26d0     chrome!ChromeMain+0x3c0ef
15 004ff0dc 63ae267e     chrome!ChromeMain+0x37f200
16 004ff0ec 63ae265e     chrome!ChromeMain+0x37f1ae
17 004ff0f8 637b7413     chrome!ChromeMain+0x37f18e
18 004ff154 637b72c3     chrome!ChromeMain+0x53f43
19 004ff188 63773813     chrome!ChromeMain+0x53df3
1a 004ff1c4 63773508     chrome!ChromeMain+0x10343
1b 004ff210 637733f3     chrome!ChromeMain+0x10038
1c 004ff220 63769ad9     chrome!ChromeMain+0xff23
1d 004ff308 6376965f     chrome!ChromeMain+0x6609
1e 004ff34c 63c55015     chrome!ChromeMain+0x618f
1f 004ff3e4 0105d036     chrome!ChromeMain360+0x105
20 004ff490 01053d0a     360se!get_start+0xc016
21 004ff62c 01052d7e     360se!get_start+0x2cea
22 004ffb1c 0111539a     360se!get_start+0x1d5e
23 004ffb68 74d6344d     360se!GetHandleVerifier+0x5d04a
24 004ffb74 76f79802     kernel32!BaseThreadInitThunk+0xe
25 004ffbb4 76f797d5     ntdll_76f40000!__RtlUserThreadStart+0x70
26 004ffbcc 00000000     ntdll_76f40000!_RtlUserThreadStart+0x1b
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

0号线程，一般是ui处理相关的线程。user32!NtUserMessageCall可能是进入内核或者是64位进程空间，猜测此例是0号线程进入内核不返回，进而出现ui卡死的现象。切换会64位空间，在看0号线程的rip

0:000:x86> !wow64exts.sw
Switched to Host mode
0:000> k
 # Child-SP          RetAddr               Call Site
00 00000000`0020da98 00000000`7499aee6     wow64win!NtUserMessageCall+0xa
01 00000000`0020daa0 00000000`749b27ef     wow64win!whNT32NtUserMessageCallCB+0x32
02 00000000`0020daf0 00000000`7499b022     wow64win!Wow64DoMessageThunk+0x8b
03 00000000`0020db30 00000000`749ed18f     wow64win!whNtUserMessageCall+0x12e
04 00000000`0020dbd0 00000000`74972776     wow64!Wow64SystemServiceEx+0xd7
05 00000000`0020e490 00000000`749ed286     wow64cpu!ServiceNoTurbo+0x2d
06 00000000`0020e550 00000000`749ec69e     wow64!RunCpuSimulation+0xa
07 00000000`0020e5a0 00000000`76db43c3     wow64!Wow64LdrpInitialize+0x42a
08 00000000`0020eaf0 00000000`76e19780     ntdll!LdrpInitializeProcess+0x17e3
09 00000000`0020efe0 00000000`76dc371e     ntdll! ?? ::FNODOBFM::`string'+0x22790
0a 00000000`0020f050 00000000`00000000     ntdll!LdrInitializeThunk+0xe
0:000> ub rip
wow64win!ZwUserGetMessage:
00000000`749bfd90 4c8bd1          mov     r10,rcx
00000000`749bfd93 b806100000      mov     eax,1006h
00000000`749bfd98 0f05            syscall
00000000`749bfd9a c3              ret
00000000`749bfd9b 0f1f440000      nop     dword ptr [rax+rax]
wow64win!NtUserMessageCall:
00000000`749bfda0 4c8bd1          mov     r10,rcx
00000000`749bfda3 b807100000      mov     eax,1007h
00000000`749bfda8 0f05            syscall
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

这里可以知道0号线程进入了syscall正在等待返回。

2.4 重建栈回溯

回到32位进程空间，观察一下0号线程的栈回溯，看看是什么api使得它进入内核空间。初看起来是user32!NtUserMessageCall，但是这里有提示WARNING: Stack unwind information not available. Following frames may be wrong.。所以还是先准确地回溯栈。
参考此文https://blog.csdn.net/magictong/article/details/28161757。
得到栈回溯如下。

004fea64  004feb28
004fea68  76f5013a ntdll_76f40000!KiUserCallbackDispatcher+0x2e
esp=004fea9c
004fea9c  74ae72b9 user32!NtUserMessageCall+0x15
004feaa0  74ae74bb user32!RealDefWindowProcWorker+0x73
004feb28  004feb48
004feb2c  74ae6a8c user32!RealDefWindowProcW+0x4a
004feb48  004feba4
004feb4c  72fe0b64 uxtheme!_ThemeDefWindowProc+0x197
004feba4  004febc0
004feba8  72fe0b96 uxtheme!ThemeDefWindowProcW+0x18
004febc0  004fec08
004febc4  74ae729a user32!DefWindowProcW+0x68
004fec08  004fec60
004fec0c  666cba83 chrome!IsSandboxedProcess+0x440ff3
004fec60  004feca0
004fec64  63940a93 chrome!ChromeMain+0x1dd5c3
004feca0  004fed1c
004feca4  6393fe82 chrome!ChromeMain+0x1dc9b2
004fed1c  004fed44
004fed20  637bd924 chrome!ChromeMain+0x5a454
004fed44  004fed84
004fed48  637bd877 chrome!ChromeMain+0x5a3a7
004fed84  004fedb0
004fed88  74ae62fa user32!InternalCallWinProc+0x23
004fedb0  004fee28
004fedb4  74ae7316 user32!UserCallWinProcCheckWow+0xd8
004fee28  004fee88
004fee2c  74ae77c4 user32!DispatchMessageWorker+0x3b5
004fee88  004fee98
004fee8c  74ae788a user32!DispatchMessageW+0xf
004fee98  004fef38
004fee9c  63ae3ff1 chrome!ChromeMain+0x380b21
004fef38  004fefc0
004fef3c  65dc4979 chrome!CreateQCMServiceForHiya+0x49b89
004fefc0  004ff000
004fefc4  63810c84 chrome!ChromeMain+0xad7b4
004ff000  004ff024
004ff004  637a274a chrome!ChromeMain+0x3f27a
004ff024  004ff060
004ff028  637a02b3 chrome!ChromeMain+0x3cde3
004ff060  004ff0b0
004ff064  6379f5bf chrome!ChromeMain+0x3c0ef
004ff0b0  004ff0dc
004ff0b4  63ae26d0 chrome!ChromeMain+0x37f200
004ff0dc  004ff0ec
004ff0e0  63ae267e chrome!ChromeMain+0x37f1ae
004ff0ec  004ff0f8
004ff0f0  63ae265e chrome!ChromeMain+0x37f18e
004ff0f8  004ff154
004ff0fc  637b7413 chrome!ChromeMain+0x53f43
004ff154  004ff188
004ff158  637b72c3 chrome!ChromeMain+0x53df3
004ff188  004ff1c4
004ff18c  63773813 chrome!ChromeMain+0x10343
004ff1c4  004ff210
004ff1c8  63773508 chrome!ChromeMain+0x10038
004ff210  004ff220
004ff214  637733f3 chrome!ChromeMain+0xff23
004ff220  004ff308
004ff224  63769ad9 chrome!ChromeMain+0x6609
004ff308  004ff34c
004ff30c  6376965f chrome!ChromeMain+0x618f
004ff34c  004ff3e4
004ff350  63c55015 chrome!ChromeMain360+0x105
004ff3e4  004ff490
004ff3e8  0105d036 360se!get_start+0xc016
004ff490  004ff62c
004ff494  01053d0a 360se!get_start+0x2cea
004ff62c  004ffb1c
004ff630  01052d7e 360se!get_start+0x1d5e
004ffb1c  004ffb68
004ffb20  0111539a 360se!GetHandleVerifier+0x5d04a
004ffb68  004ffb74
004ffb6c  74d6344d kernel32!BaseThreadInitThunk+0xe
004ffb74  004ffbb4
004ffb78  76f79802 ntdll_76f40000!__RtlUserThreadStart+0x70
004ffbb4  004ffbcc
004ffbb8  76f797d5 ntdll_76f40000!_RtlUserThreadStart+0x1b
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79

因为esp已经是004fea9c，所以

004fea64  004feb28
004fea68  76f5013a ntdll_76f40000!KiUserCallbackDispatcher+0x2e
1
2

这一段不是栈内的数据。
通过ub eip可以得知最后离开32位空间时正在执行的代码：

0:000:x86> ub eip
user32!DefWindowProcW+0x6e:
74ae72a0 90              nop
74ae72a1 90              nop
74ae72a2 90              nop
74ae72a3 90              nop
user32!NtUserMessageCall:
74ae72a4 b807100000      mov     eax,1007h
74ae72a9 b900000000      mov     ecx,0
74ae72ae 8d542404        lea     edx,[esp+4]
74ae72b2 64ff15c0000000  call    dword ptr fs:[0C0h]
1
2
3
4
5
6
7
8
9
10
11

ub 74ae6a8c查看user32!RealDefWindowProcW调用的是什么：

0:000:x86> ub 74ae6a8c
user32!RealDefWindowProcW+0x14:
74ae6a76 0f8428f10000    je      user32!RealDefWindowProcW+0x16 (74af5ba4)
74ae6a7c 6a00            push    0
74ae6a7e ff7514          push    dword ptr [ebp+14h]
74ae6a81 ff7510          push    dword ptr [ebp+10h]
74ae6a84 51              push    ecx
74ae6a85 52              push    edx
74ae6a86 50              push    eax
74ae6a87 e861ffffff      call    user32!RealDefWindowProcWorker (74ae69ed)
1
2
3
4
5
6
7
8
9
10

因此认为是user32!RealDefWindowProcW调用user32!RealDefWindowProcWorker,它里头再调用user32!NtUserMessageCall，随后脱离了32位进程空间。

3 寻找hwnd

user32!NtUserMessageCall的函数声明没有公开，从reactOS的代码ntuser.h中可以找到

BOOL
NTAPI
NtUserMessageCall(
    HWND hWnd,
    UINT Msg,
    WPARAM wParam,
    LPARAM lParam,
    ULONG_PTR ResultInfo,
    DWORD dwType, /* FNID_XX types */
    BOOL Ansi);
1
2
3
4
5
6
7
8
9
10

使用ida反汇编user32.dll，在user32!RealDefWindowProcWorker+0x73即user32.dll+174bb的位置，可以看到：

loc_7DC6748A:           ; _DWORD
push    [ebp+cchWideChar]
push    29Eh            ; _DWORD
push    0               ; _DWORD
push    [ebp+Dst]       ; _DWORD
push    [ebp+arg_C]     ; _DWORD
push    edx             ; _DWORD
push    [ebp+hWnd]      ; _DWORD
cmp     edx, 400h
jnb     loc_7DC94952
movzx   eax, ds:_MessageTable[edx]
and     eax, 3Fh
call    ds:_gapfnScSendMessage[eax*4] ; NtUserMessageCall(x,x,x,x,x,x,x) ...
jmp     loc_7DC66A56
1
2
3
4
5
6
7
8
9
10
11
12
13
14

查看栈顶的数据，可以分析出NtUserMessageCall的实际参数：

0:000:x86> dps esp
004fea9c  74ae72b9 user32!NtUserMessageCall+0x15
004feaa0  74ae74bb user32!RealDefWindowProcWorker+0x73
004feaa4  000b0586;	hWnd
004feaa8  00000112;	Msg=WM_SYSCOMMAND
004feaac  0000f020;	wParam=SC_MINIMIZE
004feab0  00000000; lParam
004feab4  00000000;	ResultInfo
004feab8  0000029e;	dwType
004feabc  00000000;	Ansi
004feac0  000b0586
004feac4  fffffffe
004feac8  00000112
004feacc  000b0586
004fead0  0000a918
004fead4  00000112
004fead8  74ae692a user32!NtUserQueryWindow+0x15
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

所以这是对hwnd=b0586窗口发生最小化命令时，出现的卡死。
欲探知这个hwnd的窗口信息，可以使用SDbgExt的插件www.nynaeve.net/?p=94。
下载后，用32位程序的windbg来load之（64位的windbg程序和windbg Preview版本无法load）：

0:000> .load D:\windbg\sdbgext\1.09\sdbgext.dll
0:000> !sdbgext.hwnd b0586
Invalid window handle 00000000000b0586
1
2
3

可能因为这是别的进程的窗口，所以从我进程的dmp里查不出这个窗口是什么。只能复现故障再用spy++找出那个窗口。

4 wow64初探

0号线程里，64位空间里的rsp和32位空间里的esp是不同的

0:000> r rsp
rsp=000000000020da98
0:000> !wow64exts.sw
Switched to 32bit mode
0:000:x86> r esp
esp=004fea9c
1
2
3
4
5
6

user32!NtUserMessageCall:
74ae72a4 b807100000      mov     eax,1007h
74ae72a9 b900000000      mov     ecx,0
74ae72ae 8d542404        lea     edx,[esp+4]
74ae72b2 64ff15c0000000  call    dword ptr fs:[0C0h]
1
2
3
4
5

call dword ptr fs:[0C0h]是如何保存32位空间的eip，retaddr；之后如何进入64位空间，如何切换到rip，如何归还eip，如何回到retAddr呢？
自己写一个demo：

#define WIN32_LEAN_AND_MEAN
#include <Windows.h>
#include <cstdio>
int main() {
	for (int i = 0; i < MAXINT; ++i) {
		printf("Press Enter\n");
		getchar();
		DefWindowProcW(HWND(0x302a2), WM_SYSCOMMAND, SC_MINIMIZE, 0);
	}
	return 0;
}
1
2
3
4
5
6
7
8
9
10
11

302a2是一个notepad.exe的窗口句柄。
编译为32位程序。放到win7 64OS内执行。这里的syswow64里的user32.dll跟故障环境里的版本有点不一样。
先执行这个exe，再用windbg64位（不要用32位的windbg否则不能在32，64之间切换）的来attach。
先断点user32!DefWindowProcW并继续执行:
bp user32!DefWindowProcW;g。
待断点触发后，再断点769772b2就是call那句：

USER32!NtUserMessageCall:
769772a4 b807100000      mov     eax,1007h
769772a9 b900000000      mov     ecx,0
769772ae 8d542404        lea     edx,[esp+4]
769772b2 64ff15c0000000  call    dword ptr fs:[0C0h]
769772b9 83c404          add     esp,4
769772bc c21c00          ret     1Ch
1
2
3
4
5
6
7

可以看到:

769772b2 64ff15c0000000  call    dword ptr fs:[0C0h]  fs:0053:000000c0=00000000
0:000:x86> bd 1
0:000:x86> t
wow64cpu!X86SwitchTo64BitMode:
73662320 ea1e2766733300  jmp     0033:7366271E
0:000:x86> t
wow64cpu!CpupReturnFromSimulatedCode:
00000000`7366271e 67448b0424      mov     r8d,dword ptr [esp] ds:00000000`0043f770=769772b9
0:000> p
wow64cpu!CpupReturnFromSimulatedCode+0x5:
00000000`73662723 458985bc000000  mov     dword ptr [r13+0BCh],r8d ds:00000000`001ffddc=7697692a
0:000> p
wow64cpu!CpupReturnFromSimulatedCode+0xc:
00000000`7366272a 4189a5c8000000  mov     dword ptr [r13+0C8h],esp ds:00000000`001ffde8=0043f7f8
0:000> p
wow64cpu!CpupReturnFromSimulatedCode+0x13:
00000000`73662731 498ba42480140000 mov     rsp,qword ptr [r12+1480h] ds:00000000`7efdc480=00000000001feb70
0:000> dds r13+bc
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for VCRUNTIME140.dll - 
00000000`001ffddc  769772b9 USER32!NtUserMessageCall+0x15
00000000`001ffde0  00000023
00000000`001ffde4  00000246
00000000`001ffde8  0043f770
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

当执行完73662320 ea1e2766733300 jmp 0033:7366271E时，进程空间变为64位了。
接下来就是把RetAddr=dword ptr[esp]存入r8d。然后又把r8d存入dword ptr [r13+0BCh]。esp被存入dword ptr [r13+0C8h]。这里的r13估计就是64位进程空间的代码里用于存放32位空间寄存器和RetAddr的一个标识位。
一路f11按下去，可以看到：

wow64win!NtUserMessageCall:
00000000`736afda0 4c8bd1          mov     r10,rcx
0:000> p
wow64win!ZwUserMessageCall+0x3:
00000000`736afda3 b807100000      mov     eax,1007h
0:000> p
wow64win!ZwUserMessageCall+0x8:
00000000`736afda8 0f05            syscall
1
2
3
4
5
6
7
8

这里就是这个线程真正进入内核的地方。
再一路f11按下去，可以看到：

wow64cpu!CpuSimulate+0x141:
00000000`736626f1 4989a42480140000 mov     qword ptr [r12+1480h],rsp ds:00000000`7efdc480=0000000000000000
wow64cpu!CpuSimulate+0x149:
00000000`736626f9 41c7460423000000 mov     dword ptr [r14+4],23h ds:00000000`001febe4=00000023
00000000`73662701 41b82b000000    mov     r8d,2Bh
00000000`73662707 418ed0          mov     ss,r8w
00000000`73662711 458b8dbc000000  mov     r9d,dword ptr [r13+0BCh] ds:00000000`001ffddc=769772b9
00000000`73662718 45890e          mov     dword ptr [r14],r9d
00000000`7366271b 41ff2e          jmp     fword ptr [r14]
1
2
3
4
5
6
7
8
9

又是r13+0bc，dword ptr[r13+0bch]里面的就是RetAddr，这里jmp过去，就回到USER32!NtUserMessageCall的call的下一句了。
至于esp如何从64位的rsp还原，就不得而知了。