VS2003 C++ 中异常的堆大小限制
我有一个使用大量数据的 C++ 应用程序，并且在测试时注意到它的内存不足，但仍有足够的可用内存.我已将代码简化为示例测试用例，如下所示；
I have a C++ app that uses large arrays of data, and have noticed while testing that it is running out of memory, while there is still plenty of memory available. I have reduced the code to a sample test case as follows;
void MemTest()
{
size_t Size = 500*1024*1024; // 512mb
if (Size > _HEAP_MAXREQ)
TRACE("Invalid Size");
void * mem = malloc(Size);
if (mem == NULL)
TRACE("allocation failed");
}
如果我创建一个新的 MFC 项目，包括这个函数，并从 InitInstance 运行它，它在调试模式下工作正常(内存按预期分配)，但在释放模式下失败(malloc 返回 NULL).单步执行释放到 C 运行时，我的函数被内联我得到以下内容
If I create a new MFC project, include this function, and run it from InitInstance, it works fine in debug mode (memory allocated as expected), yet fails in release mode (malloc returns NULL). Single stepping through release into the C run times, my function gets inlined I get the following
// malloc.c
void * __cdecl _malloc_base (size_t size)
{
void *res = _nh_malloc_base(size, _newmode);
RTCCALLBACK(_RTC_Allocate_hook, (res, size, 0));
return res;
}
调用_nh_malloc_base
Calling _nh_malloc_base
void * __cdecl _nh_malloc_base (size_t size, int nhFlag)
{
void * pvReturn;
// validate size
if (size > _HEAP_MAXREQ)
return NULL;
'
'
并且 (size > _HEAP_MAXREQ) 返回 true，因此我的内存没有被分配.关注大小会返回预期的 512MB，这表明该程序正在链接到一个具有小得多的 _HEAP_MAXREQ 的不同运行时库.Grepping _HEAP_MAXREQ 的 VC++ 文件夹显示预期的 0xFFFFFFE0，所以我无法弄清楚这里发生了什么.任何人都知道会导致此问题的任何 CRT 更改或版本，还是我遗漏了一些更明显的东西?
And (size > _HEAP_MAXREQ) returns true and hence my memory doesn't get allocated. Putting a watch on size comes back with the exptected 512MB, which suggests the program is linking into a different run-time library with a much smaller _HEAP_MAXREQ. Grepping the VC++ folders for _HEAP_MAXREQ shows the expected 0xFFFFFFE0, so I can't figure out what is happening here. Anyone know of any CRT changes or versions that would cause this problem, or am I missing something way more obvious?
根据 Andreas 的建议，在此装配视图下查看此显示以下内容；
As suggested by Andreas, looking at this under this assembly view shows the following;
--- f:vs70builds3077vccrtbldcrtsrcmalloc.c ------------------------------
_heap_alloc:
0040B0E5 push 0Ch
0040B0E7 push 4280B0h
0040B0EC call __SEH_prolog (40CFF8h)
0040B0F1 mov esi,dword ptr [size]
0040B0F4 cmp dword ptr [___active_heap (434660h)],3
0040B0FB jne $L19917+7 (40B12Bh)
0040B0FD cmp esi,dword ptr [___sbh_threshold (43464Ch)]
0040B103 ja $L19917+7 (40B12Bh)
0040B105 push 4
0040B107 call _lock (40DE73h)
0040B10C pop ecx
0040B10D and dword ptr [ebp-4],0
0040B111 push esi
0040B112 call __sbh_alloc_block (40E736h)
0040B117 pop ecx
0040B118 mov dword ptr [pvReturn],eax
0040B11B or dword ptr [ebp-4],0FFFFFFFFh
0040B11F call $L19916 (40B157h)
$L19917:
0040B124 mov eax,dword ptr [pvReturn]
0040B127 test eax,eax
0040B129 jne $L19917+2Ah (40B14Eh)
0040B12B test esi,esi
0040B12D jne $L19917+0Ch (40B130h)
0040B12F inc esi
0040B130 cmp dword ptr [___active_heap (434660h)],1
0040B137 je $L19917+1Bh (40B13Fh)
0040B139 add esi,0Fh
0040B13C and esi,0FFFFFFF0h
0040B13F push esi
0040B140 push 0
0040B142 push dword ptr [__crtheap (43465Ch)]
0040B148 call dword ptr [__imp__HeapAlloc@12 (425144h)]
0040B14E call __SEH_epilog (40D033h)
0040B153 ret
$L19914:
0040B154 mov esi,dword ptr [ebp+8]
$L19916:
0040B157 push 4
0040B159 call _unlock (40DDBEh)
0040B15E pop ecx
$L19929:
0040B15F ret
_nh_malloc:
0040B160 cmp dword ptr [esp+4],0FFFFFFE0h
0040B165 ja _nh_malloc+29h (40B189h)
寄存器如下；
EAX = 009C8AF0 EBX = FFFFFFFF ECX = 009C8A88 EDX = 00747365 ESI = 00430F80EDI = 00430F80 EIP = 0040B160 ESP = 0013FDF4 EBP = 0013FFC0 EFL = 00000206
EAX = 009C8AF0 EBX = FFFFFFFF ECX = 009C8A88 EDX = 00747365 ESI = 00430F80 EDI = 00430F80 EIP = 0040B160 ESP = 0013FDF4 EBP = 0013FFC0 EFL = 00000206
所以比较似乎是针对正确的常数，即@040B160 cmp dword ptr [esp+4],0FFFFFFE0h，还有 esp+4 = 0013FDF8 = 1F400000 (my 512mb)
So the compare does appear to be against the correct constant, i.e. @040B160 cmp dword ptr [esp+4],0FFFFFFE0h, also esp+4 = 0013FDF8 = 1F400000 (my 512mb)
第二次根据 Andreas 的帖子，问题实际上出在 HeapAlloc 中.使用 HeapCreate & 为大对象更改为新的单独堆HeapAlloc 并没有帮助缓解问题，也没有尝试使用具有各种参数的 VirtualAlloc.一些进一步的实验表明，在分配一大段连续内存失败的情况下，两个较小的块产生相同的总内存是可以的.例如在 300MB malloc 失败的情况下，2 x 150MB malloc 可以正常工作.所以看起来我需要一个新的数组类，它可以存在于许多较大的内存片段中，而不是单个连续块中.不是什么大问题，但在当今这个时代，我本来希望 Win32 多一点.
Second edit: Problem was actually in HeapAlloc, as per Andreas' post. Changing to a new seperate heap for large objects, using HeapCreate & HeapAlloc, did not help alleviate the problem, nor did an attempt to use VirtualAlloc with various parameters. Some further experimentation has shown that where allocation one large section of contiguous memory fails, two smaller blocks yielding the same total memory is ok. e.g. where a 300MB malloc fails, 2 x 150MB mallocs work ok. So it looks like I'll need a new array class that can live in a number of biggish memory fragments rather than a single contiguous block. Not a major problem, but I would have expected a bit more out of Win32 in this day and age.
上次以下内容产生了 1.875GB 的空间，尽管不连续
Last edit: The following yielded 1.875GB of space, albeit non-contiguous
#define TenMB 1024*1024*10
void SmallerAllocs()
{
size_t Total = 0;
LPVOID p[200];
for (int i = 0; i < 200; i++)
{
p[i] = malloc(TenMB);
if (p[i])
Total += TenMB; else
break;
}
CString Msg;
Msg.Format("Allocated %0.3lfGB",Total/(1024.0*1024.0*1024.0));
AfxMessageBox(Msg,MB_OK);
}
推荐答案
可能是调试器在发布模式中欺骗了你?在发布模式下，单步执行和变量值都不可靠.
May it be the cast that the debugger is playing a trick on you in release-mode? Neither single stepping nor the values of variables are reliable in release-mode.
我在发布模式下在 VS2003 中尝试了您的示例，当单步执行时，它起初看起来像是代码落在 return NULL 行上，但是当我继续单步执行时，它最终会继续进入 HeapAlloc，我猜是这个函数失败了，查看反汇编 if (size > _HEAP_MAXREQ) 显示以下内容:
I tried your example in VS2003 in release mode, and when single stepping it does at first look like the code is landing on the return NULL line, but when I continue stepping it eventually continues into HeapAlloc, I would guess that it's this function that's failing, looking at the disassembly if (size > _HEAP_MAXREQ) reveals the following:
00401078 cmp dword ptr [esp+4],0FFFFFFE0h
所以我不认为 _HEAP_MAXREQ 有问题.
so I don't think it's a problem with _HEAP_MAXREQ.
相关文章