System.OutOfMemoryException
The exception that is thrown for errors in an arithmetic, casting, or conversion operation.
Minimum version: >= 1.1 >= Core 1.0
Statistics
12
How to handle it
try
{
}
catch (System.OutOfMemoryException e)
{
}try
{
}
catch (System.OutOfMemoryException e) when (e.Message.Contains("something"))
{
}try
{
}
catch (System.OutOfMemoryException e) when (LogException(e))
{
}
private static bool LogException(Exception e)
{
logger.LogError(...);
return false;
}How to avoid it
We haven't written anything about avoiding this exception yet. Got a good tip on how to avoid throwing System.OutOfMemoryException? Feel free to reach out through the support widget in the lower right corner with your suggestions.
Links
YouTube videos
Possible fixes from StackOverflow
You may want to read this: "“Out Of Memory” Does Not Refer to Physical Memory" by Eric Lippert.
In short, and very simplified, "Out of memory" does not really mean that the amount of available memory is too small. The most common reason is that within the current address space, there is no contiguous portion of memory that is large enough to serve the wanted allocation. If you have 100 blocks, each 4 MB large, that is not going to help you when you need one 5 MB block.
Key Points:
- the data storage that we call “process memory” is in my opinion best visualized as a massive file on disk.
- RAM can be seen as merely a performance optimization
- Total amount of virtual memory your program consumes is really not hugely relevant to its performance
- "running out of RAM" seldom results in an “out of memory” error. Instead of an error, it results in bad performance because the full cost of the fact that storage is actually on disk suddenly becomes relevant.
Each character requires 2 bytes (as a char in .NET is a UTF-16 code unit). So by the time you've reached 800 million characters, that's 1.6GB of contiguous memory required1. Now when the StringBuilder needs to resize itself, it has to create another array of the new size (which I believe tries to double the capacity) - which means trying to allocate a 3.2GB array.
I believe that the CLR (even on 64-bit systems) can't allocate a single object of more than 2GB in size. (That certainly used to be the case.) My guess is that your StringBuilder is trying to double in size, and blowing that limit. You may be able to get a little higher by constructing the StringBuilder with a specific capacity - a capacity of around a billion may be feasible.
In the normal course of things this isn't a problem, of course - even strings requiring hundreds of megs are rare.
1 I believe the implementation of StringBuilder actually changed in .NET 4 to use fragments in some situations - but I don't know the details. So it may not always need contiguous memory while still in builder form... but it would if you ever called ToString.
Check that you are building a 64-bit process, and not a 32-bit one, which is the default compilation mode of Visual Studio. To do this, right click on your project, Properties -> Build -> platform target : x64. As any 32-bit process, Visual Studio applications compiled in 32-bit have a virtual memory limit of 2GB.
64-bit processes do not have this limitation, as they use 64-bit pointers, so their theoretical maximum address space (the size of their virtual memory) is 16 exabytes (2^64). In reality, Windows x64 limits the virtual memory of processes to 8TB. The solution to the memory limit problem is then to compile in 64-bit.
However, object’s size in .NET is still limited to 2GB, by default. You will be able to create several arrays whose combined size will be greater than 2GB, but you cannot by default create arrays bigger than 2GB. Hopefully, if you still want to create arrays bigger than 2GB, you can do it by adding the following code to you app.config file:
<configuration>
<runtime>
<gcAllowVeryLargeObjects enabled="true" />
</runtime>
</configuration>
You don't have a continuous block of memory in order to allocate 762MB, your memory is fragmented and the allocator cannot find a big enough hole to allocate the needed memory.
- You can try to work with /3GB (as others had suggested)
- Or switch to 64 bit OS.
- Or modify the algorithm so it will not need a big chunk of memory. maybe allocate a few smaller (relatively) chunks of memory.
IMAGE_FILE_LARGE_ADDRESS_AWARE is only relevant for 32 bit processes. The reason is that the address space on 32 bit Windows is split in two: 2 GB for kernel space and 2 GB for user space. To address 2 GB you need 31 bits. I.e. the pointers in a 32 bit application do not need the last bit for addressing.
Some applications may have used this extra bit for custom purposes, so if the Windows memory manager suddenly hands them a real 32 bit address they can't handle that. By enabling the IMAGE_FILE_LARGE_ADDRESS_AWARE flag the application basically tells the OS that it can handle the entire 32 bit addressable space.
If you run a IMAGE_FILE_LARGE_ADDRESS_AWARE application on 32 bit Windows you can access 3 GB. If you run the same 32 bit application on 64 bit Windows the process actually gets the entire 4 GB address space.
If you run a 64 bit application on 64 bit Windows the user address space is 8 TB (with another 8 TB set aside for kernel address space). .NET applications set to AnyCPU will automatically be 64 bit applications on x64, so you don't have to do anything to address the additional memory.
Keep in mind, however, that the CLR imposes a 2 GB limit on any single object, so while your application may use a lot of memory, you cannot create a 2 TB array for instance. More info in this question: Single objects still limited to 2 GB in size in CLR 4.0?
Source: Stack Overflow