Some days ago a flow running into JCAPS 5.1 produced this exception:
java.lang.OutOfMemoryError: unable to create new native thread
The first attempt, especially if you are used to ICAN 5.0, would be to add memory to the JVM's heap with the -Xmx flag. But forget for a while about the misleading "OutOfMemoryError" and focus to the rest of the message: it is clearly telling that the JVM was asking the O.S. to create a native thread, but that was not possible. It does not mean you don't have enough heap. In fact, the mentioned flow was already running into a Logicalhost with 1024 Mb of memory and there was no sign that it was not enough.
Depending on your operating system and JVM version, you can have a pretty different per-thread stack size which affects both the maximum number of native threads you can start and the overall consumed memory. See the Java HotSpot VM Options:
Thread Stack Size (in Kbytes). (0 means use default stack size)
Solaris x86: 320 (was 256 prior in 5.0 and earlier);
Sparc 64 bit: 1024;
Linux amd64: 1024 (was 0 in 5.0 and earlier);
all others 0.
When your application is trying to start too many threads you might need to decrease the default stack assigned to each thread using the -Xss parameter, so that each single thread has less stack but you can create more of them. For some operating systems this is not enough, you should decrease the O.S stack size using the "ulimit -s" command.
Currently, some stack sizes are:
ThreadSS VMThreadSS CompilerThreadSS default_stack_size
SPARC 32 512K 0 C2:2048K C1:0 not used
SPARC 64 1024K 0 C2:2048K C1:0 not used
Solaris i486 256K 0 C2:2048K C1:0 not used
Linux i486 0 0 0 512K
Linux ia64 0 0 0 1024K
Win32 i486 0 0 0 ASSERTs:1024K 0
Win32 ia64 0 0 0 ASSERTs:1024K 0
1) 0 for VMThreadSS and CompilerThreadSS implies
use of ThreadSS value
2) 0 for ThreadSS implies use of default_stack_size
Generally speaking you should usually start testing your flows with a little JVM heap, the default JCAPS 512 Mb is normally enough. Then you should increase step by step this value only and only if your processes are allocating big data structures requiring more heap. A good step size could be 256 Mb. It is never a brilliant idea to set your domain's heap to, say, 1.5 Gb by default only because this seems to let you sleep well. Additionally to the mentioned thread problem a bigger heap will lead to more complex and longer garbage collection cycles, penalizing performances in the medium term. It is then a good idea to set the same value for -Xms and -Xmx to help the GC.