java.lang.OutOfMemoryError: unable to create new native thread

Symptoms

Some days ago a flow running into JCAPS 5.1 produced this exception:

java.lang.OutOfMemoryError: unable to create new native thread

The first attempt, especially if you are used to ICAN 5.0, would be to add memory to the JVM's heap with the -Xmx flag. But forget for a while about the misleading "OutOfMemoryError" and focus to the rest of the message: it is clearly telling that the JVM was asking the O.S. to create a native thread, but that was not possible. It does not mean you don't have enough heap. In fact, the mentioned flow was already running into a Logicalhost with 1024 Mb of memory and there was no sign that it was not enough.

Diagnosis

Depending on your operating system and JVM version, you can have a pretty different per-thread stack size which affects both the maximum number of native threads you can start and the overall consumed memory. See the Java HotSpot VM Options:

Thread Stack Size (in Kbytes). (0 means use default stack size)
Sparc: 512;
Solaris x86: 320 (was 256 prior in 5.0 and earlier);
Sparc 64 bit: 1024;
Linux amd64: 1024 (was 0 in 5.0 and earlier);
all others 0.

When your application is trying to start too many threads you might need to decrease the default stack assigned to each thread using the -Xss parameter, so that each single thread has less stack but you can create more of them. For some operating systems this is not enough, you should decrease the O.S stack size using the "ulimit -s" command.

Currently, some stack sizes are:

           ThreadSS  VMThreadSS  CompilerThreadSS   default_stack_size
                    
SPARC 32       512K      0  C2:2048K   C1:0      not used
SPARC 64      1024K      0  C2:2048K   C1:0      not used
Solaris i486   256K      0  C2:2048K   C1:0      not used
                       
Linux i486       0       0  0                      512K
Linux ia64       0       0  0                     1024K
                
Win32 i486       0       0  0  ASSERTs:1024K         0
Win32 ia64       0       0  0  ASSERTs:1024K         0

Notes:
  1) 0 for VMThreadSS and CompilerThreadSS implies
     use of ThreadSS value

  2) 0 for ThreadSS implies use of default_stack_size

Generally speaking you should usually start testing your flows with a little JVM heap, the default JCAPS 512 Mb is normally enough. Then you should increase step by step this value only and only if your processes are allocating big data structures requiring more heap. A good step size could be 256 Mb. It is never a brilliant idea to set your domain's heap to, say, 1.5 Gb by default only because this seems to let you sleep well. Additionally to the mentioned thread problem a bigger heap will lead to more complex and longer garbage collection cycles, penalizing performances in the medium term. It is then a good idea to set the same value for -Xms and -Xmx to help the GC.