[Iplant-api-dev] Error occurred during initialization of VM

[Matthew Vaughn]

That’s a safe plan. The nodes are empty when your job is routed to them. 

One place we do see issues is when we are running large numbers of JVM jobs either in parallel or serially on a node using, say, the TACC Launcher. There, we see memory fragmentation (plus the fact that the allocations you are asking for via -Xmx are for the heap only and not the rest of the JVM) that eventually leads to some of the tasks failing with memory allocation issues. In this case, the solution is to dial back the heap size requests and/or the number of concurrent tasks -OR- implement a retry mechanism to deal with failed JVM starts.

On Dec 12, 2013, at 4:25 PM, Damian Gessler <dgessler at iplantcollaborative.org> wrote:

> Roger that; thanks. So I'll use idev and srun for interactive testing, 
> but my wrapper.sh will just call 'java ....' as regular command line call.
> 
> (And I'll just 'assume' that if I call for -Xmx4096m or -Xmx8192m, that 
> the user [JVM] will get it).
> 
> D.
> 
> On 12/12/13 3:22 PM, Matthew Vaughn wrote:
>> No. idev just gets you, from the login, access to a compute node so you can test out your code in a production environment. You need to do NOTHING SPECIAL to route jobs to the compute nodes if your code is using the JVM
>> 
>> 
>> On Dec 12, 2013, at 4:20 PM, Damian Gessler <dgessler at iplantcollaborative.org> wrote:
>> 
>>> So do we need to do anything special in wrapper.sh (e.g, call srun or
>>> idev explicitly), or is that staging done automagically?
>>> 
>>> On 12/12/13 3:18 PM, Matthew Vaughn wrote:
>>>> What I do not know is what type of JVM alloc errors a user may get when
>>>>> running a Java app under fAPI. Perhaps Rion or Matt can clarify.
>>>> 
>>>> The API is just sending jobs off to compute nodes - - if you can get your Java code working on those nodes, it will work under the API.
>>>> 
>>>> 
>>> _______________________________________________
>>> Iplant-api-dev Mailing List: Iplant-api-dev at iplantcollaborative.org
>>> List Info and Archives: http://mail.iplantcollaborative.org/mailman/listinfo/iplant-api-dev
>>> One-click Unsubscribe: http://mail.iplantcollaborative.org/mailman/options/iplant-api-dev/vaughn%40tacc.utexas.edu?unsub=1&unsubconfirm=1
>> 
> _______________________________________________
> Iplant-api-dev Mailing List: Iplant-api-dev at iplantcollaborative.org
> List Info and Archives: http://mail.iplantcollaborative.org/mailman/listinfo/iplant-api-dev  
> One-click Unsubscribe: http://mail.iplantcollaborative.org/mailman/options/iplant-api-dev/vaughn%40tacc.utexas.edu?unsub=1&unsubconfirm=1