[Iplant-api-dev] Creating a system/app/job combo using CLI/FORK scheduler with AGAVE

[Rion Dooley]

Hi Brian,

I threw this together to help illustrate the process. I can probably clean it up a bit, but it should give you a good walk-through. I personally walked through the example using one of the atmosphere VM’s and jobs were turning around in 45-60 seconds, so it should be a relatively quick exercise.

https://bitbucket.org/taccaci/agave-fork-sample/overview

Inside the html folder of the project is a simple web app that will allow you to submit the sample app. As long as you run it on localhost, you should be fine authenticating. Let me know if you hit any stumbling blocks. And, as always, your feedback is much appreciated.

--
Rion




On Jul 24, 2014, at 5:12 PM, Matthew Vaughn <vaughn at tacc.utexas.edu<mailto:vaughn at tacc.utexas.edu>> wrote:

I'll post an example tomorrow that we've been using for the Arabidopsis Information Portal project (I'm out of the office today)

Matt

--

Matthew W. Vaughn, Ph.D.
Director, Life Sciences Computing
Texas Advanced Computing Center
The University of Texas at Austin
(949) 436-6642


________________________________________
From: iplant-api-dev-bounces at iplantcollaborative.org<mailto:iplant-api-dev-bounces at iplantcollaborative.org> [iplant-api-dev-bounces at iplantcollaborative.org<mailto:iplant-api-dev-bounces at iplantcollaborative.org>] on behalf of Brian Corrie [bcorrie at sfu.ca<mailto:bcorrie at sfu.ca>]
Sent: Thursday, July 24, 2014 5:10 PM
To: Matthew Vaughn
Cc: Discussion of iPlant API development
Subject: [Iplant-api-dev] Creating a system/app/job combo using CLI/FORK        scheduler with AGAVE

Hello All,

I am trying to create a AGAVE system/app/job combination that uses the
FORK scheduler and the CLI executionType. I am using the AGAVE CLI
interface.

I am having trouble finding an example on the web site and I can't seem
to find the right combination of system, app, and job configurations.

Does anyone have such an example? My problems revolve around queues and
jobs submission. One fundamental question I have is. does a system that
uses CLI/FORK need a queue? I wouldn't think so, but thought I would check.

Here is my problem:

When I create a System without a queue, create an App that uses that
System, and submit a Job using that App, I get the following response
from the job submission:

Invalid nodeCount. No queue found on bugaboo-cli-bcorrie that support
jobs with 1 nodes.

So it looks like I need a queue.

If I create a system with a queue named default, when I submit a job the
job is successfully created but the submission phase of the job fails
three times and then the overall job fails. I am not sure what is wrong
at the submission phase.

bcorrie at itivm6$ jobs-history 0001406239429461-5056a550b8-0001-007
Job accepted and queued for submission.
Skipping staging. No input data associated with this job.
Preparing job for submission.
Attempt 1 to submit job
Attempt 1 failed to submit job. Failed to submit cli job.
Attempt 2 to submit job
Attempt 2 failed to submit job. Failed to submit cli job.
Attempt 3 to submit job
Attempt 3 failed to submit job. Failed to submit cli job.
Cleaning up remote work directory.
Completed cleaning up remote work directory.
Unable to submit job after 3 attempts. Job cancelled.

Does anyone have any advice or experience. Note, I have no problem
whatsoever running jobs as an HPC job using queues and the PBS scheduler
using AGAVE, it is the "simple" CLI/FORK jobs that are getting me.

I suspect I am missing something simple???

Thanks in advance for any help...

Brian






_______________________________________________
Iplant-api-dev Mailing List: Iplant-api-dev at iplantcollaborative.org<mailto:Iplant-api-dev at iplantcollaborative.org>
List Info and Archives: http://mail.iplantcollaborative.org/mailman/listinfo/iplant-api-dev
One-click Unsubscribe: http://mail.iplantcollaborative.org/mailman/options/iplant-api-dev/vaughn%40tacc.utexas.edu?unsub=1&unsubconfirm=1

_______________________________________________
Iplant-api-dev Mailing List: Iplant-api-dev at iplantcollaborative.org<mailto:Iplant-api-dev at iplantcollaborative.org>
List Info and Archives: http://mail.iplantcollaborative.org/mailman/listinfo/iplant-api-dev
One-click Unsubscribe: http://mail.iplantcollaborative.org/mailman/options/iplant-api-dev/dooley%40tacc.utexas.edu?unsub=1&unsubconfirm=1

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.iplantcollaborative.org/pipermail/iplant-api-dev/attachments/20140725/7b08c775/attachment.html