[Iplant-api-dev] [Fwd: Re: fAPI jobs are failing]

[Rion Dooley]

Well, XSEDE  is deprecating support for the majority of the technologies Foundation uses to interact with its infrastructure. In preparation for upcoming retirement of their legacy auth server and information services, they have begun fiddling with several server configurations. They don’t have a test mechanism in place to determine the impact of these changes on 3rd party services, so if they forget to add a server or two to their firewall rules, they won’t catch it until someone complains.  Even then, they may or may not restore service depending on the perceived necessity. To compound things, a couple sites providing hosting for critical infrastructure pieces have had networking and hardware issues over the past week.

I believe the question you have asked several times in different ways over the last week boils down to why Foundation has gone from stable over the last 3 years to suddenly and increasingly unstable over the last couple months. The answer is that it was built on top of infrastructure and services that either no longer exist, or are being aggressively killed by their maintainers. For example, the fundamental mechanism Foundation uses for data and system-level authentication will be disabled in XSEDE on May 5.  This will essentially kill job submission barring another development cycle. Even then, the only system available via Foundation at this time is Lonestar 4 which will be retired in June.

I’ve been asked several time why we are not actively patching and updating Foundation in response to the changing national HPC landscape. The answer is that we did address this 2 years ago when we released Agave. Not only is it immune to every single issue you’re hitting with Foundation, but Agave:

* Averages over three 9’s of availability

 http://status.agaveapi.co/

* Is significantly more performant
* Allows you to utilize an ever-expanding range of systems  and services:
* And is battle tested constantly by users around the world:

http://preview.agaveapi.co/dashboard/#/index

This is why we’ve been aggressively, and often annoyingly, encouraging people to migrate off of Foundation and onto Agave for the last year. Agave IS the result of us hearing every question you’ve sent. It’s also the result of us hearing the questions, concerns, needs, and wishes from the entire community.   It’s not our policy or our nature to leave anyone out in the cold. You can take one look at the list of language libraries, tutorials, boilerplate web applications, 3rd party integrations, live and static API documentation, and our excellent CLI tools to see how much time we spend trying to provide a better developer experience.

Are we perfect? LOL. Hardly, but have we ignored even a single user? No, and we’re not going to start now. It’s just that sometimes the answer we want to hear isn’t the answer that will actually help us. This is one of those situations.

To help you out as best I can, I’ve taken the last day to do some aggressive caching and replication of the underlying infrastructure Foundation relies upon so it will be a bit more resistant to the coming changes, but let me be very clear:

You only have a matter of weeks before Foundation will essentially be crippled by XSEDE.

The answer in the short and long term is to migrate over to Agave. If you have any questions about how to port things over, please let us know. We have tutorials, guides, and people to help you through the process.

—
Rion

On Apr 2, 2015, at 11:48 AM, Ghiban, Cornel <ghiban at cshl.edu<mailto:ghiban at cshl.edu>> wrote:

Hi Rion,

Could you, please, tell me what's going on with the Foundation API?

Thanks,
Cornel

-------- Forwarded Message --------
From: "Ghiban, Cornel" <ghiban at cshl.edu<mailto:ghiban at cshl.edu>>
To: Cornel Ghiban <ghiban at cshl.edu<mailto:ghiban at cshl.edu>>
Cc: Discussion of iPlant API development
<iplant-api-dev at iplantcollaborative.org<mailto:iplant-api-dev at iplantcollaborative.org>>
Subject: Re: [Iplant-api-dev] fAPI jobs are failing
Date: Thu, 2 Apr 2015 13:39:37 +0000

Hi,

This is still happening:
'message' => 'Failed to submit job 68095:
org.iplantc.service.jobs.exceptions.JobException:               Welcome
to the Stampede Supercomputer


Could anyone tell me what's going on?

Thanks,
Cornel

On Wed, 2015-04-01 at 20:32 +0000, Ghiban, Cornel wrote:
Hi all,

All 44 jobs submitted today have failed today with similar error
messages:

'message' => 'Failed to submit job 68085:
org.iplantc.service.jobs.exceptions.JobException:               Welcome
to the Stampede Supercomputer'

'message' => 'Failed to submit job 68090:
org.iplantc.service.jobs.exceptions.JobException:               Welcome
to the Stampede Supercomputer

'message' => 'Failed to submit job 68091:
org.iplantc.service.jobs.exceptions.JobException:               Welcome
to the Stampede Supercomputer

'message' => 'Failed to submit job 68092:
org.iplantc.service.jobs.exceptions.JobException:               Welcome
to the Stampede Supercomputer


Thanks,
Cornel



_______________________________________________
Iplant-api-dev Mailing List: Iplant-api-dev at iplantcollaborative.org<mailto:Iplant-api-dev at iplantcollaborative.org>
List Info and Archives: http://mail.iplantcollaborative.org/mailman/listinfo/iplant-api-dev
One-click Unsubscribe: http://mail.iplantcollaborative.org/mailman/options/iplant-api-dev/ghiban%40cshl.edu?unsub=1&unsubconfirm=1


_______________________________________________
Iplant-api-dev Mailing List: Iplant-api-dev at iplantcollaborative.org<mailto:Iplant-api-dev at iplantcollaborative.org>
List Info and Archives: http://mail.iplantcollaborative.org/mailman/listinfo/iplant-api-dev
One-click Unsubscribe: http://mail.iplantcollaborative.org/mailman/options/iplant-api-dev/ghiban%40cshl.edu?unsub=1&unsubconfirm=1


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.iplantcollaborative.org/pipermail/iplant-api-dev/attachments/20150402/b7ca5cae/attachment.html