[Iplant-api-dev] [Fwd: Re: fAPI jobs are failing]
Ghiban, Cornel
ghiban at cshl.edu
Thu Apr 2 12:46:46 MST 2015
Hi Rion,
I know you've been telling us to move to Agave, but I never heard of a
hard deadline for Foundation. Now that we have one, we have to do
something about it :)
Thanks,
Cornel
On Thu, 2015-04-02 at 19:36 +0000, Rion Dooley wrote:
> Well, XSEDE is deprecating support for the majority of the
> technologies Foundation uses to interact with its infrastructure. In
> preparation for upcoming retirement of their legacy auth server and
> information services, they have begun fiddling with several server
> configurations. They don’t have a test mechanism in place to determine
> the impact of these changes on 3rd party services, so if they forget
> to add a server or two to their firewall rules, they won’t catch it
> until someone complains. Even then, they may or may not restore
> service depending on the perceived necessity. To compound things, a
> couple sites providing hosting for critical infrastructure pieces have
> had networking and hardware issues over the past week.
>
>
> I believe the question you have asked several times in different ways
> over the last week boils down to why Foundation has gone from stable
> over the last 3 years to suddenly and increasingly unstable over the
> last couple months. The answer is that it was built on top of
> infrastructure and services that either no longer exist, or are being
> aggressively killed by their maintainers. For example, the fundamental
> mechanism Foundation uses for data and system-level authentication
> will be disabled in XSEDE on May 5. This will essentially kill job
> submission barring another development cycle. Even then, the only
> system available via Foundation at this time is Lonestar 4 which will
> be retired in June.
>
>
> I’ve been asked several time why we are not actively patching and
> updating Foundation in response to the changing national HPC
> landscape. The answer is that we did address this 2 years ago when we
> released Agave. Not only is it immune to every single issue you’re
> hitting with Foundation, but Agave:
>
>
> * Averages over three 9’s of availability
>
>
> http://status.agaveapi.co/
>
>
> * Is significantly more performant
> * Allows you to utilize an ever-expanding range of systems and
> services:
> * And is battle tested constantly by users around the world:
>
>
> http://preview.agaveapi.co/dashboard/#/index
>
>
>
> This is why we’ve been aggressively, and often annoyingly, encouraging
> people to migrate off of Foundation and onto Agave for the last year.
> Agave IS the result of us hearing every question you’ve sent. It’s
> also the result of us hearing the questions, concerns, needs, and
> wishes from the entire community. It’s not our policy or our nature
> to leave anyone out in the cold. You can take one look at the list of
> language libraries, tutorials, boilerplate web applications, 3rd party
> integrations, live and static API documentation, and our excellent CLI
> tools to see how much time we spend trying to provide a better
> developer experience.
>
>
> Are we perfect? LOL. Hardly, but have we ignored even a single user?
> No, and we’re not going to start now. It’s just that sometimes the
> answer we want to hear isn’t the answer that will actually help us.
> This is one of those situations.
>
>
> To help you out as best I can, I’ve taken the last day to do some
> aggressive caching and replication of the underlying infrastructure
> Foundation relies upon so it will be a bit more resistant to the
> coming changes, but let me be very clear:
>
>
> You only have a matter of weeks before Foundation will essentially be
> crippled by XSEDE.
>
>
> The answer in the short and long term is to migrate over to Agave. If
> you have any questions about how to port things over, please let us
> know. We have tutorials, guides, and people to help you through the
> process.
>
>
> —
> Rion
>
> > On Apr 2, 2015, at 11:48 AM, Ghiban, Cornel <ghiban at cshl.edu> wrote:
> >
> > Hi Rion,
> >
> > Could you, please, tell me what's going on with the Foundation API?
> >
> > Thanks,
> > Cornel
> >
> > -------- Forwarded Message --------
> > From: "Ghiban, Cornel" <ghiban at cshl.edu>
> > To: Cornel Ghiban <ghiban at cshl.edu>
> > Cc: Discussion of iPlant API development
> > <iplant-api-dev at iplantcollaborative.org>
> > Subject: Re: [Iplant-api-dev] fAPI jobs are failing
> > Date: Thu, 2 Apr 2015 13:39:37 +0000
> >
> > Hi,
> >
> > This is still happening:
> > 'message' => 'Failed to submit job 68095:
> > org.iplantc.service.jobs.exceptions.JobException:
> > Welcome
> > to the Stampede Supercomputer
> >
> >
> > Could anyone tell me what's going on?
> >
> > Thanks,
> > Cornel
> >
> > On Wed, 2015-04-01 at 20:32 +0000, Ghiban, Cornel wrote:
> > > Hi all,
> > >
> > > All 44 jobs submitted today have failed today with similar error
> > > messages:
> > >
> > > 'message' => 'Failed to submit job 68085:
> > > org.iplantc.service.jobs.exceptions.JobException:
> > > Welcome
> > > to the Stampede Supercomputer'
> > >
> > > 'message' => 'Failed to submit job 68090:
> > > org.iplantc.service.jobs.exceptions.JobException:
> > > Welcome
> > > to the Stampede Supercomputer
> > >
> > > 'message' => 'Failed to submit job 68091:
> > > org.iplantc.service.jobs.exceptions.JobException:
> > > Welcome
> > > to the Stampede Supercomputer
> > >
> > > 'message' => 'Failed to submit job 68092:
> > > org.iplantc.service.jobs.exceptions.JobException:
> > > Welcome
> > > to the Stampede Supercomputer
> > >
> > >
> > > Thanks,
> > > Cornel
> > >
> > >
> > >
> > > _______________________________________________
> > > Iplant-api-dev Mailing List:
> > > Iplant-api-dev at iplantcollaborative.org
> > > List Info and Archives:
> > > http://mail.iplantcollaborative.org/mailman/listinfo/iplant-api-dev
> > > One-click Unsubscribe:
> > > http://mail.iplantcollaborative.org/mailman/options/iplant-api-dev/ghiban%40cshl.edu?unsub=1&unsubconfirm=1
> >
> >
> > _______________________________________________
> > Iplant-api-dev Mailing List: Iplant-api-dev at iplantcollaborative.org
> > List Info and Archives:
> > http://mail.iplantcollaborative.org/mailman/listinfo/iplant-api-dev
> >
> > One-click Unsubscribe:
> > http://mail.iplantcollaborative.org/mailman/options/iplant-api-dev/ghiban%40cshl.edu?unsub=1&unsubconfirm=1
> >
> >
>
>
More information about the Iplant-api-dev
mailing list