[api-dev] Agave CLI SDK files-import question/help

[Ghiban, Cornel]

Hi,

I'm looking into importing some files (from the output files of a job set with archive=false) and it looks like +files-imports is not capable of handling such files:

$ jobs-output-list 1202088293959003675-ee4acae9fffff7a7-0001-007 fastx_out/WT_rep2-fx2291_fastqc.html
WT_rep2-fx2291_fastqc.html

$ files-import -V -U https://agave.iplantc.org/jobs/v2/1202088293959003675-ee4acae9fffff7a7-0001-007/outputs/media/fastx_out/WT_rep2-fx2291_fastqc.html -W ghiban at cshl.edu ghiban/tmp/
Calling curl -sk -H "Authorization: Bearer ..." -X POST --data-urlencode "urlToIngest=https://agave.iplantc.org/jobs/v2/1202088293959003675-ee4acae9fffff7a7-0001-007/outputs/media/fastx_out/WT_rep2-fx2291_fastqc.html" --data-urlencode "notifications=ghiban at cshl.edu" --data-urlencode "fileType=" --data-urlencode "fileName=" https://agave.iplantc.org/files/v2/media/ghiban/tmp/?pretty=true
{
  "status" : "success",
  "message" : null,
  "version" : "2.1.6-rbd07feb",
  "result" : {
    "name" : "WT_rep2-fx2291_fastqc.html",
    "uuid" : "3459001805835145701-242ac112-0001-002",
    "owner" : "ghiban",
    "internalUsername" : null,
    "lastModified" : "2016-02-23T10:14:46.712-06:00",
    "source" : "https://agave.iplantc.org/jobs/v2/1202088293959003675-ee4acae9fffff7a7-0001-007/outputs/media/fastx_out/WT_rep2-fx2291_fastqc.html",
    "path" : "ghiban/tmp/WT_rep2-fx2291_fastqc.html",
    "status" : "STAGING_QUEUED",
    "systemId" : "data.iplantcollaborative.org",
    "nativeFormat" : "raw",
    "_links" : {
      "self" : {
        "href" : "https://agave.iplantc.org/files/v2/media/system/data.iplantcollaborative.org/ghiban/tmp/WT_rep2-fx2291_fastqc.html"
      },
      "system" : {
        "href" : "https://agave.iplantc.org/systems/v2/data.iplantcollaborative.org"
      },
      "history" : {
        "href" : "https://agave.iplantc.org/files/v2/history/system/data.iplantcollaborative.org/ghiban/tmp/WT_rep2-fx2291_fastqc.html"
      }
    }
  }
}

$ files-history  ghiban/tmp/WT_rep2-fx2291_fastqc.html
File/folder queued for staging
File/folder queued for staging
File/folder queued for staging
Transfer in progress
Transfer in progress
Transfer in progress
Transfer in progress
Your scheduled transfer of https://agave.iplantc.org/jobs/v2/1202088293959003675-ee4acae9fffff7a7-0001-007/outputs/media/fastx_out/WT_rep2-fx2291_fastqc.html failed after 3 attempts with the following error message: No such file or directory. Please check the source url, https://agave.iplantc.org/jobs/v2/1202088293959003675-ee4acae9fffff7a7-0001-007/outputs/media/fastx_out/WT_rep2-fx2291_fastqc.html and make sure it is a  valid URI or path on your default system. If you feel there was an error and this problem persists, please contact your api administrator for assistance.

File exists, here's a GET REST call to it:

$ curl -sk -H "Authorization: Bearer ..." -X GET https://agave.iplantc.org/jobs/v2/1202088293959003675-ee4acae9fffff7a7-0001-007/outputs/media/fastx_out/WT_rep2-fx2291_fastqc.html > /tmp/WT_rep2-fx2291_fastqc.html
$ echo $?
0
$ ls -l /tmp/WT_rep2-fx2291_fastqc.html

I'd like to mention that submitting jobs with input files like this on work well with jobs.

Thanks,
Cornel