Asynchronous Processing API

Asynchronous Processing API is currently in beta release.

Asynchronous Processing API (or shortly "Async API") allows you to process more data with a single request than Processing API. This is possible because the processing results are not returned immediately but are - after some time - delivered in your object storage. We recommend using Async API when you want to process bigger images, when you do not want to deal with tiled results and when it is not crucial for you to get the processing results immediately.

General approach

The Async API allows you to process the data in a similar way as Processing API; you define input data, area of interest and time range in the body of an Async API request and Sentinel Hub will process the data as defined in your evalscript. When using Async API keep in mind that:

• The maximum output image size can not exceed 10000 pixels in any dimension.
• Evalscript can be either sent directly in the request or it can be stored in S3 and referenced in an async request ( see parameter evalscriptReference in Async API reference for more details). This allows you to use bigger evalscripts.
• The processing is asynchronous, which means that you do not get results in a response of your request. Instead they are delivered in your object storage.
• A copy of your Async API request is also stored in your object storage. After the request is processed, the copy is updated and additional information about the cost of the request is added.
• Only a limited number of async requests can run concurrently for each Sentinel Hub user. The exact limit depends on the account type.
• The processing time depends on request size and on the current and past load of the service. Generally, the first request is the slowest while the subsequent requests run faster.
• The cost is the same as with Process API, except that the minimal cost is 10 PU per request.

Async API deployments

AWS bucket access

The Async API uses AWS S3 buckets to:

• read evalscript from a S3 bucket (this is optional because an evalscript can also be provided directly in a request),
• write results of processing to a S3 bucket.

One bucket or different buckets can be used for these purposes.

Bucket regions

The buckets to which the results of asynchronous processing are written must be in the same region as the Async API deployment. The only available region at the moment is eu-central-1 (Frankfurt).

Bucket policy

The IAM user or IAM role (depending on which of access methods described below is used) must have permissions to read and/or write to the corresponding S3 bucket.

The bucket from which the evalscript file is read must have GetObject S3 permission listed in its policy. The bucket where the results are stored must additionally have a PutObject permission in its policy.

Example of a valid bucket policy for the IAM user and a bucket that's used for both reading input data and writing results:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Batch statistical permissions",
      "Effect": "Allow",
      "Principal": {
        "AWS": "<iam-user-arn->"
      },
      "Action": [
        "s3:GetObject",
        "s3:PutObject"
      ],
      "Resource": [
        "arn:aws:s3:::<bucket-name>/*"
      ]
    }
  ]
}

Access your bucket using assume IAM role flow

In order to let Sentinel Hub access the bucket, the recommended option is to provide ARN of your IAM role that has access to the bucket:

s3 = {
  "url": "s3://<your-bucket>/<path>",
  "iamRoleARN": "<your-IAM-role-ARN>",
}

When creating an IAM role, you have the option to add an additional layer of security by specifying the externalId parameter. In case you opt for this, make sure you use your SH domain account id for its value, which you can find in the User settings page in the Dashboard.

If your IAM role is shared among several principals, and you want to distinguish between their activities by setting roleSessionName in trust policy of each principal, set its value for SH principal to sentinelhub.

Example of a trust policy for IAM role specifying permissions for SH user:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::614251495211:root"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "<your-SH-domain-account-id>"
        },
        "StringLike": {
          "sts:RoleSessionName": "sentinelhub"
        }
      }
    }
  ]
}

To give IAM role access to the bucket, you can either add permission policy to the IAM role or set bucket policy in a similar way as above, using your IAM role as principal.

To learn how to create a role for IAM user, check this link.

Access your bucket using accessKey and secretAccessKey

The other option is to provide accessKey and secretAccessKey pair in your request. To learn how to configure an access key and access key secret on AWS S3, check this link, specifically, under the Programmatic access section. Note that IAM user, to which your access key and secret are linked to, must have permissions to read and/or write to the corresponding S3 bucket.

Once you have accessKey and secretAccessKey, you can use them in a s3 object of your request like this:

s3 = {
  "url": "s3://<your-bucket>/<path>",
  "accessKey": "<your-bucket-access-key>",
  "secretAccessKey": "<your-bucket-access-key-secret>",
  "region": "<your-bucket-aws-region>"
}

The above JSON for accessing the S3 bucket can be sent as:

• (optional) evalscriptReference.s3 to specify the bucket where evalscript .js file is available,
• ouput.delivery.s3 to specify the bucket where the results will be stored.

Check Async API reference for more information.

Checking the status of the request

While the request is running, you can get its status (see this example). Once the processing is finished the request is deleted from our system. If you try to check its status after it has been deleted, you will get a '404 Not Found' response even if the request was processed successfully.

Troubleshooting

In case anything goes wrong when creating an Async request, we will return an error message immediately. If anything goes wrong once the Async request has been created, we will deliver an "error.json" file with an error message to your object storage (S3).

Examples

Example of a Async API request