Polling for new reports

A scheduled job usually only needs to know whether Datalink has generated a new report since its last run. The /datalink/report endpoint lets it check without downloading anything.

Check for a new report

A HEAD request returns the response headers of the newest report, without a body. The X-Xeneta-Report-Id header tells you which report is the newest.
A GET or HEAD request with the If-None-Match: "<report_id>" header returns 304 - Not Modified for as long as that report is still the newest. Once a newer report is available, GET returns 200 with the new report object, and HEAD returns 200 with its headers.

The If-None-Match header accepts the ETag value exactly as it was returned, for example "jB47D", as well as weak (W/"jB47D") and comma-separated values.

Neither a HEAD request nor a 304 response creates a download link.

📘
Before the first report
A configuration that has not generated a report yet returns 404 with the code no_report_yet and a Retry-After: 300 header. Be sure to wait at least that many seconds before asking again.

Polling workflow

Store the report_id of the last report you downloaded.
On each run, send a GET request with the If-None-Match: "<report_id>" header.
On 304, there is nothing new, so you can stop.
On 200, download the file from url within 300 seconds, and then store the new report_id.
On 404 with the code no_report_yet, try again after the number of seconds in the Retry-After header.

Check with curl

curl -sS -I -H "X-Auth: $XENETA_API_KEY" -H 'If-None-Match: "jB47D"' \
  "https://api.xeneta.com/v3.0/datalink/report?configuration_id=xArk"

A 304 response means there is nothing new. A 200 response with a new ETag means a newer report is available, so request it with GET and download it. A 404 response with the code no_report_yet means you should try again later.

Sample polling script

The sample Python script below follows the workflow above. It keeps the report_id of the last downloaded report in a local file and downloads a report only when a newer one is available.

📘
This script serves as an example and is not meant to be used as-is, so be sure to adjust it to your specific needs.

import os
import pathlib
import sys

import requests

API_URL = "https://api.xeneta.com/v3.0/datalink/report"
CONFIGURATION_ID = "xArk"
STATE_FILE = pathlib.Path("last_report_id.txt")

headers = {"X-Auth": os.environ["XENETA_API_KEY"]}
if STATE_FILE.exists():
    headers["If-None-Match"] = f'"{STATE_FILE.read_text().strip()}"'

resp = requests.get(
    API_URL, params={"configuration_id": CONFIGURATION_ID}, headers=headers, timeout=60
)

if resp.status_code == 304:
    print("No new report")
    sys.exit(0)

if resp.status_code == 404 and resp.json()["detail"]["code"] == "no_report_yet":
    print(f"No report yet, try again in {resp.headers['Retry-After']} seconds")
    sys.exit(0)

resp.raise_for_status()
report = resp.json()

file_path = pathlib.Path(f"{report['report_id']}.{report['format']}")
with requests.get(report["url"], stream=True, timeout=60) as download:
    download.raise_for_status()
    with file_path.open("wb") as file:
        for chunk in download.iter_content(chunk_size=1024 * 1024):
            file.write(chunk)

STATE_FILE.write_text(report["report_id"])
print(f"Downloaded {file_path}")