Daily Batches

Especially for larger ledgers where there may be thousands of updates per day, Live Data supports exporting a daily batch file containing all of the changes for the prior calendar UTC day.

📘

Note

Setup of daily batch deliveries is currently done through support. Self-service features are coming soon.

File Format

The files are formatted as gzipped jsonl files, with the extension .jsonl.gz

Schema

The jsonl schema in the files matches the schema defined the Understanding Ledger Data Fields page.

Update Triggers

You can choose any of the support change dates to use when generating the daily file described at Getting Job Change Data.

Delivery Time

Files are usually delivered between 0200 and 1000 UTC the day following the batch report. If you require a specific delivery time or window, please contact support.

Delivery Locations

Daily batches currently support the following delivery location:

S3

Files can be delivered to a Live Data bucket you can access, or can be shipped to a bucket of your choosing. Either way, configure your AWS S3 integration and then contact [email protected] with the ledger, integration name or id, and trigger preference for your batch delivery.

In all cases, files are delivered to the path: ledgers/<ledger_id>/streams/daily/<delivery_date>/<file_name> where:

• ledger_id is the ID of the ledger being delivered
• delivery_date is the date of the batch - the day that all updates in the batch occurred - in the format YYYYMMDD
• file_name is the file name, which includes the batch date as well as a numeric identifier in case the data is split into many files (splits occur at ~300MB uncompressed)