> For the complete documentation index, see [llms.txt](https://docs.veza.com/4yItIzMvkpAvMVFAamTf/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.veza.com/4yItIzMvkpAvMVFAamTf/developers/api/oaa/rest-api/multipart-upload.md).

# Multipart Upload

### Overview

For OAA payloads that exceed 100 MB after compression, you can use multipart upload to split the payload into smaller chunks and upload them sequentially. The Veza platform assembles the chunks server-side before processing.

Multipart upload is useful when:

* The payload is too large for a single HTTP request
* Network reliability is a concern for large transfers
* You need to monitor upload progress for very large data sources

{% hint style="info" %}
The standard `:push` endpoint supports payloads up to 100 MB (compressed or uncompressed). Use multipart upload when your payload approaches or exceeds this limit, or when you need more reliable transfer for large payloads.
{% endhint %}

### How it works

1. **Start the upload** by sending a `start` request — the server creates an upload session and returns an `upload_id`.
2. **Upload chunks sequentially**, each base64-encoding a raw slice of the payload. Include the `upload_id` from step 1 and an incrementing `sequence_number`.
3. **Finalize** by sending a `complete` request with the `upload_id` and total `sequence_count`.

Each chunk must be base64-encoded from raw bytes before sending. The server decodes and reassembles chunks in sequence order when the complete operation is received.

### API endpoint

```
POST /api/v1/providers/custom/{id}/datasources/{data_source_id}:parts
```

#### Start request body

| Field       | Type   | Description     |
| ----------- | ------ | --------------- |
| `operation` | string | Must be `start` |

The response returns an `upload_id` (UUID string) to use in all subsequent requests for this upload session.

#### Chunk request body

| Field             | Type    | Description                                                                    |
| ----------------- | ------- | ------------------------------------------------------------------------------ |
| `operation`       | string  | Must be `upload`                                                               |
| `upload_id`       | string  | UUID returned by the start response                                            |
| `sequence_number` | integer | Sequence number for this chunk, starting from 1. Maximum 99 chunks per upload. |
| `data`            | string  | Base64-encoded chunk of the raw payload                                        |

#### Finalization request body

Send a separate POST to the same endpoint after all chunks are uploaded:

| Field            | Type    | Description                     |
| ---------------- | ------- | ------------------------------- |
| `operation`      | string  | Must be `complete`              |
| `upload_id`      | string  | UUID from the start response    |
| `sequence_count` | integer | Total number of chunks uploaded |

#### Example

```bash
PAYLOAD_FILE="large_payload.json"
CHUNK_SIZE=10485760  # 10 MB per chunk

# 1. Start the upload session
RESPONSE=$(curl -s -X POST "https://$VEZA_URL/api/v1/providers/custom/$PROVIDER_ID/datasources/$DATASOURCE_ID:parts" \
  -H "authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"operation":"start"}')
UPLOAD_ID=$(echo "$RESPONSE" | python3 -c "import sys,json; print(json.load(sys.stdin)['upload_id'])")

# 2. Split into raw chunks and upload each
SEQUENCE=0
split -b $CHUNK_SIZE "$PAYLOAD_FILE" /tmp/oaa_chunk_
for CHUNK_FILE in $(ls /tmp/oaa_chunk_* | sort); do
  SEQUENCE=$((SEQUENCE + 1))
  ENCODED=$(base64 -w 0 < "$CHUNK_FILE")  # use -b 0 on macOS

  curl -s -X POST "https://$VEZA_URL/api/v1/providers/custom/$PROVIDER_ID/datasources/$DATASOURCE_ID:parts" \
    -H "authorization: Bearer $API_KEY" \
    -H "Content-Type: application/json" \
    -d "{
      \"operation\": \"upload\",
      \"upload_id\": \"$UPLOAD_ID\",
      \"sequence_number\": $SEQUENCE,
      \"data\": \"$ENCODED\"
    }"
done

# 3. Finalize the upload
curl -s -X POST "https://$VEZA_URL/api/v1/providers/custom/$PROVIDER_ID/datasources/$DATASOURCE_ID:parts" \
  -H "authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"operation\": \"complete\",
    \"upload_id\": \"$UPLOAD_ID\",
    \"sequence_count\": $SEQUENCE
  }"

rm /tmp/oaa_chunk_*
```

### Using the Python SDK

The `oaaclient` Python SDK handles multipart upload automatically when enabled:

```python
from oaaclient.client import OAAClient

client = OAAClient(url=veza_url, token=veza_api_key)
client.enable_multipart = True

# Push as usual - the SDK automatically chunks large payloads
client.push_application(provider_name, data_source_name, application_object=app)
```

When `enable_multipart` is set to `True`, the SDK automatically switches to multipart for payloads exceeding 50 MB (uncompressed) and:

* Splits the payload into chunks and base64-encodes each one individually
* Starts an upload session and tracks the `upload_id`
* Uploads each chunk sequentially with the correct `sequence_number`
* Sends the completion operation to trigger server-side assembly

### Notes

* All chunks for a given `upload_id` must be uploaded before sending the completion operation.
* You can re-upload a chunk with the same `sequence_number` and `upload_id` if needed — the last upload for a given sequence number is used. All chunks must be present before the completion operation is sent.
* Chunk order is determined by `sequence_number`, not upload order.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.veza.com/4yItIzMvkpAvMVFAamTf/developers/api/oaa/rest-api/multipart-upload.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.