API calls for managing and updating custom data sources
Use these REST API calls to manage and update custom providers and data sources with Open Authorization API.
Creates a custom provider and returns the provider ID.
Lists all configured custom providers.
Returns details for an individual custom provider.
Delete a custom provider by ID.
Return all data sources for a Custom Provider ID.
You can constrain large responses by adding a filter to the request query string. Include the operator (eq), and value, for example:
CURL <VEZA_URL>/api/v1/providers/custom?filter=name eq "GitHub"&order_by=state
Register a new datasource for a custom provider. There can be more than one datasource for a single provider.
Returns details for a single datasource.
Unbind a datasource from a custom provider, and delete it.
To push authorization metadata for a custom datasource, you can specify the source and provider IDs, and upload a payload with the entities and permissions in JSON format.
A warning is returned for any non-critical errors during payload processing. These can indicate incomplete or inaccurate data in the payload that do not prevent processing, but may warrant attention.
CSV data must base64 encoded into the JSON body of the request.
The populated template can be compressed and encoded, for significantly reduced payload size.
Specify the compression_type. Currently supported: GZIP.
If compression is selected, Veza will expect the payload json_data as a compressed, base64-encoded string.
To compress using shell commands:
Size is typically not an issue when updating custom datasources. However, you may want to compress large payloads. The maximum body size is 100MB (compressed or uncompressed).
Veza expects the populated template as a single JSON string, enclosed in the request body json_data field. Any "s and non-ASCII characters must be escaped.
To convert a template to JSON string using Python, the json.dumps() method could be used:
You can optionally add an icon for your custom provider by uploading a PNG or SVG file (less than 64kb) as a base64-encoded string:
Upload a custom icon to display for an OAA provider.
Return the type and string-encoded icon for a custom provider.
Delete the icon associated with an OAA provider.
For , this endpoint pushes CSV data to an existing datasource. Typically, you will first create the integration and define column mappings using the "Add Integration" flow in Veza.
Methods for working with Custom Data Providers and Sources
The populated template can describe additional resources and sub-resources, such as individual databases, repositories, or views.
Follow best practices for managing API keys: Do not save credentials in connector code. Consider using a secrets manager to securely store API keys and make them available at run time.
To add a custom application, you will need to:
Create a new custom provider and data source.
push the entity and authorization data in a JSON payload.
The response will return the Provider's ID, which you will need to create and manage the data sources:
Name the provider generically after the application or SaaS provider. Use the same Provider for all data sources for that application.
If you are creating providers dynamically, your OAA integration should check if the provider and data source exists before creating a new one.
The response will include the data source ID:
Datasources should be unique to the data collected by an OAA integration instance. For example, if an application has a "prod" and "dev" instance, creating a datasource for each will enable independent updates to each environment.
Name the data source uniquely based on the application instance to discover. Try to include the hostname or organization name in the data source. For example, don't use "GitHub" use "GitHub - Acme Inc" or "Portal - prod.corp.com"
Note that a provider id is required in both the request path and body.
json_data must contain the populated OAA template as a single JSON string (escaping any unsafe characters such as ").
This document provides a basic overview of the API requests for creating and updating an OAA data source. These steps and can be adapted for your client or programming language of choice. You can also use the oaaclient Python module to handle Veza authentication, register the data source, and push the populated template.
While registering sources and pushing authorization metadata with is relatively straightforward, it is important to understand how Veza organizes custom providers and data sources as endpoints:
You will first register a new custom application provider with a (specifying the app name and template).
The determines the application type of the provider's custom data sources (identity_provider or application).
Each custom provider can have one or more data sources (such as different instances or domains), generated using .
You can for each custom data source.
All custom data sources are shown on the Configuration > Apps & Data Sources menu, and can be retrieved using .
You should typically name the provider based on the generic application provider (such as GitHub) and the data source after the top-level instance (such as GitHub - Organization). See the for more information about parsing and identifying entities using metadata from the source application.
Your requests will need to include a Veza API Key. For OAA APIs, using a is recommended. Provide it as the bearer auth token in the header of each request, for example:
Use to register a new top-level custom provider:
Each provider needs at least one data source. Create one with
Once the data source and provider are active, publish the payload with . The request body must include the Provider ID and Data Source ID.
Specifying the Provider ID and Data Source ID, perform the same used for the initial push.
To update an existing data source, use the operations and operations to get the provider and data source IDs.
The maximum number of results to be returned. Fewer results may be returned even when more pages exist.
The token specifying the specific page of results to retrieve.
The maximum number of results to be returned. Fewer results may be returned even when more pages exist.
The token specifying the specific page of results to retrieve.