Kapiche provides an Export API that allows you to programmatically fetch enriched data directly from Kapiche for use in your own data warehouse and backend systems.

The API is provided over HTTPS endpoints and is available via an allow list per customer.

At the current time, there is one export data schema available: document-queries.

To make use of the API, two steps are necessary:

With those two steps completed, an HTTPS API call can now be made to the Kapiche API server to obtain export data.

Optional: Specific IP Address prefixes can be added to an allow-list to improve security even further. If your API calls will always be made from a specific IP Address, that IP address, or a common prefix of a range of IP addresses, can be configured.

NOTE: Only an Admin user can complete the steps mentioned above.

In the sections that follow, each of these requirements will be described in more detail.

Let’s assume that the site name of your site is acme. This can be found by looking at your URL in Kapiche once you have logged in to the product:

In the above URL, the site name is acme. Secondly, your Kapiche Admin must obtain a token. For this code example, assume for now that the token is xxx. Finally, when the Export API is enabled on a particular analysis, a base URL for that analysis will be created.

Given those 3 pieces of information, an API call can be made like this, for example using curl:

In the call above,

This type of export is discussed further in the section Available Export Data Schemas.

In the following sections, each of the required components of the API will be explained.

The Admin user for your site can add and delete API tokens on the screen shown below, under the Site Administration section:

When adding a token, the token is presented in a temporary dialog as shown below:

This token is a secret and must be safeguarded like a password. Keep it in your password manager, and only provide it to authorized persons, such as a developer who must write code to access the export API programmatically.

When the dialog is closed, the token will be copied to your clipboard to make it easy for you to paste into your password manager.

NOTE: Kapiche does not store these tokens: only salted, hashed values are stored. Kapiche will not be able to provide a lost token.

If you lose a token, or you need to replace a token that may have been compromised, simply delete tokens in this screen and create a new one. It is good practice to periodically replace tokens with new ones.

When the Export API is enabled for a particular analysis, a base URL will be created that is specific to that analysis:

In the image above, the analysis 18-44 age group in the project grocery has been enabled for the export API. In this image above, the base URL is given as:

This URL must be used for all exports on this analysis. However, this is not the full URL for extracting data. Different kinds of exported data can be obtained by appending an extra component to the URL. For example, to export data in the form of row-based documents including all Saved Themes in that analysis, append document-queries/. The full URL to access that type of export (from this analysis) would be the following:

The different kinds of exports, as well as their configurable options, are described in more detail next.

This export data format contains the following fields (or columns):

This endpoint provides these additional optional configuration parameters:

These are discussed in more detail below.

Pagination

As more data is added to a project, the number of documents will grow. Large datasets must be exported in batches. By default, the batch size is 50 000 documents per call. Each document is identified a document_id integer key that is assigned internally by Kapiche. This key can be used to control the start point for each export API call.

For example, consider the following call:

This call will return a data export starting at document_id 1, and containing up to 50000 documents. The HTTP response of the above call will contain a header indicating the next document id that follows the resulting data.

Here is an example of the response headers:

Automation code can use this information to periodically fetch new data for export. The specific header to use is Kapiche-Next-Document-Id. The value given is what must be specified as the start document id for the next API call, to retrieve the next batch of data.

If there is no new data beyond the given start_document_id then the HTTP response status code will be 204. In an automation setup, that HTTP status code simply means the automation system must wait until new data has been added to the project. Typically an automation is set up to run weekly, or on some similar predictable cadence.

Reshaping

The same data can be provided in a reshaped format where the Saved Themes information is pivoted into a vertical structure. This format can be requested by adding a URL query parameter vertical-themes=true.

In this format, the data is restructured so that each document row is repeated for each Saved Theme that document appears in. In this reshaped format, it is sometimes also convenient to split up theme names into separate columns if a hierarchy has been created in the name.

For example, consider two Saved Themes called Sales // Discount and Sales // Returns. If the URL query parameter theme-level-separator=// is provided, these names will be split into separate columns for each level.

Consider the following example exported data. This is obtained by default from the document-queries export format with no extra parameters:

If the parameter vertical-themes=true is provided in the Export API call, the following structure will be produced:

Note:

Finally, if the additional parameter theme-level-separator=// is provided, theme levels will be split:

The full URL for the above export data structure would look like this: