Datasources

Last updated 6 months ago

Datasources

Overview

If you have data available from a RESTful API then you can configure the datasource feature to request this data when a query is made.

Using a datasource has several advantages:

it ensures the data is the most up-to-date
it reduces the questions the reasoning engine might ask
it only makes the request if necessary, therefore if it's a pay-per-use API it can limit costs
it requires less technical effort compared to injecting data

Setting up a datasource

Before you begin, make sure you have API access and reviewed the API documentation for your chosen datasource. Ensure you have details of the endpoints, authentication methods, request parameters and response formats. Setting up a datasource involves the following tasks:

1. Adding a datasource

To add a datasource, click on the canvas and select Add Datasource, give is a friendly name and select the subject of the datasource.

Name

A user-friendly name can be provided for the datasource to make it clearer on the graph. This is optional, if a name is not provided the datasource URL will be used instead.

Subject

A datasource must be linked to the subject of the request. e.g. if the request is about a vehicle, the vehicle reg concept might be selected. If it's about an employee, an employee concept might be selected.

A datasource can only be linked to a concept that is the subject of a relationship. Concepts that are only the object of a relationship will not be available to select in the dropdown.

If you need the data from this concept for the request (e.g. vehicle reg), you can represent it in the request as a variable {{%S}}.

It is common to try and configure a datasource on the concept that you want to apply the data to. i.e. I want to get a credit score from a credit scoring API, so I'll add a datasource to the credit score concept.

In this example the credit score is an object of a relationship Person > has a credit score > Credit Score. Therefore the datasource should be configured on the subject of the relationship and the response will be configured to get the credit score and create a fact.

2. Obtaining additional data required for the request

If the request requires additional dynamic information from the knowledge map, this can be configured by setting up additional input variables.

This allows you to tell the datasource where in the knowledge map the data you need is located, by selecting the appropriate relationship.

For example, you may also need to provide a date range as a request parameter that must be dynamic per request. Expanding the Input Variables section and selecting the relationships that will contain the start date and end date provides a method for getting this data at runtime. This data can then be used in the request by enclosing the variable in double curly braces e.g. {{%START_DATE}}

If the required data is not linked directly to the datasource's subject by a single relationship (as per the example above), you will need to select multiple relationships that describes the path to the required data.

The example below shows how the Organisation ID can be used in the request, even though it is not directly linked to the datasource subject of Employee, by selecting the two relationships needed to reach the required data.

3. Configuring the API request

Once you have setup your datasource to obtain all the data required for the request, you can now configure the request itself.

Consult the API documentation to ensure you are aware of the API endpoints, request parameters, authorisation and headers.

It is best practice to use an external tool such as Postman to construct and test the API request with static data to confirm the request works and that it returns the expected data.

You can then transfer this configuration into Rainbird and replace the static data with the correct variables.

URL

Configure the method, hostname and path for your datasource request.

The methods supported include GET (default) and POST.

To use variables within the request, ensure they are enclosed in double-curly braces e.g. {{%VARIABLE}}

CDATA

If required for your datasource, CDATA can be provided with the request. Consult your API documentation to determine if this is a requirement.

Headers

If required for your datasource, configure the necessary headers. Variables can be used as header values.

4. Generating facts from the response

The purpose of a datasource is to generate facts from this external source, which the reasoning engine can use to make decisions.

However, the data needs to be mapped from the API response (most likely a JSON structure) into the structure required by your knowledge map (subject>relationship>object). This is where the mapping from JSON to your knowledge map is configured.

From the Output facts section you can select the relationships you want to create facts for. Facts can only be created for relationships that share the same subject as the datasource, therefore the dropdown will only display applicable relationships. The subject is disabled, as values from the API response can only be applied to the object.

In the object field, you need to configure the path to the value you require from the API response. This uses a JSON path utility to navigate and filter the response to obtain the value you need for the object in question.

It is recommended to have an example of the expected API response so you can identify where in the structure you need to obtain the data.

There are many free tools available online to view the response data so it's easy to read and navigate, such as JSON Viewer .

There are also tools such as JSON Path Finder, which displays the response, allows you to select the value you need and will build the path for you. (Please note: these will create a path using dot notation which isn't supported, but changing the . to / should be sufficient. An example of this tool showing the (unsupported) dot notation is shown below:

To learn how to use datasources with some practical examples, head over to the Academy.