API Import

API Import

Imports data from an API that returns CSV or JSON data

Connect to any API without code

You don't even need to know what an API is to get started, but it really helps if you do a little homework first. If this is feeling like it's going to be over your head, or if you just want a refresher, please take a look at our API learning series. They are friendly, fun, and informative!

Ready to just get started with a damn API? Try out one of our API Snippets to get a head start!

Something important to remember is that APIs follow general standards, but are all unique. Each one may require slightly different settings. Because APIs are used to allow computer systems to communicate, they are very precise. When reading API docs and setting up API steps in Parabola, you will need to be precise as well.

Choosing a verb for your API Request

The API Import supports POST and GET operations. Also known as verbs.

A GET is a common way to request data from an API. You can send additional data in the URL or as headers, depending on the API.

A POST is another way to request data, though it is more commonly used to make changes, such as adding a new user to a table. Some APIs, such as those that generate reports, will require a POST to request data. In that case, you can also pass request information to the API in a JSON blob in the body of the request.

Sending a body in your API request

A GET cannot send a body in its request. A POST can send a body in its request.

In Parabola, the body of the request will always be sent in JSON. That means you need to make sure your API is happy to accept JSON, and you need to make sure that whatever you put in that body field is valid JSON.

The API Import will show an error if you do not enter valid JSON, but if you need help figuring out why your JSON is invalid, there are a number of online tools that can help you out.

Simple JSON looks like this:

{
   "key1":"value1",
   "key2":"value2",
   "key3":"value3"
}

But it can be nested and have other complications. Luckily, your API docs will most likely show you the JSON that needs to be sent. If you get stuck, reach out.

Connecting to an API that requires authentication

Many API's require some sort of authentication to access their data. If you are unclear about how to authenticate your request to the API, check the documentation for that API. Many times, they will let you know how to authenticate your request.

Built-in authentication methods

A few forms of authentication are built into this source - Bearer Token (also known as token authentication), Username/Password (also known as basic authentication), and OAuth 2.0. If your API says that it needs a token, follow their instructions and copy that token into the field that is shown when you select the Bearer Token option on this source. If their example shows something like Bearer xxxxxxxxxxxxxx then remove the Bearer part when copying the token, since we will add it automatically. If the documentation says to use your login information to authenticate, then choose the Username/Password option and supply that information. OAuth 2.0 is a secure way that APIs authenticate but can be complex to set up. Read our guide to it here: https://learn.parabola.io/docs/how-to-connect-to-an-api-with-oauth-2-authentication

Custom Headers

The API you are trying to access data from may need extra information passed in the header of the request in order to fulfill your request. For example, it may have multiple versions, and may ask you to send something like accept-version : 1.0.0. In that case, click the button to create a Custom Header, and type in the key for it, which, in this case, would be accept-version and then supply the value information of 1.0.0.

The data returned by an API may not look right

JSON API responses can be arranged in many different ways. If you see a response of some data, but not the data you were expecting, or just a single line of data, then you may need to provide a Top Level Key. This is a value which tells the API Import which part of the response you want to unpack. To do so, click the button in the settings that says Add Top Level Key and choose from the keys that were found in your data. Choosing a key like Result usually will fetch the proper data for you once you press Update Settings.

If you are still seeing JSON data after adding Top Level and Nested Keys (keys beneath the Top Level Key for further unpacking), you may need to use the JSON Flattener to further manipulate the JSON data.

Advanced settings in the API Import

Screen Shot 2019-09-03 at 5.45.29 PM

Using pagination to get more results

Many APIs employ some sort of pagination to send back responses that contain many results. You will usually find mention of this in their docs, and it can take a few forms. Parabola supports the most common pagination styles

Most of the Advanced Settings in the API Import have to do with pagination. Some of the settings work together.

Traditional Pagination fields

Pagination Key
Pagination Start Value
Increment Pagination Value

These fields are used together to do pagination that looks like ?page=1. You can define what the name of the pagination parameter is (in this case its page), what the start value is (1 or 0 normally), and the number to use to increment that page value on each successive call (usually 1).

Limit fields

Limit Key
Limit Size

These fields are fairly simple. Many APIs will have a page limit size that you can set. They will default to something like 50 results but support up to 500 per page. If you want to get a different number of results back from the API other than the default, use this field. You will see something in their docs that mention a limit, which would look something like ?page_size=100. In this case, the Limit Key is page_size and the Limit Size is 100.

You can use the Limit fields in combination with any pagination style.

Cursor based pagination fields

Cursor Key
Pagination Key (sometimes)

Cursor based pagination can be done in two ways in Parabola. The first is Full URL Cursors. In this style, the API will send your data back to you and include a URL in the response that can be used to fetch the next page of results. In this flavor, you need to set the Pagination Key to some random string of characters (because it will be ignored).

The second way that cursor based pagination can be used in Parabola is with Cursor parameter values. This flavor of pagination sends back a hash or a value that needs to be sent back as a query parameter in the next request. Something like ?next_page=jsdf9823hrnt-298HA_Hf9n32mf902983ur. You need to set the Pagination Key to the key that the API specifies for this to work.

In both flavors, you need to set the Cursor Key to the path where the pagination URL or cursor parameter value is located in the response of the API.

Setting up the Cursor Key field

When you receive a JSON response from your API that has a pagination cursor in it, it will be somewhere away from the data you care about. Like this:

Setting up the Cursor Key field

When you receive a JSON response from your API that has a pagination cursor in it, it will be somewhere away from the data you care about. Like this:

{
    "response": {
       "data": [
           {
                "id": 1,
                "color": "blue",
                "material": "rubber",
                "size": 5,
                "squeaks": true
            },
            {
                "id": 2,
                "color": "yellow",
                "material": "tennis",
                "size": 9,
                "squeaks": false
            },
            {
                "id": 3,
                "color": "green",
                "material": "tennis",
                "size": 3,
                "squeaks": false
            },
            {
                "id": 4,
                "color": "red",
                "material": "plastic",
                "size": 13,
                "squeaks": true
           },
    ],
    "length": 4,
    "has_more": true,
    "next_page_cursor": "https://myAPI.service.com/resource?next_page=jsdf9823hrnt-298HA_Hf9n32mf902983ur"
  }
}

The field we care about is next_page_cursor as it contains our next URL. Because it has the Full URL, we can use the first flavor of cursor pagination discussed above. That is to set the Pagination Key to anything (it just cannot be blank) and to set the Cursor Key field to the location of this URL. That location is response.next_page_cursor in JSON Dot Notation.

If the response said "next_page_cursor": "jsdf9823hrnt-298HA_Hf9n32mf902983ur" instead, then you would need to set the Pagination Key field to be some key to use with that cursor value. In this case that would be next_page.

JSON Dot Notation, for this use, is a way to tell Parabola how to walk through the JSON to identify the field that want to set as the Cursor Key. You start with the first key, which in this case is response, then to go one level deeper into it, you use a dot ., and then keep going until you end with the key of the value you want.

Max Requests per minute

In the API Import, many requests would be made when using pagination. This field is by default set to 60 requests per minute - that is 1 per second. Every API will have different rate limits that dictate how frequently you can call the API before it starts limiting your usage.

This field is the maximum number of requests that Parabola will send in a minute, not necessarily the actually number. If the API takes 3 seconds to return each page, the API Import will only be able to send 20 requests per minute.

After 60 minutes of working, this step will kill itself if it is still running.

Max Pages to Fetch

This field represents the maximum number of pages that will be pulled into Parabola before the step finishes. You will want this field value to be greater than the number of pages that you are expecting to need to pull in. Default is 5.

Encoding your URL

If you are merging data into your URL, the data that is being merged in could have characters at some point that are not allowed in URLs. That would be things like spaces and accented characters. If you have experienced this, or anticipate it, use the option to Encode URL in the advanced settings.

How to deal with errors from your API

APIs give errors when something is not right. To further understand what to do when you get an error from your API, read this doc.

A hands on guide to connecting to an API

Read about connecting to, troubleshooting, and working with APIs here!

If you are having any trouble connecting to an API, please reach out to us immediately and we will help you get set up.

Start automating with Parabola

Parabola is free to use for teams of any size. We also offer paid plans with additional features and support.