Pull from an API

Pull from an API

Pulls data from an API that returns CSV or JSON data

Connect to any API without code

You don't even need to know what an API is to get started, but it really helps if you do a little homework first. If this is feeling like it's going to be over your head, or if you just want a refresher, please take a look at our API learning series. They are friendly, fun, and informative!

Something important to remember is that APIs follow general standards, but are all unique. Each one may require slightly different settings. Because APIs are used to allow computer systems to communicate, they are very precise. When reading API docs and setting up API steps in Parabola, you will need to be precise as well.

The first thing to do

You first need to find and open the API documentation for the service that you want to connect to. If their docs are complicated, you may need to navigate around until you find something called an API Reference, or something similar. You know you are in the right place if there are lots of URLs in the page content, and blocks that look like code.

Here are some examples of API references to help you find what you are looking for:

Spotify's API reference

Pull from Stripe step's API docs

Shopify's API reference

The Pull from an API step settings explained

These are the basic settings for a Pull from an API step. There are more under the Advanced Settings section, but first we will discuss the non-advanced options.

Request Type

You have two options - GET and POST. In the docs for the API you are trying to connect to, you should see these exact words used.

Most likely, you will use a GET.

If you want to GET all artists from Spotify, you would see this in their docs, which tells you to use a GET:

More details

A GET is a common way to request data from an API. You can send additional data in the URL or as headers, depending on the API.

A POST is another way to request data, though it is more commonly used to make changes, such as adding a new user to a table. Some APIs, such as those that generate reports, will require a POST to request data. In that case, you can also pass request information to the API in a JSON blob in the body of the request.

API Endpoint URL

APIs use URLs just like a website does. Each URL represents a different thing you can fetch or modify with the API.

Your docs will have URLs associated with different things that you can request. For example, Spotify has URLs for artists, albums, songs, and more. That makes sense, since those are the types of things that Spotify works with.

An API URL will most likely give you clues about what it can do. Something like https://api.spotify.com/v1/artists looks like it will list out all artists, whereas https://api.spotify.com/v1/artists/{id} looks like it requires an artist ID, and that it will give information about a specific artist, not all artists, since an ID was included.

You can only access API URLs that exist in the service's docs. So everything you need will be in that API reference!

Authentication

Most API's require some sort of authentication to access their data. API docs will usually have a section on this. Try searching for the word Authentication in the docs pages.

A few forms of authentication are built into this source:

Bearer Token

Use this when your API docs say to send your API token or API key, or however they refer to it, as a bearer token. Or, if you see this:

The part that indicates it is a bearer token is this:

-H "Authorization: Bearer sk_test_WiyegCaE6iGr8eSucOHitqFF"

Username/Password

This is most commonly called Basic Authentication or just Basic. If you see the word basic around an authentication section, then you should use this type.

Most of the time, you just put in the username and password that you have for this service, and they use it to authentication you.

Sometimes, an API may tell you to send your API key as a username or a password or both. In this case, put the API key in the appropriate field, and if the other is not mentioned, leave it blank.

Many times, if the API docs are showing you a cURL example, you will see something like this:

Notice there is the URL, followed by:

-u sk_test_WiyegCaE6iGr8eSucOHitqFF:

That is what tips you off that this is Basic Auth. The -u part signifies Basic Auth, and then the colon at the end tells you this is being sent as a username. In normal Basic Auth, you send the username and password in the following format username:password so if you see a -u then you know that anything to left of the colon is the username and to the right is the password. In this example, the API key is sent as a username.

OAuth2.0

This is a secure way that APIs authenticate but can be complex to set up. Read our guide to it here: https://learn.parabola.io/docs/how-to-connect-to-an-api-with-oauth-2-authentication

You will see the words OAuth2.0 in your docs if you need to use this. Usually there will be a whole guide to it, such as the one Spotify has.

Other ways to authenticate with an API

There are other ways that many APIs use to authenticate. Sometimes they just want the API key in the URL, like how Parsehub does it. In that case, you would do something like this: https://www.parsehub.com/api/v2/projects?api_key=t46_TcQaFFT

Or your API may ask for the API key in a specific header, such as how Zoho CRM does it. They say:

Which tells you that you need to use a Custom Header called Zoho-oauthtoken to pass in the API token to their API.

Custom Headers

APIs have things called Headers. They are just extra information that is sometimes needed. In the documentation for your service, you will either see the word Header used, or you will see a cURL example with a -H option, if the API requires a custom header or two.

In Webflow's API docs, you can see that they need a header to tell the API which version you are using. They also are showing the API key sent as a header. But wait! That header looks familiar. That is actually a Bearer Token! So the version is the only header that we need to send.

The name of the header is accept-version and. its value is 1.0.0. In Parabola, if you don't see the name of the header you need in the dropdown list, just type it in, or paste it in, and click on it to set a new custom header name.

Sending a POST body in your API request

A GET cannot send a body in its request. A POST can send a body in its request.

In Parabola, the body of the request will always be sent in JSON. That means you need to make sure your API is happy to accept JSON, and you need to make sure that whatever you put in that body field is valid JSON. If you see JSON In the API docs, then it probably accepts JSON.

The Pull from an API step will show an error if you do not enter valid JSON, but if you need help figuring out why your JSON is invalid, there are a number of online tools that can help you out. I use this one.

Simple JSON looks like this:

{
   "key1":"value1",
   "key2":"value2",
   "key3":"value3"
}

But it can be nested and have other complications. Luckily, your API docs will most likely show you the JSON that needs to be sent. If you get stuck, reach out.

JSON in API docs looks like this:

Nested or Top Level Keys

When you set up an API Pull, and it successfully pulls data into your flow, you may see an option appear inside of this dropdown. It will not be there at first, and is only present if it can be used.

If you see an option in this menu, use it. If, when you use it, a new button shows up underneath this field that says Add Nested Key use that as well.

If you see many options in the dropdown, choose the column name that you care most about.

This option unpacks your JSON arrays into nice, easy to use tables.

Advanced settings in the Pull from an API step

You will never need to use all of these fields, but sometimes you will need to use a few!

Getting more data than you are currently getting (Pagination)

APIs like to send data back in pages. By default, you only get 1 page. You will need to ask for me.

In your API docs, there is probably a section called Pagination or that uses that word. Search for it.

Page based pagination

If you see the mention of a numerical page number that you increment to get each page, then you are in the right place.

Intercom uses this style of pagination. And it looks like this in their docs:

The first page from this endpoint would look like this: https://api.intercom.io/users?page=0

In Parabola that would be set up with these settings:

That will fetch the first 5 pages from that URL. With those 3 settings in Parabola, you are naming the parameter page, setting it initially to 0, and then incrementing it by 1 each time.

Offset and Limit based pagination

If you see the word offset in your API docs, this is the style for you.

Spotify uses this style. Their docs say:

They want us to declare a limit of how many items to have in each page, and then use an offset to cycle through those pages. In this case, you will set up Parabola like this:

We are using the first 3 settings to say that we have a parameter called offset which starts at 0 and goes up by 10 each time. And we have another parameter called limit that is set to 10.

Cursor based pagination

Sometimes APIs use something that looks weird for their pagination. Each call may return a cursor key to use to get the next page. Something like "nextPageCursor": "b342f5367c664d3c99aa56f44f95ab0a" or "nextPageUrl": "https://api.squarespace.com/1.0/commerce/inventory?cursor=b342f5367c664d3c99aa56f44f95ab0a"

Squarespace sends back a full URL as well as a cursor to use to get the next page. Their docs show this:

And they show:

You can send this in two ways. Either use the cursor or use the full URL.

If you use the cursor, you need to send it in a parameter called cursor, so in Parabola that would look like this:

If you want to use the full URL, you would replace the Cursor Key with pagination.nextPageURL

Max Requests per minute

In the Pull from an API step, many requests would be made when using pagination. This field is by default set to 60 requests per minute - that is 1 per second. Every API will have different rate limits that dictate how frequently you can call the API before it starts limiting your usage.

This field is the maximum number of requests that Parabola will send in a minute, not necessarily the actually number. If the API takes 3 seconds to return each page, the Pull from an API  step will only be able to send 20 requests per minute.

After 60 minutes of working, this step will stop running.

Max Pages to Fetch

This field represents the maximum number of pages that will be pulled into Parabola before the step finishes. You will want this field value to be greater than the number of pages that you are expecting to need to pull in. Default is 5.

Encoding your URL

If you are merging data into your URL, the data that is being merged in could have characters at some point that are not allowed in URLs. That would be things like spaces and accented characters. If you have experienced this, or anticipate it, use the option to Encode URL in the advanced settings.

How to deal with errors from your API

APIs give errors when something is not right. To further understand what to do when you get an error from your API, read this doc.

A hands on guide to connecting to an API

Read about connecting to, troubleshooting, and working with APIs here!

If you are having any trouble connecting to an API, please reach out to us immediately and we will help you get set up.

Start automating with Parabola

Parabola is free to get started. Learn more about our pricing here.