API Umbrella Documentation¶

What Is API Umbrella?¶

API Umbrella is an open source API management platform for exposing web service APIs. The basic goal of API Umbrella is to make life easier for both API creators and API consumers. How?

Make life easier for API creators: Allow API creators to focus on building APIs.
- Standardize the boring stuff: APIs can assume the boring stuff (access control, rate limiting, analytics, etc.) is already taken care if the API is being accessed, so common functionality doesn’t need to be implemented in the API code.
- Easy to add: API Umbrella acts as a layer above your APIs, so your API code doesn’t need to be modified to take advantage of the features provided.
- Scalability: Make it easier to scale your APIs.
Make life easier for API consumers: Let API consumers easily explore and use your APIs.
- Unify disparate APIs: Present separate APIs as a cohesive offering to API consumers. APIs running on different servers or written in different programming languages can be exposed at a single endpoint for the API consumer.
- Standardize access: All your APIs are can be accessed using the same API key credentials.
- Standardize documentation: All your APIs are documented in a single place and in a similar fashion.

Getting Started

Installation¶

Installing From Binary Packages¶

Debian 9 (Stretch)

$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 367404D553B42995
$ echo "deb http://dl.bintray.com/nrel/api-umbrella-debian stretch main" | sudo tee /etc/apt/sources.list.d/api-umbrella.list
$ sudo apt-get update
$ sudo apt-get install api-umbrella

Debian 8 (Jessie)

$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 367404D553B42995
$ echo "deb http://dl.bintray.com/nrel/api-umbrella-debian jessie main" | sudo tee /etc/apt/sources.list.d/api-umbrella.list
$ sudo apt-get update
$ sudo apt-get install api-umbrella

Enterprise Linux 7 (CentOS/RedHat/Oracle/Scientific Linux)

$ curl https://bintray.com/nrel/api-umbrella-el7/rpm | sudo tee /etc/yum.repos.d/api-umbrella.repo
$ sudo yum install api-umbrella

Enterprise Linux 6 (CentOS/RedHat/Oracle/Scientific Linux)

$ curl https://bintray.com/nrel/api-umbrella-el6/rpm | sudo tee /etc/yum.repos.d/api-umbrella.repo
$ sudo yum install api-umbrella

Ubuntu 18.04 (Bionic)

$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 367404D553B42995
$ echo "deb http://dl.bintray.com/nrel/api-umbrella-ubuntu bionic main" | sudo tee /etc/apt/sources.list.d/api-umbrella.list
$ sudo apt-get update
$ sudo apt-get install api-umbrella

Ubuntu 16.04 (Xenial)

$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 367404D553B42995
$ echo "deb http://dl.bintray.com/nrel/api-umbrella-ubuntu xenial main" | sudo tee /etc/apt/sources.list.d/api-umbrella.list
$ sudo apt-get update
$ sudo apt-get install api-umbrella

Installing With Chef¶

If you use Chef for managing your servers, we provide a cookbook that can be used to install the binary packages and configure API Umbrella.

Running With Docker¶

In this simple example, custom API Umbrella configuration can be defined in the config/api-umbrella.yml file on host machine. This gets mounted as /etc/api-umbrella/api-umbrella.yml inside the container, which is the path for the configuration file the rest of the documentation will reference.

$ mkdir config && touch config/api-umbrella.yml
$ docker run -d --name=api-umbrella -p 80:80 -p 443:443 -v "$(pwd)/config":/etc/api-umbrella nrel/api-umbrella

Installing From Source Code¶

Installing from a binary package is recommended, if available (let us know if you’d like to see binary packages for other platforms). However, if a binary package is not available you can compile from source.

Setup¶

Start API Umbrella:
```
$ sudo /etc/init.d/api-umbrella start
```
Browse to your server’s hostname. You should land on the default homepage:

Default API Umbrella homepage

Congrats! You’re now up and running with API Umbrella. There are a variety of things you can do to start using the platform. Read on for a quick tutorial:

Quick Tutorial¶

Add API Backends¶

Out of the box, API Umbrella doesn’t know about any APIs. You must first configure the API backends that will be proxied to.

In this example, we’ll proxy to Google’s Geocoding API (but you’ll more likely be proxying to your own web services).

Step 1: Login to the web admin and navigate to the “API Backends” section under the “Configuration” menu.

Step 2: Add a new backend:

_images/add_api_backend_example.png Add API Backend Example

Step 3: Navigate to the “Publish Changes” page under the “Configuration” menu and press the Publish button.

Google’s API should now be available through the API Umbrella proxy.

Make an API request¶

Assuming you added the Google Geocoding example as an API backend, you should now be able to make a request to Google’s Geocoding API proxied through your local API Umbrella instance:

http://your-api-umbrella-host/google/maps/api/geocode/json?address=Golden,+CO&sensor=false&api_key=**YOUR_KEY_HERE**

You can see how API Umbrella layers its authentication on top of existing APIs by making a request using an invalid key:

http://your-api-umbrella-host/google/maps/api/geocode/json?address=Golden,+CO&sensor=false&api_key=INVALID_KEY

View Analytics¶

Login to the web admin. Navigate to the “Filter Logs” section under the “Analytics” menu. As you make API requests against your API Umbrella server, the requests should start to show up here (there may be a 30 second delay before the requests show up in the analytics).

_images/analytics.png Analytics

Next Steps¶

API Backends¶

Caching¶

API Umbrella provides a standard HTTP caching layer in front of your APIs (using Apache Traffic Server). In order to utilize the cache, your API backend must set HTTP headers on the response. In addition to the standard Cache-Control or Expires HTTP headers, we also support the Surrogate-Control header.

Surrogate-Control¶

The Surrogate-Control header will only have an effect on the API Umbrella cache. This header will be stripped before the response is delivered publicly.

Surrogate-Control: max-age=(time in seconds)

Cache-Control: s-maxage¶

The Cache-Control: s-maxage header will be respected by the API Umbrella cache, as well as any other intermediate caches between us and the user.

Cache-Control: s-maxage=(time in seconds)

Cache-Control: max-age¶

The Cache-Control: max-age header will be respected by the API Umbrella cache, intermediate caching servers, and the user’s client.

Cache-Control: max-age=(time in seconds)

Expires¶

The Expires header will be respected by the API Umbrella cache, intermediate caching servers, and the user’s client.

Expires: (HTTP date)

HTTP Headers¶

Request Headers¶

After API Umbrella verifies an incoming request (a valid API key, below rate limits, etc), it will then proxy the incoming request to your API backend. The request your API backend receives will have additional HTTP headers added to the request. These headers can optionally be used to identify details about the requesting user.

X-Api-User-Id¶

A unique identifier for the requesting user (UUID format). This should be used if your API backend needs to uniquely identify the user making the request.

X-Api-User-Id: (UUID)

Example:

X-Api-User-Id: d44a13a0-926a-11e3-baa8-0800200c9a66

X-Api-Roles¶

If the user accessing the API has roles assigned to them, these will be present in the X-Api-Roles header as a comma-separated list of roles:

X-Api-Roles: (comma separated list)

Example:

X-Api-Roles: write_permissions,private_access

X-Forwarded-For¶

Used for identifying the original IP address of the client. See X-Forwarded-For for usage and details.

X-Forwarded-For: (comma separated list)

Example:

X-Forwarded-For: 203.0.113.54, 198.51.100.18

X-Forwarded-Proto¶

The original protocol of the client’s request (either http or https). This can be used to determine how the client originally connected to the API regardless of what protocol is being used for API backend communication.

X-Forwarded-Proto: (http or https)

Example:

X-Forwarded-Proto: https

X-Forwarded-Port¶

The original port of the client’s request (for example, 80 or 443). This can be used to determine how the client originally connected to the API regardless of what port is being used for API backend communication.

X-Forwarded-Port: (number)

Example:

X-Forwarded-Port: 443

X-Api-Umbrella-Request-Id¶

A unique string identifier for each individual request. This can be used in log data to trace a specific request through multiple servers or proxy layers. This same header will be set as a response header that the API consumer will receive for tracing purposes too.

X-Api-Umbrella-Request-Id: (unique string identifier)

Example:

X-Api-Umbrella-Request-Id: aelqdj9lfoe7c2itheg0

X-Api-Key (Deprecated)¶

Currently the API passed in by the user is passed along to API backends in the X-Api-Key header. However, this header is deprecated and will be removed in the future. Instead, the X-Api-User-Id should be used if you need to uniquely identify the requesting user.

X-Api-Key: (api key)

Example:

X-Api-Key: 5WH3bgykjP9ihtrRl5ib9nQY5NzUGOixdXjBnx18

Response Headers¶

API Umbrella may add certain HTTP headers to the public response, or other headers can be set by the API backend for internal usage by API Umbrella.

X-Api-Umbrella-Analytics-Custom1, X-Api-Umbrella-Analytics-Custom2, X-Api-Umbrella-Analytics-Custom3¶

These 3 optional response headers can be set by the API backend and will be logged by the API Umbrella analytics system. These can be used to log custom, application-specific values to the analytics system for later querying and analysis. These headers will be stripped and removed by API Umbrella before returning to the API consumer, so the values will not be part of the public HTTP response. If the HTTP header value length exceeds 400 characters, only the first 400 characters will be logged in the analytics system.

X-Api-Umbrella-Analytics-Custom1: (any value)
X-Api-Umbrella-Analytics-Custom2: (any value)
X-Api-Umbrella-Analytics-Custom3: (any value)

Example:

X-Api-Umbrella-Analytics-Custom1: my-custom-value

X-Api-Umbrella-Request-Id¶

A unique string identifier for each individual request. This can be used in log data to trace a specific request through multiple servers or proxy layers. This header will be returned to the API consumer and is not something API backends can change. This same header will be set as a request header that the API backend will receive for tracing purposes too.

X-Api-Umbrella-Request-Id: (unique string identifier)

Example:

X-Api-Umbrella-Request-Id: aelqdj9lfoe7c2itheg0

Role Restrictions¶

API Umbrella’s “roles” feature can be used to restrict access to APIs so that only certain API keys may access certain APIs.

Adding Roles to API Keys¶

To grant specific API keys a role:

In the Admin, under Users > API Users, find the API key you want to add roles to.
Under Permissions > Roles, enter roles to assign to this API key.
- You can name roles however you’d like.
- Existing roles will auto-complete, but new roles can be created by entering a new name.

Enforcing Role Requirements With API Umbrella¶

If you’d like for API Umbrella to enforce role restrictions, then role requirements can be defined within the API Backend configuration:

In the Admin, under Configuration > API Backends, choose your API Backend to edit.
Under Global Request Settings > Required Roles, enter roles to require.
- You can name roles however you’d like.
- Existing roles will auto-complete, but new roles can be created by entering a new name.
- If multiple roles are set, then the API key must have all of the roles.
Save changes to the API Backend and publish the configuration changes.

Once configured, then only API keys with the required roles will be able to access your API backend. If an API key lacks all of the required roles then API Umbrella will reject the request with a 403 Forbidden error and your API will never receive the request.

Sub-URL Role Requirements¶

The API Backend’s “Sub-URL Request Settings” can be used to define more granular role requirements. For example, this could be used to require roles on just a single API URL path, or to only require roles for POST/PUT write access.

Enforcing Role Requirements Inside Your API¶

Instead of enforcing role requirements at the API Umbrella proxy layer, you can also utilize the role information in other ways within your API backend. If you have more complex authorization logic, then this may be easier to implement within your API’s code.

On each request with a valid API key that’s passed to your API backend, there’s an X-Api-Roles HTTP header. This contains a comma-delimited list of all the roles assigned to the API key that’s making the request. Your APIs can parse this HTTP header and use it to decide whether access should be permitted or denied.

API Users¶

Roles¶

API Umbrella’s “roles” feature can be used to restrict access to APIs so that only certain API keys may access the API.

See the API Backends’ Role Restrictions documentation for more detail.

Admin Accounts¶

Permissions¶

API Scopes are combined with the Admin Groups to create a granular permission system within the API Umbrella admin. This might be useful if you have multiple organizations or departments that should only have access to certain parts of the API Umbrella admin.

An API Scope defines a hostname and a path prefix. This determines the API backends and analytics that an admin is allowed to view. For example, an admin may be authorized to interact with example.com/foo/* apis, but not example.com/bar/* apis.

Next, you setup a (permissions) group, which defines the specific permissions admins can perform within API scopes. For example, you may want some admins to only be able to view analytics, while others should be able to also setup API backends.

As a quick example, say you set up an API Scope with a host of example.com and a path prefix of /foo. You then create a group that uses that scope and grants the Analytics and API Backend Configuration - View & Manage permissions. Then, you assign that group to a specific admin account.

Now, any admin that belongs to that group can log in and view analytics, but only for requests beginning with example.com/foo/*. They would not be able to view analytics for example.com/bar/*. Similarly, because they were granted the API Backend permission, that user could edit or create new API backends, but only as long as the API backend they’re interacting with starts with example.com/foo/* for it’s public URL. However, while this specific admin group could add and edit API backends, they couldn’t actually publish the backend changes and make them live, since they were not granted the API Backend Configuration - Publish permission.

Analytics¶

Website Backends¶

By default, API Umbrella ships with a very basic public website. Any URL that does not match the routes defined by your API backends will get routed to your website backend. The default website provided by API Umbrella is intended to be customized or replaced. There are several different approaches to managing and hosting your website content:

Using an External Website Backend
Using the Example Static Site

External Website Backends¶

If you already have a website or content management system you’d like to use for managing your API website, you can point API Umbrella’s website backend to wherever your website is hosted:

In the API Umbrella admin, under the “Configuration” menu pick “Website Backends” and then click “Add Website Backend”
Fill in the details for where your underlying website backend is hosted.
- The “Frontend Host” field can be used if you’d like API Umbrella to handle multiple domain names and present different websites for each domain.
Save your website backend, and publish the changes under Configuration > Publish Changes.

Example Static Site¶

If you don’t already have a preferred way for managing your websites, API Umbrella ships with a basic, example website. The default API Umbrella website content comes from the api-umbrella-static-site repository. This repository provides a Middleman site that can be forked and customized. As a static site, this site will compile to static HTML files and can be hosted in a variety of simple locations (GitHub Pages, an S3 Bucket, or on the API Umbrella servers).

Deployment¶

If you fork and customize the static site repository, you can then deploy it in a variety of ways. A few examples include:

External (GitHub Pages, S3 Bucket, etc): You can deploy the resulting HTML files to these external locations, and then configure API Umbrella to point to these locations as you would any external website backend.
On the API Umbrella servers: You’ll need to configure your API Umbrella servers to allow for deployments, then you’ll need to adjust the deploy configuration in config/deploy/production.rb to point to your servers, and then you should be able to deploy via cap production deploy.

Example Forks¶

Here are a couple examples of website repositories based on the api-umbrella-static-site repo, and deployed with GitHub Pages:

REST API¶

Admin Authentication¶

By default, API Umbrella’s admin can be accessed with local admin accounts that can be created and managed without any further configuration.

The admin can also be configured to authenticate using external login providers (like Google, Facebook, or LDAP). These external login providers can be used in combination with local admin accounts, or local accounts can be disabled and external providers can be used exclusively.

General Configuration¶

Custom authentication settings can be defined in /etc/api-umbrella/api-umbrella.yml. The following example shows some general configuration options:

web:
  admin:
    auth_strategies:
      enabled:
        - local
        - github
        - google
    initial_superusers:
      - your.email@example.com
    username_is_email: true

web.admin.auth_strategies.enabled: An array of authentication providers that should be enabled for logging into the admin (defaults to local). Available providers:
- local
- cas
- facebook
- github
- gitlab
- google
- ldap
- max.gov
web.admin.initial_superusers: An array of superuser admin accounts that should be created on startup (defaults to none). Subsequent admin accounts can be created via the admin interface, so this setting is only needed for initial setup.

When the local login provider is enabled (default), you will be given an opportunity to create an admin account on your first visit to the admin tool, so this option should not be necessary (unless you want to prevent the first visitor from being allowed to create an admin account).

This option is primarily useful when the local login provider is disabled and you’re exclusively using external login providers.
web.admin.username_is_email: Whether or not the admin’s username is also their email address (defaults to true). Setting this to false allows for a separate non-email based username to be assigned to admin accounts.