Introduction

Airbyte doesn't support SSL encryption at the time of writing this article. To secure Airbyte in a production environment, you need to deploy it behind a reverse proxy like NIGNX and let the reverse proxy handle the encryption to the outside world. This guide will cover the installation of Docker, Airbyte and NGINX. To follow this guide, you need the following prerequisites:

• Ubuntu 22.04

• SSL certificate

• DNS entry for Airbyte deployment (e.g. airbyte.<your-domain>.com)

Docker

Docker is required to run Airbyte. If you have already installed Docker Engine & Docker Compose, then you can skip this section.

Note: Don't use the Snap version of Docker that can be installed during the Ubuntu Server installation process. This version is known to cause all kinds of problems with Airbyte.

Docker installation

To install Docker Engine and Docker Compose (plugin), follow the guide on the docker website or the instructions below (commands are from the docker website but may change in the future).

# uninstall old versions
sudo apt-get remove docker docker-engine docker.io containerd runc

# install dependencies
sudo apt-get update
sudo apt-get install ca-certificates curl gnupg lsb-release

# add GPG key
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

# set up repository
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

# Install Docker Engine, containerd, Docker Compose
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin

Verify Docker installation

To verify you have set up Docker correctly, run the hello-world image.

sudo docker run hello-world

# output
Hello from Docker!
This message shows that your installation appears to be working correctly.

If your output looks similar to the above, then you have installed Docker correctly.

Airbyte

If you have Airbyte already up and running, skip this section.

Airbyte installation

The Airbyte installation is described in the official docs. You can also follow the commands below.

# pull Airbyte from GitHub
git clone https://github.com/airbytehq/airbyte.git

# start Airbyte
cd airbyte
docker compose up

Verify Airbyte installation

If Airbyte starts up and you visit your server on http://<server-ip>:8000 (Port 8000), you should see a login prompt. The default credentials are specified in the .env file in your Airbyte directory and default to username: airbyte; password: password. After logging in, you should see something like below:

NGINX

If you have NGINX already installed, skip to the config part.

NGINX installation

NGINX is available via apt, which makes the installation process as easy as it gets.

# install NGINX
sudo apt install nginx

If you now visit your server on http://<server-ip> (Port 80), you should see the NGINX welcome message.

NGINX config

Now we need to configure NGINX as a reverse proxy to handle all incoming requests with SSL encryption. I assume that you have created a DNS entry that you will use for your Airbyte deployment. You also need a valid SSL certificate to use SSL authentication.

Then just simply copy the config below and fill in your domain name and the path to your SSL certificate. If you don't know where to put your SSL certificate, I would recommend storing it under /etc/ssl/certs/.

# /etc/nginx/sites-enabled/reverse-proxy.conf
server {
  listen 443 ssl;
  server_name airbyte.<your-domain>.com;
  client_max_body_size 200M;  # see below
  ssl_certificate <path-to-your-cert>.crt.pem; 
  ssl_certificate_key <path-to-your-key>.key.pem;

  location / {
    proxy_pass http://127.0.0.1:8000;
    proxy_set_header Cookie $http_ccokie;  # Airbyte auth cookie
    proxy_read_timeout 3600;  # see below
  }
}

Explanation

• client_max_body_size defaults to 1M. This needs to be increased for the Airbyte API to work. 200M seems to work for me, but you can play around with the values.

• proxy_set_header Cookie $http_cookie is only needed if you use Airbytes basic authentication. You can also use configure authentication on the NGINX site and disable authentication for Airbyte (set username and password to "" in .env).

• proxy_read_timeout defaults to 60 seconds, which is not enough for some Airbyte API operations like schema discovery. 1 hour (3600 seconds) works for me but you can again play around with the values and see what works for you.

After adding the reverse-proxy.conf you can restart NGINX and go to your configured domain (e.g. https://airbyte.<your-domain>.com). You should see the same login prompt and Airbyte UI as before, just with SSL encryption enabled!

HTTP to HTTPS redirect

You can also set an automatic redirect so you don't have to add https:// to your domain name every time you want to visit Airbyte. Simply add a redirect block to your existing config.

# /etc/nginx/sites-enabled/reverse-proxy.conf

# http to https redirect
server {
  listen 80;
  server_name airbyte.<your-domain>.com;
  return 301 https://airbyte.<your-domain>.com$request_uri;
}

# config from earlier
server {
  listen 443 ssl;
  server_name airbyte.<your-domain>.com;
  client_max_body_size 200M;
  ssl_certificate <path-to-your-cert>.crt.pem; 
  ssl_certificate_key <path-to-your-key>.key.pem;

  location / {
    proxy_pass http://127.0.0.1:8000;
    proxy_set_header Cookie $http_ccokie;
    proxy_read_timeout 3600;
  }
}

Don't forget to delete the default config at /etc/nginx/sites-enabled/default and restart NGINX