# Air gap installation with Embedded Cluster (Beta)

This topic describes how to install an application with Replicated Embedded Cluster on a virtual machine (VM) or bare metal server with no outbound internet access. For online installations, see [Online installation with Embedded Cluster](installing-embedded). For an introduction to Embedded Cluster, see [Embedded Cluster overview](embedded-overview).

## Overview

Building an air gap bundle for a release with an Embedded Cluster Config produces both an application air gap bundle and an Embedded Cluster air gap bundle. You can use the application air gap bundle for air gap installations with Replicated kURL or with Replicated KOTS in an existing cluster. Use the Embedded Cluster air gap bundle for air gap installations with Embedded Cluster.

The Embedded Cluster air gap bundle contains the assets normally found in an application air gap bundle (`airgap.yaml`, `app.tar.gz`, and an images directory). It also includes an `embedded-cluster` directory with the assets needed to install the infrastructure (Embedded Cluster/k0s and [extensions](embedded-config#extensions)).

During installation with Embedded Cluster in air gap environments, Embedded Cluster deploys a Docker registry to the cluster to store application images. The installer preloads infrastructure images (for Embedded Cluster and Helm extensions) and the Helm charts on each node at installation time.

## Limitations and known issues

Embedded Cluster installations in air gap environments have the following limitations and known issues:

* If you pass `?airgap=true` to the `replicated.app` endpoint but an air gap bundle is not built for the latest release, the API will not return a 404. Instead it will return the tarball without the air gap bundle (as in, with the installer and the license in it, like for online installations).

* Images used by Helm extensions must not refer to a multi-architecture image by digest. Air gap bundles include only x64 images, and the digest for the x64 image will be different from the digest for the multi-architecture image, preventing Kubernetes from locating the image in the bundle. An example of a chart that does this is ingress-nginx/ingress-nginx chart. For an example of how to set digests to empty strings and pull by tag only, see [extensions](embedded-config#extensions) in _Embedded Cluster Config_.

* Embedded Cluster loads images for Helm extensions directly into containerd so that they are available without internet access. But if an image used by a Helm extension has **Always** set as the image pull policy, Kubernetes will try to pull the image from the internet. If necessary, use the Helm values to configure `IfNotPresent` as the image pull policy to ensure the extension works in air gap environments.

* On the channel release history page, the links for **Download air gap bundle**, **Copy download URL**, and **View bundle contents** pertain to the application air gap bundle only, not the Embedded Cluster bundle.

## Prerequisites

Before you install, complete the following prerequisites:

* Ensure that your installation environment meets the Embedded Cluster requirements. See [Embedded Cluster Requirements](/embedded-cluster/v3/installing-embedded-requirements).

* The application release that you want to install must include an [Embedded Cluster Config](/embedded-cluster/v3/embedded-config).

* The license used to install must have the **Embedded Cluster Enabled** license field enabled. See [Create and Manage Customers](/vendor/releases-creating-customer).

## Install using the Embedded Cluster UI

This section describes how to install using the interactive Embedded Cluster installer UI.

:::note
These steps assume you download installation assets on a host that has internet access. Then copy them into the air-gapped environment where Embedded Cluster will run.
:::

### Use the Enterprise Portal for a guided install

To use the guided install experience in the Enterprise Portal:

1. Log in to the [Replicated Enterprise Portal](/vendor/enterprise-portal-about) with a user for your test customer.

1. Follow the Linux install instructions for Embedded Cluster in an air gap environment.

### Install using instructions from the Vendor Portal

To install from the Vendor Portal:

1. In the [Vendor Portal](https://vendor.replicated.com), open the channel where you promoted the target release so Replicated can build the air gap bundle. Do one of the following:

   * If you have **Automatically create airgap builds for newly promoted releases in this channel** enabled on the channel, wait until the build completes.
   * If automatic air gap builds are not enabled, open the **Release history** page for the channel and build the air gap bundle manually.

   :::note
   Errors from building the application air gap bundle or the Embedded Cluster infrastructure appear in the Vendor Portal when present.
   :::

1. Open **Customers** and select the target customer.

1. On the **Manage customer** tab, in **License options**, enable **Airgap Download Enabled**.

1. At the top of the customer page, click **Install instructions** and select **Embedded Cluster**. (The control may appear as **Embedded Cluster install instructions**.)

   :::note
   You can also open the Enterprise Portal to get install and upgrade instructions.
   :::

1. In the **Embedded Cluster install instructions** dialog, enable **Install in an air gap environment**.

   ![Customer install instructions drop down button](/images/customer-install-instructions-dropdown.png)

   [View a larger version of this image](/images/customer-install-instructions-dropdown.png)

1. In the **Select a version** field (or equivalent), choose the application version you promoted to that channel for testing, or leave the latest version selected if that is what you want to install.

1. On a machine **with internet access**, run the download command from the dialog to save the air gap installation assets as a `.tgz` archive.

1. Copy the `.tgz` file to the air-gapped machine where you will install.

1. On the air-gapped machine, extract the `.tgz` using the command from the install instructions dialog. You should have at least:

   * The installer binary
   * The license file
   * The air gap bundle (`APP_SLUG.airgap`)

1. On the air-gapped machine, run the `install` command from the dialog. It must include `--airgap` pointing at your `.airgap` file. For example:

   ```bash
   sudo ./APP_SLUG install --license license.yaml --airgap APP_SLUG.airgap
   ```

   Replace `APP_SLUG` with your application slug and use the correct paths for your license and bundle files. Add any other flags you need. For the full set of flags, see [install](embedded-cluster-install).

1. Complete any prompts from the install command (for example, acceptance of a self-signed certificate and installer password).

1. On a machine with browser access to the installer (often the same air-gapped host, or another host on the same network), open the URL from the install output to open the installer UI.

1. Log in to the installer with the password you set when you ran the install command.

1. On the **Configure** page, enter the configuration values for your application.

1. On the **Set Up** page, provide any information required for the cluster installation.

1. On the **Run** page, start the installation and wait for it to complete.

1. On the **Finish** page, confirm completion. If your KOTS Application custom resource defines links, they appear on this page.

   :::note
   Links to port-forwarded services are not available on this page.
   :::

## Install using the CLI (headless)

With headless installations, you provide all the necessary installation assets with the installation command rather than through the UI. These assets include the license file and the application config values. Any preflight checks defined for the application run automatically from the command line rather than displaying in the UI. You can also ignore preflight checks during headless installations when testing in development environments. See [Ignore preflight checks during installation](#ignore-preflights) on this page.

To install headlessly in an air gap environment:

1. Complete the Vendor Portal steps to build the air gap bundle, enable **Airgap Download Enabled** for the customer, and open **Install instructions** with **Install in an air gap environment** selected. For more detail, see steps 1 through 6 in [Install using the Replicated Vendor Portal](#install-using-the-replicated-vendor-portal).

1. In the **Select a version** field (or equivalent), choose the application version to install.

1. On a machine **with internet access**, run the first two commands from the dialog (download the archive, then extract it).

1. Copy the extracted directory (or the `.tgz` and extract on the air-gapped host) to the air-gapped machine.

1. On the air-gapped machine, run `install` with:

   * `--airgap` and the `APP_SLUG.airgap` file.
   * `--headless` to skip the interactive installer UI.
   * `--config-values` with the path to your [ConfigValues](/reference/custom-resource-configvalues) file.
   * `--installer-password` with the password you will use to access the installer later.

   Example:

   ```bash
   sudo ./APP_SLUG install --license license.yaml --airgap APP_SLUG.airgap --headless --config-values PATH_TO_CONFIGVALUES --installer-password INSTALLER_PASSWORD
   ```

   Replace placeholders with values for your environment. Add any other flags you need. For the full set of flags, see [install](embedded-cluster-install).

1. Monitor the command output until the install completes, then access your application as you normally do.

## (Optional) Ignore preflight checks during installation {#ignore-preflights}

You can ignore both application-level preflight checks and Embedded Cluster host preflight checks during installation. When you ignore preflight checks, the installation proceeds despite any preflight failures. This is useful for automated installations in development environments. For more information about the `--ignore-host-preflights` and `--ignore-app-preflights` options, see [install](embedded-cluster-install).

Ignoring host preflight checks is _not_ recommended for production installations.

* To ignore preflight checks:

    ```bash
    sudo ./APP_SLUG install --license license.yaml --ignore-host-preflights --ignore-app-preflights --yes
    ```
    Where:
    * `APP_SLUG` is the unique slug for the application
    * The `--yes` flag addresses the prompt for `--ignore-host-preflights` to continue with installation