Appendix: Lab Details

Pull Secrets

A pull secret is JSON-formated data that combines authentication information for one or more Image Registries into a single file. You can find your own pull secret in the Red Hat Hybrid Cloud Console.

Connected cluster’s, and systems that use oc-mirror to download the OpenShift installation images, are required to have valid authentication for the quay.io registry. The other registries in your pull secret allow your OpenShift cluster to build a sample application catalog for your developers, fetch official content from Red Hat, and participate in the Insight monitoring service.

The pull secret of a disconnected cluster is only required to have authentication details for the mirror-registry and any other registries that your applications depend upon.

OpenShift provides a global pull secret, and allows individual namespaces / projects to create their own pull secrets. It is not allowed to have multiple authentication for a single registry (e.g. defined in the global and namespaced pull secrets.)

The jq tool makes it easy to inspect and reformat the contents of a pull secret. And you can add and remove entries in a pull secret using podman.

jq . ~/.docker/config.json
{
  "auths": {
    "cloud.openshift.com": {
      "auth": "dXNlcm5hbWU6cGFzc3dvcmQ=",
      "email": "groovy@disco.lab"
    },
    "quay.io": {
      "auth": "dXNlcm5hbWU6cGFzc3dvcmQ=",
      "email": "groovy@disco.lab"
    },
    "registry.connect.redhat.com": {
      "auth": "dXNlcm5hbWU6cGFzc3dvcmQ=",
      "email": "groovy@disco.lab"
    },
    "registry.redhat.io": {
      "auth": "dXNlcm5hbWU6cGFzc3dvcmQ=",
      "email": "groovy@disco.lab"
    }
  }
}

The "auth" string is base64-encoded data. If you decode the string you will see that it says username:password

The install-config.yaml file doesn’t allow multi-line pull secrets. You can collapse / condense / compact multi-line JSON with jq -c

cat /run/user/1001/containers/auth.json

jq -c . /run/user/1001/containers/auth.json
{
  "auths": {
    "mirror-registry.disco.lab:8443": {
      "auth": "aW5pdDpkaXNjb3Bhc3M="
    }
  }
}

{"auths":{"mirror-registry.disco.lab:8443":{"auth":"aW5pdDpkaXNjb3Bhc3M="}}}

Add and remove _pull secret entries using podman

podman login --authfile $HOME/my-pull-secret.json mirror-registry.disco.lab:8443

podman logout --authfile $HOME/my-pull-secret.json mirror-registry.disco.lab:8443
Username: init
Password: discopass
Login Succeeded!

Removed login credentials for mirror-registry.disco.lab:8443

Two ways to oc-mirror - Partially Disconnected -vs Fully Disconnected

The oc-mirror plugin supports two mirroring scenarios.

If your network security allows it, oc-mirror can simultaneously download the OpenShift installation images and upload it into the mirror-registry in the disconnected environment.

This is documented as a partially disconnected environment.

The other scenario, described in this workshop, requires oc-mirror to download the OpenShift installation images into a .tar file on a lowside system.

After the .tar file is created, it must be moved into the [.highside]# highside disconnected environment#.

Another instance of oc-mirror will open the .tar file and upload the contents into the mirror-registry.

This is documented as a fully disconnected environment.

Proxy

The highside network restricts outbound traffic with a Squid proxy running on the nat system. The nat system / proxy prevents any egress traffic not listed in its /etc/squid/whitelist.txt file.

Note that access from the highside systems to quay.io, openshift.com, and redhat.com is DENIED.

ssh nat "sudo cat /etc/squid/whitelist.txt"
.amazonaws.com
.cloudfront.net
.aws.ce.redhat.com

There may be situations where you wish to add more exceptions here, such as container or package repositories.

Now that our air gap is in place, let’s start prepping the low side.

openshift-install details

The OpenShift installation process is initiated from the highside server. There are a handful of different ways to install OpenShift, but for this lab we’re going to be using installer-provisioned infrastructure (IPI). By default, the installation program acts as an installation wizard, prompting you for values that it cannot determine on its own and providing reasonable default values for the remaining parameters. After the wizard portion of openshift-install completes, you then customize the install-config.yaml file that was produced to specify the disconnected pieces. With the standard values and the disconnected values added to the install-config.yaml file, openshift-install can automatically provisions the underlying infrastructure for the cluster.

Examples of the underlying infrastructure include:

  • Virtual Machines / Cloud instances

  • DNS records

  • Firewall / Security Groups

  • Load Balancers

  • IAM permissions that allow the cluster to scale and self-heal

You may notice that nodes are provisioned through a process called Ignition. This concept is out of scope for this workshop, but if you’re interested to learn more about it, you can read up on it in the documentation here.

IPI is the recommended installation method in most cases because it leverages full automation in installation and cluster management, but there are some key considerations to keep in mind when planning a production installation in a real world scenario.

  • You may not have access to the infrastructure APIs. Our lab is going to live in AWS, which requires connectivity to the .amazonaws.com domain to talk to AWS, as well as the .aws.ce.redhat.com domain to talk to Red Hat Update Infrastructure (RHUI). We accomplish this by using an allowed list on a Squid proxy running on the high side, but a similar approach may not be achievable or permissible for everyone. We’ll discuss this further later in the lab.

  • You may not have sufficient permissions with your infrastructure provider. Our lab has full admin in our AWS enclave, so that’s not a constraint we’ll need to deal with. In real world environments, you’ll need to ensure your account has the appropriate permissions which sometimes involves negotiating with security teams.

Once configuration has been completed, we can kick off the OpenShift Installer and it will do all the work for us to provision the infrastructure and install OpenShift. Here’s a diagram describing everything we’ve discussed so far:

disco diagram