Setup the High Side
This lab will help you setup the highside system while the oc-mirror download is working.
Recall from the architecture diagram (below) that the highside system needs to be setup to run an image registry inside the disconnected network.
OpenShift provides a mirror-registry that is purpose-built for this task.
The mirror-registry is a simplified, and portable, version of Red Hat’s Quay Image Registry.
You can use any image registry you like, as long as it supports the Docker v2 API, uses TLS encryption, and requires authenticated image pulls. Examples of alternative image registries include.
-
Harbor
-
JFrog Artifactory
-
Sonatype Nexus Repository
-
Red Hat Quay Registry (enterprise)
Lab Overview
The tasks in this lab include:
1. Transfering the installation content to the highside system using rsync
2. Connecting to the highside system and proving that it’s disconnected
3. Installing the mirror-registry
-
Trusting the generated TLS certificate
-
Logging in and creating a new pull secret
4. Uploading OpenShift’s installation images into your mirror-registry
Out-of-Order operations
In order to make good use of our time together, we will perform the tasks described above out-of-order.
We will start by copying the mirror-registry file from the jump system to the highside system, and installing it.
Installing the mirror-registry takes ~8 minutes.
We will demonstrate that the highside system is disconnected while we wait for the download to complete.
Then we will end this section by transferring the ~25GB of OpenShift installation images across the air gap and in to the disconnected mirror-registry.
Creating a Mirror Registry
Begin by using rsync to copy the mirror-registry file from the lowside system to the highside system.
We will run another rsync command a bit later to copy the installation images that oc-mirror is still downloading.
Please run the following commands in the other tmux pane that is logged in to the jump system.
rsync -avP /mnt/low-side-data/mirror-registry.tar.gz highside:/mnt/high-side-data/Warning: Permanently added 'highside' (ED25519) to the list of known hosts.
sending incremental file list
mirror-registry.tar.gz
706,802,330 100% 133.34MB/s 0:00:05 (xfr#1, to-chk=0/1)
sent 706,975,008 bytes received 35 bytes 128,540,916.91 bytes/sec
total size is 706,802,330 speedup is 1.00
Logging in to the highside system
In the real world, customers often access their disconnected systems using a VPN, a jump server, or with dedicated workstations. In your workshop environment, an exception was created in the firewall that allows the jump system to SSH to the highside system. Password-less SSH has been enabled for your convenience.
Please log in to the highside system using ssh from the jump system.
ssh highside[lab-user@jump ~]$ ssh highside Warning: Permanently added 'highside' (ED25519) to the list of known hosts. Last login: Wed May 1 07:56:13 2024 from 10.0.6.23 [lab-user@highside ~]$
Installing the mirror-registry
Now that you are logged into the highside system, change to the high side transfer directory (/mnt/high-side-data) and extract the mirror-registry files.
cd /mnt/high-side-data
tar -xzvf mirror-registry.tar.gzimage-archive.tar execution-environment.tar mirror-registry
The automation that installs and configures the mirror-registry takes ~5 minutes to complete.
The only option that we will provide is one to set the password for the init user to discopass.
./mirror-registry install --initPassword discopass... INFO[2024-05-04 15:43:41] Quay installed successfully, config data is stored in /home/lab-user/quay-install INFO[2024-05-04 15:43:41] Quay is available at https://ip-10-0-51-47.ec2.internal:8443 with credentials (init, discopass)
|
We can’t move on to the next step of trusting the TLS certificate until the |
Trusting the mirror-registry’s TLS certificate
The mirror-registry installer creates its own self-signed TLS certificate that is not trusted by anything, not even the highside system where it was installed.
The procedure to trust the mirror-registry’s self-signed TLS certificate is simple.
The mirror-registry install options allow users to provide their own certificate, if they were issued one, using the --sslCert option.
|
Copy the Root Certificate Authority file (rootCA.pem) that the mirror-registry created in to the Red Hat Enterprise Linux CA trust directory.
Then run the update-ca-trust command.
sudo cp -v $HOME/quay-install/quay-rootCA/rootCA.pem /etc/pki/ca-trust/source/anchors/
sudo update-ca-trust'/home/lab-user/quay-install/quay-rootCA/rootCA.pem' -> '/etc/pki/ca-trust/source/anchors/rootCA.pem'
Log in to the mirror-registry
After the mirror-registry TLS certificate has been trusted, log in with podman.
The username is init and the password discopass.
podman login -u init -p discopass $(hostname):8443Login Succeeded!
The podman login command creates an authentication file / pull secret at $XDG_RUNTIME_DIR/containers/auth.json, for example: /run/user/1001/containers/auth.json.
|
The
|
Transfer the installation content
Make sure the oc-mirror download completed successfully before proceeding!
If oc-mirror is still downloading the OpenShift installation images, please jump forward to Prove that highside is disconnected.
But don’t forget to come back!
You can confirm the download finished by looking in your tmux pane to see if your prompt has returned, and that the .tar archive file was created.
... info: Mirroring completed in 2m47.63s (135.1MB/s) Creating archive /mnt/low-side-data/mirror_seq1_000000.tar [lab-user@jump low-side-data]$
After the oc-mirror command has completed, use rsync on the jump system to copy the installation content into /mnt/high-side-data on the highside system.
|
Please run the |
rsync -avP /mnt/low-side-data/ highside:/mnt/high-side-data/sending incremental file list
./
.oc-mirror.log
186,850 100% 146.94MB/s 0:00:00 (xfr#1, to-chk=124/126)
imageset-config.yaml
469 100% 458.01kB/s 0:00:00 (xfr#2, to-chk=123/126)
mirror_seq1_000000.tar
22,931,079,168 92% 125.16MB/s 0:00:15 (xfr#2, to-chk=122/126)
... a long, but amazing, transfer ...
publish/
publish/.metadata.json
332,183 100% 332.37kB/s 0:00:00 (xfr#66, to-chk=0/127)
sent 30,795,621,525 bytes received 1,565 bytes 131,324,618.72 bytes/sec
total size is 30,788,095,434 speedup is 1.00
Prove that highside is disconnected
Looking at the workshop environment diagram, you can see how the highside system is disconnected. The highside network doesn’t provide any direct route to the internet.
|
The highside systems are configured to use a 1. The highside system allows inbound SSH & HTTP connections from the jump system 2. The highside system is allowed to install RHEL RPMs from the repos inside the Amazon AWS Cloud (not from the public repos at https://cdn.redhat.com) 3. Your disco.lab cluster will be allowed to talk to the Amazon AWS Cloud APIs. More details about this permission will be provided in the next lab 4. 🛑 Nothing else is allowed into or out of the highside network The Appendix has more information about the |
Please use the following commands to prove that the highside system is unable to connect to openshift.com and quay.io.
Recall that:
-
The
ocandopenshift-installtools were downloaded from openshift.com -
The OpenShift installation images were downloaded from quay.io
If you try to access a blocked website, like quay.io, you will see an Access Denied message like this.
curl -I quay.ioHTTP/1.1 403 Forbidden Server: squid/5.5 Date: Mon, 29 Apr 2024 20:08:15 GMT X-Squid-Error: ERR_ACCESS_DENIED 0
If you try to access an allowed website, like the Red Hat Update Infrastructure (RHUI) repos inside the Amazon AWS Cloud, you will see a message like this.
curl -I https://rhui.{aws_default_region}.aws.ce.redhat.comHTTP/1.1 200 OK Server: nginx/1.20.1 Date: Mon, 29 Apr 2024 20:15:51 GMT
If you skipped ahead to prove that the highside systems are really disconnected while the oc-mirror download was still working, it’s time to go back to Transfer the installation content.
Upload OpenShift’s installation images into the mirror-registry
Now that the highside system has: a) received the installation images, b) trusts the mirror-registry’s TLS certificate, and c) podman has logged in and created a new pull secret, you are ready to upload the installation images.
Please begin by adding the oc, oc-mirror and openshift-install commands to the PATH.
Make sure you are using the tmux pane that is logged in to the highside system.
|
sudo mv -v /mnt/high-side-data/oc /bin/
sudo mv -v /mnt/high-side-data/oc-mirror /bin/
sudo mv -v /mnt/high-side-data/openshift-install /bin/renamed '/mnt/high-side-data/oc' -> '/bin/oc' renamed '/mnt/high-side-data/oc-mirror' -> '/bin/oc-mirror' renamed '/mnt/high-side-data/openshift-install' -> '/bin/openshift-install' [lab-user@highside ~]$
With the mirroring tools installed and configured, it is now time to run oc-mirror again, this time on the highside system, to upload the installation images in to the mirror-registry.
The upload process takes ~15 minutes in this workshop environment.
oc-mirror will create several results files in the directory where it was ran.
|
Please change directories to /mnt/high-side-data, and use the tmux pane that is logged in to the highside system to run the next command.
cd /mnt/high-side-data
oc-mirror --from=/mnt/high-side-data/mirror_seq1_000000.tar docker://$(hostname):8443Logging to .oc-mirror.log
Checking push permissions for ip-10-0-54-198.us-west-2.compute.internal:8443
Publishing image set from archive "/mnt/high-side-data/mirror_seq1_000000.tar" to registry "ip-10-0-54-198.us-west-2.compute.internal:8443"
ip-10-0-54-198.us-west-2.compute.internal:8443/
openshift/release
blobs:
file://openshift/release sha256:d8190195889efb5333eeec18af9b6c82313edd4db62989bd3a357caca4f13f0e 1.404KiB
file://openshift/release sha256:cd8c26ed660bfc4447a17563df1dc5374f1cbd2ced1d7ae0af135fabaff1f7d2 17.86KiB
... a long, mind-expanding, upload ...
info: Mirroring completed in 18m10.33s (39.33MB/s)
Rendering catalog image "ip-10-0-8-121.us-west-2.compute.internal:8443/redhat/redhat-operator-index:v4.14" with file-based catalog
Writing image mapping to oc-mirror-workspace/results-1714533240/mapping.txt
Writing UpdateService manifests to oc-mirror-workspace/results-1714533240
Writing CatalogSource manifests to oc-mirror-workspace/results-1714533240
Writing ICSP manifests to oc-mirror-workspace/results-1714533240
[lab-user@highside ~]$
|
You can click the Desktop button and use Firefox to login to your new Reminder: The credentials for your DISCO - Registry are: |
|
As a convenience, the jump system has been configured to automatically trust the highside mirror-registry’s TLS certificate. This automation means workshop participants don’t have to:
You’re welcome! |