But, sometimes we have to deal with an environment that does not have internet access (disconnected environment), thus preventing us from installing Operators.
To make things harder, some operators need related images (declared on its CSV) to work properly.
Creating a local disconnected registry and upload operator images to it is not enough to get things up and running.
So, what should I do to solve this?
Before you proceed, it is important to understand a few things. Some Red Hat Operators are self contained, and don’t need related images. But some of them do. The process of pruning an index catalog is suitable for Operators that don’t need extra images. You must know how the operator was built to be able to consider which method is right for you.
Use a command like the one below to get some information about the operator.
# oc get packagemanifests quay-operator -o yaml
(...)\
relatedImages:
- registry.redhat.io/quay/clair-rhel8@sha256:bc7eab15a3047951f559fa9e5eb1c4f8993796307a1404198e2391efd7759b27
- registry.redhat.io/quay/quay-builder-rhel8@sha256:7175525fc0db5ee2f35230d029d39d8fe6f2b733d7a91b89d8144b6216bb61e6
- registry.redhat.io/quay/quay-builder-qemu-rhcos-rhel8@sha256:ea4fca6fa4a1dbf32ed0d300e930c46fa0b53cdeeaf1cb1b055abda1e161a43f
- registry.redhat.io/rhel8/postgresql-10@sha256:a52ad402458ec8ef3f275972c6ebed05ad64398f884404b9bb8e3010c5c95291
- registry.redhat.io/rhel8/redis-6@sha256:b15335d88b240bd6bbedeb7fe26f0a1de2f6c330af9758b0fb2edab7ca69f6df
- registry.redhat.io/quay/quay-operator-rhel8@sha256:2b17143553e30e62aa15f9c63d0be55b89dd02c5d032aa58c279dbb8bb5100b1
- registry.redhat.io/quay/quay-rhel8@sha256:0bfb107a07c6b275d8064e3fba32836cf94d20383edb115da6ee25d1fa52ec9d
(...)
What options are available?
There is an very easy approach, which is to mirror the entire Operator Lifecycle Manager (OLM) catalog, but this will take up to 1.5TB of disk space as the time of this writing (November 2022), making this approach nearly impossible to adopt on the vast majority of disconnected OpenShift clusters.
Maybe that’s not what you’re looking for.
Some operators require related Images that aren’t downloaded in the index prune stage. Look at your Operator CSV to search if it requires additional images. For an example, OCS requires the following related images:
image:
registry.redhat.io/ocs4/ocs-operator-bundle@sha256:64cdf16dbf585e2829b1c145bc2f4649716b0e8487bd863b4f2578d64a14ee9d"
relatedImages:
registry.redhat.io/rhceph/rhceph-4-rhel8@sha256:4b16d6f54a9ae1e43ab0f9b76f1b0860cc4feebfc7ee0e797937fc9445c5bb0a
registry.redhat.io/openshift4/ose-csi-external-provisioner@sha256:e90420f29e9cbf9e7c23ad910c721bea793811fc76c852aef2fa826aa6c3b2c6
registry.redhat.io/ocs4/mcg-core-rhel8@sha256:2e9993a365a1032d84610563817453aade2de343e1e1abfc09b4e8d7f24f91c0
registry.redhat.io/ocs4/ocs-rhel8-
(...)
Now that you understand that some operators need additional content, let’s understand how to solve this issue.
You can do the whole process of creating a new catalog containing all the necessary content 100% manually, but everything becomes a little more time-consuming and complex.
To make things easier, you can use a python script developed by the Red Hat CoP to download the operator images from a predefined list, alongside with it’s related images.
- OpenShift Version: 4.8
- Registry: You already have a local private registry up and running.
- Local Registry URL: registry.rhbrlabs.com
- Bastion has access to the internet.
You don’t have a local private registry? I wrote about building one. Click here to check it out.
Install the python script requisites according to the README.md file. Then do the logins to the required registries:
[root@bastion ocp]# podman login registry.rhbrlabs.com:5000
[root@bastion ocp]# podman login registry.redhat.io
[root@bastion ocp]# podman login quay.io
This will generate the auth.json file (/run/user/0/containers/auth.json). Save it to be able to use it later.
Edit the files offline_operators_list and offline_operators_list.yaml to include the operators you wish to mirror.
In my scenario, I needed Quay and OCS with their related images. These operators also need local-storage (not declared in the CSV). It is also important to install cluster-logging.
✅ I’d recommend always enabling both.
[root@bastion openshift-disconnected-operators-master]# cat offline_operators_list
local-storage-operator
ocs-operator
quay-operator
openshift-marketplace
ocs-must-gather-rhel8
odf-operator
cluster-logging
[root@bastion openshift-disconnected-operators-master]# cat offline_operators_list.yaml
operators:
- name: local-storage-operator
- name: ocs-operator
- name: quay-operator
- name: openshift-marketplace
- name: ocs-must-gather-rhel8
- name: odf-operator
- name: cluster-logging
Now, run the mirror process:
[root@bastion openshift-disconnected-operators-master]# ./mirror-operator-catalogue.py \
--catalog-version v4.8 \
--authfile ~/ocp/localregistry/openshift-disconnected-operators-master/auth.json \
--registry-olm registry.rhbrlabs.com:5000/repository \
--registry-catalog registry.rhbrlabs.com:5000/repository \
--operator-file ~/ocp/localregistry/openshift-disconnected-operators-master/offline_operators_list \
--icsp-scope=namespace \
--ocp-version 4.8 \
--operator-channel 4.8
At the end of the process, you should see messages like this:
Finished mirroring related images.
Creating Image Content Source Policy YAML...
Creating Mapping File...
Creating Image manifest file...
Creating Catalog Source YAML...
Catalogue creation and image mirroring complete
See Publish folder for the image content source policy and catalog source yaml files to apply to your cluster
The mirror process will generate some YAML files in the publish directory.
[root@bastion openshift-disconnected-operators-master]# ls publish/
image_manifest.txt mapping.txt mirror_log.txt olm-icsp.yaml rh-catalog-source.yaml
Disable the default OLM catalog sources:
[root@bastion publish]# oc patch OperatorHub cluster --type json -p '[{"op": "add", "path": "/spec/disableAllDefaultSources", "value": true}]'
Apply the yaml files in the publish directory. The image content source policy will create a new MCO (Machine Config Operator) render which will start a rolling reboot of your cluster nodes. Keep this in mind before performing this step.
⚠️ You have to wait until that is complete before attempting to install operators from the catalog.
[root@bastion publish]# oc create -f olm-icsp.yaml
imagecontentsourcepolicy.operator.openshift.io/olm-image-content-source created
[root@bastion publish]# oc create -f rh-catalog-source.yaml
catalogsource.operators.coreos.com/custom-redhat-operators created
After Operator Lifecycle Manager (OLM) automatically polls the index image referenced in the catalog source at its regular interval (30 minutes), verify that the new packages are successfully added:
[root@bastion localregistry]# oc get packagemanifests -n openshift-marketplace\
NAME CATALOG AGE
quay-operator Red Hat Operators 42m
ocs-operator Red Hat Operators 42m
cluster-logging Red Hat Operators 42m
local-storage-operator Red Hat Operators 42m
Now go to your Openshift Operator Hub and check the available Operators.
The Operators added has its related images, making them complete in terms of requisites. Install them to check if everything is in place.
You can have as many custom catalogs as you need. Go to OpenShift Global Configuration and search for Operator Hub. The list of available catalogs can be found there.
That’s it. This one was a little tricky to sort out. Hope this post was useful to you. Drop a message if you have questions. See ya!
Did you like the content? Check out these other interesting articles! 🔥
Please support this content by clicking on one of our advertisers’ banners. ❤️