Introduction

Red Hat OCP 4.16 and brought some new changes and configurations to installations of OpenShift clusters. There are two main methods of installing OpenShift clusters, UPI (User Provisioned Infrastructure) and IPI (Installer Provisioned Infrastructure). In this guide we will show How to Install Disconnected OCP 4.16 on vSphere Using ACM with the IPI method.

For more information about ACM (Advanced Cluster Management) operator and installation check out our post about How to Install ACM on a Disconnected OpenShift.

Prerequisites

  • OCP Hub cluster 4.16+ with ACM operator v2.13.2+.
  • Created network VLAN for cluster’s management network.
  • Created DNS records for cluster’s api and ingress.
  • Created a DHCP vm as bastion and configured it.
  • Created cluster certificate.
  • Mirror registry – If needed create a mirror registry using: Creating Mirror Registry

You may reference to the following documentation about an OCP cluster installation on vSphere.

Step 1 – Mirroring Images

In this part we will mirror an OCP release image and download ova image for vSphere environment. For this example we will use OCP 4.16.27.

  • First visit the following mirror website and download the following image.
  • Create the following ImageSetConfiguration file to mirror the relevant release image:
apiVersion: mirror.openshift.io/v1alpha2
kind: ImageSetConfiguration
archiveSize: 50                                                    
mirror:
  platform:
    channels:
    - name: stable-4.16
       minVersion:  '4.16.27'
       maxVersion: '4.16.27'
      type: ocp

3. Run the following command to mirror the release image:

$ oc-mirror --config isc.yaml file://.

NOTE: Follow the following documentation to install oc-mirror.

  • Transfer the generated tar file and the downloaded ova image to your environment.

Mirror The Release Image To Local Mirror Registry

After you successfully transferred the files to your environment, push the mirrored images to your local mirror registry. Run the following command to mirror the images from your disk to registry:

$ oc-mirror --from <path-to-tar-file> docker://quay.io

Step 2 – Create Installations Files

In this section we will create the installation files for our installation. For this guide we will call our cluster: lab1

NOTE: All the following files need to applied in your hub cluster.

First we need to create a ClusterImageSet object in order to us be able to use our release image. This file will be applied to your hub cluster.

apiVersion: hive.openshift.io/v1
kind: ClusterImageSet
metadata:
  name: imageset4.16.27-x86-64
spec:
  releaseImage: quay.io/ocp-release/4.16.27-x86_64

Next we will create a namespace for our cluster:

apiVersion: v1
kind: Namespace
metadata:
  name: lab1
  labels:
    cloud: vSphere
    vendor: OpenShift

After we have our namespaces we need to create several secrets containing your vSphere credentials:

apiVersion: v1
kind: Secret
metadata:
  name: vsphere-creds
  namespace: lab1
  labels:
    cloud: vSphere
    vendor: OpenShift
stringData:
  username: <vcenter-username>
  password: <vcenter-password>
---
apiVersion: v1
kind: Secret
metadata:
  name: vsphere-certs
  namespace: lab1
  labels:
    cloud: vSphere
    vendor: OpenShift
stringData:
  .cacert: |
    <ca-certificate>

In addition we need secrets for our pull secret and ssh key:

apiVersion: v1
kind: Secret
metadata:
  name: pull-secret
  namespace: lab1
  labels:
    cloud: vSphere
    vendor: OpenShift
stringData:
  .dockerconfigjson: |
    <pull-secret-content>
---
apiVersion: v1
kind: Secret
metadata:
  name: ssh-private-key
  namespace: lab1
  cloud: vSphere
  vendor: OpenShift
stringData:
  sshprivatekey: |
    <ssh-provate-key>

The next secret will contain our install-config configuration. This file contains the main installation configurations.

apiVersion: v1
kind: Secret
metadata:
  name: install-config
  namespace: lab1
  labels:
    cloud: vSphere
    vendor: OpenShift
stringData:
  install-config.yaml: |
    apiVersion: v1
    metadata:
      name: lab1
    baseDomain: <base-domain> # For example octopus.local
    controlPlane:
      hyperthreading: Enabled
      name: master
      replicas: 3
      platform:
        vsphere:
          cpus: 8
          coresPerSocket: 4
          memoryMB: 32768
          osDisk:
            diskSizeGB: 120
    compute:
    - hyperthreading: Enabled
      name: "worker"
      replicas: 1
      platform:
        vsphere:
          cpus: 4
          coresPerSocket: 2
          memoryMB: 16384
          osDisk:
            diskSizeGB: 120
    platform:
      vsphere:
        apiVIPs:
        - <cluster-api-IP>
        ingressVIPs:
        - <cluster-ingress-IP>
        vCenter: <vcenter-url>
        username: <vcneter-username>
        password: <vcneter-password>
        datacenter: <vcneter-datacenter>
        folder: /<vcneter-datacenter>/vm/<vc-folder>
        clusterOSImage: <path-to-ova-image>
        defaultDatastore: <vcneter-datacenter>
        cluster: <vcneter-cluster>
        network: <cluster's-portgroup>
    networking:
      networkType: OVNKubernetes
      machineNetwork:
      - cidr: <cluster's-subnet/24>
    imageContentSources:
    - mirrors:
      - quay.io/ocp4/openshift4
      source: quay.io/openshift-release-dev/ocp-release
    - mirrors:
      - quay.io/ocp4/openshift4
      source: quay.io/openshift-release-dev/ocp-v4.0-art-dev
    sshKey: |
      <ssh-publick-key>
    additionalTrustBundle: |
      <ca-certtificate>

NOTE: You may combine all the secrets above to one .yaml file.

After we have the secrets prepared we need to create the clusterdeployment file. This file triggers the ACM to start and perform the installation.

apiVersion: hive.openshift.io/v1
kind: ClusterDeployment
metdata:
  name: lab1
  namespace: lab1
  labels:
    cloud: vSphere
    vendor: OpenShift
spec:
  baseDomain: <base-domain> # For example octopus.local
  clusterName: 'lab1'
  controlPlaneConfig:
    servingCertificates: {}
  installAttempsLimit: 1
  installed: false
  platform:
    vsphere:
      cluster: <vcenter-cluster>
      certificatesSecretRef:
        name: vsphere-certs
      vCenter: <vceter-url>
      datacenter: <vcenter-datacenter>
      defaultDatastore: <vcenter-datastore>
      network: <cluster-portgroup>
  provisioning:
    installConfigSecretRef:
      name: install-config
    sshPrivateKeySecretRef:
      name: ssh-private-key
    imageSetRef:
      name: imageset4.16.27-x86-64
    pullSecretRef:
      name: pull-secret

Our last files will be ManagedCluster and KlusterAddonConfig. Those objects are necessary in order to help your ACM manage your cluster.

apiVersion: cluster.open-cluster-management.io/v1
kind: ManagedCluster
metadata:
  name: lab1
  labels:
    cloud: vSphere
    vendor: OpenShift
spec:
  hubAcceptsClient: true
  leaseDurationSeconds: 60
---
apiVersion: agent.open-cluster-management.io/v1
kind: KlusterAddonConfig
metadata:
  name: lab1
  namespace: lab1
  labels:
    cloud: vSphere
    vendor: OpenShift
spec:
  clusterName: lab1
  clusterNamespace: lab1
  applicationManager:
    enabled: true
  certPolicyController:
    enabled: true
  clusterLables:
    cloud: auto-detect
    vendor: auto-detect
  iamPolicyController:
    enabled: true
  policyController:
    enabled: true
  searchController:
    enabled: true

Step 3 – Cluster Installation

After we have all of our configurations all we have left is to apply the manifests and install the cluster.
Apply the manifests in the following order:

$ oc apply -f namesapce.yaml
$ oc apply -f secrets.yaml
$ oc apply -f clusterdeployment.yaml
$ oc apply -f managedcluster.yaml

NOTE: Use oc cluster-info command to make sure you are connected to the right cluster

Acceptance Criteria

After you ran all the manifests above you should see the installation running in the ACM All Clusters tab. After the cluster finishes to install without any errors, the cluster status should show importing .
Afterwords, you can run your Day2 operations and complete the cluster’s deployment on it’s fullest.

Summary

This article walks you through a basic OCP IPI and shows how to install Disconnected OCP 4.16 on vSphere Using ACM. It covers the needed mirroring and installation configurations to help you build your clusters with ACM operator on the best and efficient way.

For more details and other installation methods refer to Install OpenShift On Any Platform.