preloader
  • Home
  • OpenShift Antrea CNI integration with NSX-T Data Center

OpenShift Antrea CNI is a plugin that, among other things, integrates the NSX-T Data Center with the OpenShift container orchestration platform, providing network connectivity for Kubernetes pods with advanced connectivity features, such as security policies and load balancing, for better access control.

blog-thumb

The integration of Antrea CNI with NSX-T enables detailed network visibility and control.

In the VMWare ecosystem, we have NSX-T. The integration between CNI Antrea and NSX-T makes it possible to micro-manage Kubernetes accesses, which can be advantageous in terms of security and compliance plans, and data flow control.

On the OpenShift side, this works by replacing the standard SDN with NSX-T Antrea (networkType: antrea).

This article is a complement to my previous article on Antrea CNI, which can be found by clicking here

🛑 IMPORTANT: Remember that this article is a complement to the other one I mentioned. In other words, the OpenShift cluster must be installed according to everything described there.


The following software versions were used in this laboratory:

* Antrea 1.7.0

* Antrea interworking 0.11.0


VMWare’s documentation

The information contained in this material follows the steps according to the official documentation.


Private and public key pair

Integration with NSX Manager takes place via trust, which is why we need a key pair.

The public counterpart of the key will need to be inserted into NSX Manager for the integration to take place successfully.

To create them, use openssl as shown below.


$ openssl genrsa -out cluster-ocp-prd.key 2048

$ openssl req -new -key cluster-ocp-prd.key -out cluster-ocp-prd.csr -subj "/C=BR/ST=SP/L=Sao_Paulo/O=RHT/OU=Infra/CN=my-ocp-cluster-name"

$ openssl x509 -req -days 3650 -sha256 -in cluster-ocp-prd.csr -signkey cluster-ocp-prd.key -out cluster-ocp-prd.crt
 

🔴 NOTE: The my-ocp-cluster-name statement should reflect the name of your OpenShift cluster, and be consistent throughout.


To use these keys in the configuration YAMLs, convert them to BASE64.


$ cat cluster-ocp-prd.key | base64 -w 0

$ cat cluster-ocp-prd.crt | base64 -w 0
 

Preparing the files

In this example, the files needed refer to the Interworking templates.

If you have any doubts about the templates used, please refer to my previous article, which shows you in detail how to obtain them.


Antrea Interworking Manifests

In my environment, the files are stored as shown below. I suggest using a similar organization to make the process easier to understand.


$ pwd
~/antrea_1.7.0/antrea/interworking/antrea-interworking-0.11.0

$ ll ~/antrea_1.7.0/antrea/interworking/antrea-interworking-0.11.0/
total 1030492
drwxrwxr-x. 1 root root         38 May 12  2023 bin
-rw-rw-r--. 1 root root       8709 Apr 30 15:22 bootstrap-config.yaml
-rw-rw-r--. 1 root root       1856 Apr 19 16:52 bootstrap-config.yaml-bkp
-rw-rw-r--. 1 root root       3427 Apr 19 17:59 deregisterjob.yaml
-rw-rw-r--. 1 root root 1055174656 May 12  2023 interworking-debian-0.11.0.tar
-rw-rw-r--. 1 root root      19775 Apr 19 17:58 interworking.yaml
-rw-rw-r--. 1 root root       1968 May 12  2023 inventorycleanup.yaml
-rw-rw-r--. 1 root root        748 May 12  2023 ns-label-webhook.yaml
 

Template Customization

You will need to edit the following files:

- bootstrap-config.yaml

- interworking.yaml


🟡 Only edit the necessary variables.


In the bootstrap-config.yaml file, customize the following fields:

- clusterName

- NSXManagers

- tls.crt

- tls.key


The end result should be similar to the one below.

⚠️ Don’t forget to replace my-ocp-cluster-name with the name of your cluster, the IPs of the NSXManagers, and the BASE64 of the certificates.


$ cat bootstrap-config.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: vmware-system-antrea
  labels:
    app: antrea-interworking
    openshift.io/run-level: '0'
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: bootstrap-config
  namespace: vmware-system-antrea
data:
  bootstrap.conf: |
    bootstrapFrom: "Inline"

    clusterName: my-ocp-cluster-name
    
    NSXManagers: [10.128.X.1,10.128.X.2,10.128.X.3]

    vpcPath: ""
    # proxyEndpoints is optional. If proxyEndpoints.rest-api is set, NSXManagers will be ignored.
    proxyEndpoints:
      rest-api: []
      nsx-rpc-fwd-proxy: []    
---
apiVersion: v1
kind: Secret
metadata:
  name: nsx-cert
  namespace: vmware-system-antrea
type: kubernetes.io/tls
data:
  # One line base64 encoded data. Can be generated by command: cat tls.crt | base64 -w 0
  tls.crt:
    INSERT BASE64 FROM cluster-ocp-prd.crt FILE HERE
  # One line base64 encoded data. Can be generated by command: cat tls.key | base64 -w 0
  tls.key:
    INSERT BASE64 FROM cluster-ocp-prd.key FILE HERE

In the file interworking.yaml, customize all occurrences of the field below.

From: image: Replace

To: image: projects.registry.vmware.com/antreainterworking/interworking-ubi:0.11.0


The end result should be similar to the one below.

$ cat interworking.yaml

(...)
apiVersion: batch/v1
kind: Job
metadata:
  name: register
  labels:
    app: antrea-interworking
    component: register
  namespace: vmware-system-antrea
spec:
  ttlSecondsAfterFinished: 600
  template:
    spec:
      containers:
        - name: register
          image: projects.registry.vmware.com/antreainterworking/interworking-ubi:0.11.0
          imagePullPolicy: IfNotPresent
(...)
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: antrea-interworking
    component: interworking
  name: interworking
  namespace: vmware-system-antrea
spec:
  (...)
    spec:
      (...)
          image: projects.registry.vmware.com/antreainterworking/interworking-ubi:0.11.0
          imagePullPolicy: IfNotPresent
          (...)
          image: projects.registry.vmware.com/antreainterworking/interworking-ubi:0.11.0
          imagePullPolicy: IfNotPresent
          (...)
          image: projects.registry.vmware.com/antreainterworking/interworking-ubi:0.11.0
          imagePullPolicy: IfNotPresent
          (...)
          image: projects.registry.vmware.com/antreainterworking/interworking-ubi:0.11.0
          imagePullPolicy: IfNotPresent
          (...)

Note: This YAML is very large. Be careful not to change anything that isn’t necessary, or the integration will fail.


Integrating

Once the environment has been prepared as described in the previous article, and you have the Antrea CNI and NSX-T integration manifests, all you have to do is apply them.


$ oc apply -f bootstrap-config.yaml

$ oc apply -f interworking.yaml
 

Interworking

Interworking is the component that integrates the OpenShift Cluster with NSX-T.

In OpenShift, note that the POD of the Interworking operator must be running.

Interworking Operator

Interworking Operator


NSX

On the NSX-T side, we can see that the OpenShift cluster using CNI Antrea has been integrated. The integration makes it possible to observe all the details of the cluster, as well as to control network access in detail.

OCP Integrated to NSX

OCP Integrated to NSX

OCP Integrated to NSX


Did you like the content? Check out these other interesting articles! 🔥



Support us 💖

Do you like what you find here? With every click on a banner, you help keep this site alive and free. Your support makes all the difference so that we can continue to bring you the content you love. Thank you very much! 😊

comments powered by Disqus