troubleshooting.adoc

Troubleshooting Runtime Component Operator

You might encounter an issue when installing, configuring, or managing operators. You can run oc or kubectl commands to check the status of pods, operators, and custom resources and to investigate problems.

To run oc commands, you need the OpenShift command-line interface (CLI). To run kubectl commands, if you don’t have OpenShift installed, you need the Kubernetes command line tool.

Tip	The documentation shows oc commands. To run kubectl commands, replace oc with kubectl in the commands.

Troubleshooting Operator

Run the following oc commands to investigate problems with operator pods.

• Check the Runtime Component Operator.

    $ oc get pods -l app.kubernetes.io/name=runtime-component-operator

  Output from the get pods command shows the pod name and status.

    NAME                                      READY   STATUS    RESTARTS   AGE
    rco-controller-manager-854b5d79f8-svdqr   1/1     Running   0          32m

• Check the operator events. In the describe pod command, replace <pod_name> with a pod name from the get pods output.

    $ oc describe pod <pod_name>

  The following example command uses the Runtime Component Operator pod name.

    $ oc describe pod rco-controller-manager-854b5d79f8-svdqr

• Check the operator logs. In the logs command, replace <pod_name> with a pod name from the get pods output.

    $ oc logs <pod_name>

Troubleshooting Custom Resources (CR)

If the operator is running as expected, check the status of the RuntimeComponent CR instance.

Status Conditions

For versions 0.8.2 and above, the status conditions types Reconciled, ResourcesReady and Ready reflect the current status of RuntimeComponent instance in the CR.

• Reconciled will be set to true if the cluster state matches the declared state in the CR. When it is false, message will provide what error is encountered during the reconciliation process.

• ResourcesReady will be set to true if running replicas match the desired replicas or fall within the desired range: [minReplicas, maxReplicas] when auto-scaling is in use. message will show the number of running replicas.

  • Reasons: MinimumReplicasAvailable, MinimumReplicasUnavailable, ReplicaSetUpdating, ResourcesNotReady.

  • For Knative Services, the condition will be set to true if the service is successfully created.

• Ready will be set to true if the application is reconciled and the resources are ready.

Check CR Instance Status

• Check the CR status. In the get runtimecomponent command, replace <app_name> with the name of your CR instance.

    $ oc get runtimecomponent <app_name>

  The following example shows the command with my-app for <app_name> and the output.

    $ oc get runtimecomponent my-app
  
    NAME     IMAGE                        EXPOSED   RECONCILED   RESOURCESREADY   READY   AGE
    my-app   quay.io/my-repo/my-app:1.0             True         True             True    14m

  If you want more details for the conditions, run with -o wide option.

    $ oc get runtimecomponent my-app -o wide
  
    NAME    IMAGE                         EXPOSED   RECONCILED    RECONCILEDREASON    RECONCILEDMESSAGE   RESOURCESREADY    RESOURCESREADYREASON        RESOURCESREADYMESSAGE             READY   READYREASON   READYMESSAGE                                          AGE
    my-app  quay.io/my-repo/my-app:1.0              True                                                  True              MinimumReplicasAvailable    Deployment replicas ready: 4/4    True                  Application is reconciled and resources are ready.    14m

• Check and ensure that the effective CR values are what you want. In the get runtimecomponent command, replace <app_name> with the name of your CR instance.

    $ oc get runtimecomponent <app_name> -o yaml

  Check the status section of the CR. If the application is successfully reconciled and all resources are ready, the output should look like the following:

    $ oc get runtimecomponent my-app -o yaml
  
    apiVersion: rc.app.stacks/v1
    kind: RuntimeComponent
    ...
    status:
      conditions:
        - lastTransitionTime: '2022-05-19T19:53:51Z'
          status: 'True'
          type: Reconciled
        - lastTransitionTime: '2022-05-19T19:54:26Z'
          message: 'Deployment replicas ready: 4/4'
          reason: MinimumReplicasAvailable
          status: 'True'
          type: ResourcesReady
        - lastTransitionTime: '2022-05-19T19:54:26Z'
          message: Application is reconciled and resources are ready.
          status: 'True'
          type: Ready

• Check the CR events. In the describe runtimecomponent command, replace <app_name> with the name of your CR instance.

    $ oc describe runtimecomponent <app_name>

TLS Connections

If .spec.expose=true is set in the CR, then .spec.manageTLS=true and .spec.route.termination=reencyrpt are set by default. With these settings, the application must have TLS enabled on the port configured in .spec.service.port. Otherwise, the application will not receive any requests. The application must also use the TLS certificates mounted by the operator in the application pods. If this configuration is not possible, some alternatives are listed below. Other configurations are possible, but are beyond the scope of this document.

Application uses its own certificates

  apiVersion: rc.app.stacks/v1
  kind: RuntimeComponent
  ...
  spec:
    route:
        termination: passthrough

In this case, TLS traffic will be passed directly through to the application pods. The application can use its own certificates.

Application uses HTTP inside the cluster

  apiVersion: rc.app.stacks/v1
  kind: RuntimeComponent
  ...
  spec:
    route:
      termination: edge
      insecureEdgeTerminationPolicy: Redirect

In this case, insecure external traffic will be redirected to the secure port, but all secure connections will be terminated by the cluster, and the application will only receive HTTP connections.