ON THIS PAGE
 

APM Installation

APM Installation Overview

Juniper Address Pool Manager (APM) is an automated, centralized, container-based cloud-native application that network operators and administrators use to manage IP prefix resources. APM works with managed broadband network gateways (BNGs) to monitor address pools on BNGs. When the number of free addresses drops below a set threshold, the BNG raises an alarm. The alarm triggers APM to allocate unused prefixes from its global list of prefixes and provision a subset of the prefixes to the BNG as new pools.

APM can be installed on a single geography setup or on a multiple geography setup. The installation requirements and installation process for these two types of setups are different. See the followings sections for the requirements for your APM setup:

Note:

The term BNG in this document also applies to the BNG CUPS Controller.

You can deploy APM on any hardware that meets the requirements. The following sections describe:

  • APM installation requirements

  • How to install APM

  • How to adjust APM setup parameters

APM Installation Requirements

To install APM, you need the following hardware and software requirements listed in this section.

APM Requirements for a Single Geography Setup

APM installs on a single geography setup. A single geography setup consists of a multinode Kubernetes cluster. The cluster nodes may be physical or virtual machines. For availability, the cluster must have the Kubernetes control plane function running on at least three nodes and the worker function running on at least three nodes. For node economy, the Kubernetes control plane and worker functions can be combined to run on the same node (a combined node).

APM has been qualified against the single geography cluster described in Table 1.

Table 1: Single Kubernetes Cluster Setup Requirements
Category Details

Kubernetes cluster

The Kubernetes cluster requires the following:

  • Node specification:

    • Use one of the following for Kubernetes distribution:

      • Red Hat OpenShift Container Platform. For the version and configuration details, see the Address Pool Manger Release Notes for the release of APM that you are installing. (Option for production clusters).

      • Rancher Kubernetes Engine2 (RKE2). For the version and configuration details, see the Address Pool Manger Release Notes for the release of APM that you are installing. (Option for production clusters).

      • BBE Cloudsetup utility—Used for proof of concept (PoC) or demonstration clusters only.

        The BBE Cloudsetup utility builds a cluster with RKE1 distribution with versions of Longhorn CSI, MetalLB, Flannel CNI, and Docker Container Registry software.

    • CPU: 16 cores or 32 cores. Use a 32 core node if you plan on running other applications on the cluster (such as the BNG CUPS Controller application).

    • Memory: 256 gibibytes (GiB)

    • Storage: 480‌ GiB partitioned as 240 GiB root and 240 GiB /var/lib/longhorn

    • Network Interfaces: 4x10 GE (depending on your topology, more maybe required)

  • Cluster width—The cluster must have a minimum of 3 combined Kubernetes nodes with control plane, etcd, and worker functions. The 3 combined nodes supports APM and BNG CUPS Controller with a single control plane instance. If you are adding BNG CUPS Controller control plane instances, you must add another worker node for each additional pair of BNG CUPS Controller control plane instances that you add.

Jump host

 The jump host requires the following:

  • CPU: 2 core

  • Memory: 8 gibibytes (GiB)

  • Storage: 128 (GiB)

Jump host software

The jump host requires the following software:

  • PoC systems:

    • Ubuntu 22.04 LTS

    • Python 3.12-venv

    • The BBE Cloudsetup utility will install compatible versions of the Kubernetes CLI, Helm, docker-ce, and other components necessary to orchestrate the application.

  • Production systems:

    • Red Hat Enterprise Linux 9 or Ubuntu 24.04 LTS.

    • For a list of the required software components, see the Address Pool Manger Release Notes for the release of APM that you are installing.

Storage

A storage class named jnpr-bbe-storage.

Network load balancer address

Up to 2 addresses, one for APMi (the service interface between the BNG and APM) and one for Management (SSH CLI access, optional)

Registry storage

Each APM release requires approximately 2 GiB of container images.

APM Requirements for a Multiple Geography Setup

A multiple geography setup consists of two separate multinode Kubernetes clusters. The cluster nodes may be physical or virtual machines. For availability, each cluster must have the Kubernetes control plane function running on at least three nodes and the worker function running on at least three nodes. For node economy, the Kubernetes control plane and worker functions may be combined to run on the same node (combined node). Each of the two clusters is geographically separated, so that service impacting events affecting one cluster do not affect the other.

Each cluster is a workload cluster. The workload clusters provide a redundant platform on which APM runs.

For PoC installations, you cannot use the BBE cloudsetup utility to build the clusters used in a multiple geography setup. To build a PoC multiple geography setup, a separate procedure is available through support.

APM has been qualified against the multiple geography cluster described in Table 2 .

Table 2: Multiple Geography Kubernetes Cluster Setup Requirements
Category Details

Cluster

A multiple geography cluster consists of 2 workload clusters with each cluster consisting of at least 3 combined nodes.

Note:

Make sure that the cluster and service CIDRs for each workload cluster do not overlap. The internal networks of each workload cluster are connected by a Submariner IP tunnel. The internal CIDRs must be distinct.

Workload cluster

Each workload cluster requires the following:

  • Node specification:

    • Use one of the following for Kubernetes distribution:

      • Red Hat OpenShift Container Platform. For the version and configuration details, see the Address Pool Manger Release Notes for the release of APM that you are installing.

      • Rancher Kubernetes Engine2 (RKE2). For the version and configuration details, see the Address Pool Manger Release Notes for the release of APM that you are installing.

    • CPU: 16 cores or 32 cores. Use a 32 core node if you plan on running other applications on the cluster (such as the BNG CUPS Controller application).

    • Memory: 256 gibibytes (GiB)

    • Storage: 480‌ GiB partitioned as 240 GiB root and 240 GiB /var/lib/longhorn

    • Network Interfaces: 4x10 GE (depending on your topology, more maybe required)

  • Cluster width—The cluster must have a minimum of 3 combined nodes with control plane, etcd, and worker functions. The 3 combined nodes supports APM and BNG CUPS Controller with a single control plane instance. If you are adding BNG CUPS Controller control plane instances, you must add another worker node for each additional pair of BNG CUPS Controller control plane instance that you add.

This specification establishes a cluster that can run APM as well as its companion applications such as BNG CUPS Controller and BBE Event Collection and Visualization simultaneously.

Jump host

 The jump host requires the following:

  • CPU: 2 core

  • Memory: 8 gibibytes (GiB)

  • Storage: 128 (GiB)

Jump host software

The jump host requires the following software:

  • Red Hat Enterprise Linux 9 or Ubuntu 24.04 LTS.

  • For a list of the required software components, see the Address Pool Manger Release Notes for the release of APM that you are installing.

Storage

A storage class named jnpr-bbe-storage

Network load balancer addresses

Up to 2 addresses, one for APMi (the service interface between the BNG and APM) and one for Management (SSH CLI access, optional)

Registry storage

Each APM release requires approximately 2 GiB of container images.

Note:

In a single geography APM setup, you can make some basic assumptions about the cluster's parameters. You can use a quick start tool like BBE Cloudsetup to create a single geography APM. The construction of a production environment APM setup with multiple geographies and multiple clusters requires much more input from you to build.

Additional Requirements

The BNG is a Juniper Networks MX Series Junos OS router or a Juniper BNG CUPS Controller (BNG CUPS Controller).

We recommend the following releases:

  • Junos OS Release 23.4R2-S7 or later

  • BNG CUPS Controller 26.2R2 or later

For APM, confirm that you have a juniper.net user account with permissions to download the APM software package. Download and install the APM software from a machine that will not be part of the Kubernetes cluster.

Prepare for APM Installation in a Single Geography Setup

Use the procedures in this section to install a single geography APM for the first time.

Before you begin, confirm that you have met the requirements for the APM installation.

We recommend that you use a secure connection between APM and the BNG.

Before starting the APM installation, make sure that you have the following information:

Required Information:

  • Container registry details:

    • If you are using a Rancher Kubernetes Engine2 or BBE Cloudsetup created cluster:

      • External registry address

      • External registry port number (usually 5000)

    • If you are using a Red Hat OpenShift Container Platform cluster:

      • External registry (FQDN)

      • Internal (Docker) registry address

      • Internal (Docker) registry port number

Optional Information:

  • APM initial configuration file. If a configuration file is not supplied, a basic configuration file is automatically generated.
  • Storage class name for persistent volume claim (PVC) creation (default is jnpr-bbe-storage).
  • PVC Size (default is 90 MiB).
  • Archival configuration details. This is required if you are planning to mirror a copy of the APM configuration to an external server.

    • Either the name of the SSH private key file or the name of the Kubernetes secret that is present in the jnpr-apm namespace containing the SSH private key.

    • The SCP URL of the server where the configuration file will be archived. An SCP URL takes the form of scp://user-login@server-fqdn:server-port/absolute-file-path (for example, scp://user@host1.mydomain.com:30443/home/user/configs/apm).

  • Syslog server details. This is required if you are planning to export APM logs to an external syslog collector.
    Note:

    If BBE Event Collection and Visualization is detected running on the target cluster, the address and port values of the ECAV deployment will be suggested as the default.

    • Syslog server address.

    • Syslog server port number.

  • APMi Details—You may optionally provide a specific IP address to use as the external load balancer IP address for APMi on the workload cluster. If a specific address is not provided, APM attempts to allocate an external address from the network load balancer's default pool:

    • External IP address—Enter an unused IP address from a subnet that the cluster nodes and the entities are connected to.

    • Port (default is 20557)
    • TLS details. You will need one of the following:

      • None (insecure)

      • Either the key and certificate files, or the name of the Kubernetes secret that is present in the jnpr-apm namespace that contains the key and certificate information.

  • Service Account Name—The name of the Kubernetes service account used to bind certain operational privileges to the mgmt microservice. If a service account name is not provided, APM creates a service account named apm-svca during rollout.

  • SSH service type—If SSH access to the mgmt microservice is specified (ssh <ip>:<port>), you must specify whether the service should be created as a node port (NodePort) service or a load balancer (LoadBalancer) service. If LoadBalancer is selected, a MetalLB pool is created containing the supplied external IP address. The load balancer service created at rollout is assigned the external IP address from the newly created MetalLB pool.

  • DBSync service type—The apm multi-cluster status APM utility command collects the state to display from the DBSync microservice through a Kubernetes service. By default, a node port service is created for this purpose. If you select LoadBalancer, you are prompted for an external IP address and a MetalLB pool is created containing the supplied external IP address. The LoadBalancer service created at rollout is assigned the external IP address from the newly created MetalLB pool.

  • Number of worker processes for the provman microservice (default is 3).

Install the APM Application (Single Geography)

You use the procedure in this section if you are installing a single geography APM.

  1. Download the APM software package from the Juniper Networks software download page to the jump host.

    APM is available as a compressed TAR (.tgz) file. The filename includes the release number as part of the name. The release number has the format: <Major>.<Minor>.<Maintenance>

    • Major is the main release number of the product.
    • Minor is the minor release number of the product.
    • Maintenance is the revision number.
  2. Unpack the APM TAR (.tgz) file on the jump host by entering:
  3. Run the loader script after you unpack the TAR file.
  4. Use the sudo -E apm link --context context-name --version apm-version command to link to the cluster. The link command associates the loaded APM software package to the cluster context in preparation for the setup.

    • context-name is the Kubernetes context name of the cluster.

    • apm-version is the software version.

  5. If you are installing APM on a Red Hat OpenShift Container Platform cluster, log in with the OpenShift CLI and then proceed to the next step.
    If you are installing APM on a Rancher Kubernetes Engine2 or BBE Cloudsetup created cluster, proceed to the next step.
  6. You must authenticate with the container registry in order to be able to push the APM container images. How you authenticate to the registry varies depending on if you are installing APM on a Rancher Kubernetes Engine2 created cluster, a BBE Cloudsetup created cluster, or on a Red Hat OpenShift Container Platform cluster. (See the respective documentation for details).
  7. Run setup to configure your installation. The setup command does the following:

    • Collects information about the cluster environment such as container registry contact information, keys and certificates needed to secure external interfaces, persistent storage resources, and other information relevant to supporting APM features.

    • Establishes the operational parameters for the Kubernetes deployment.

      If you did not use either the bbecloudsetup option or the template file-name option with the setup command, you need to complete these prompts during the setup:

      Note:

      Only use either the bbecloudsetup option or the template file-name option. You cannot use both options.

      • If you are using Rancher Kubernetes Engine2 or BBE Cloudsetup to create your cluster:

        • External registry address.

        • External registry port number.

      • If you are using a Red Hat OpenShift Container Platform cluster:

        • External registry (FQDN)

        • Internal (Docker) registry address

        • Internal (Docker) registry port number

    Note:

    When running setup, you can interact with the setup process by entering ^d.

    If you want to change a value after entering it, enter ^d. After entering ^d, the value you previously entered is removed and the default value is used for the question. You can use the ^d operation for any setup questions that are optional, or for which a list of values can be provided.

    Following is an example setup command with a sample output:

    Note:

    context context-name is the only required option for the setup command.

    The options that you can use with the setup command are listed in the following:

    • context context-name—The Kubernetes context name of the cluster.

    • h, help—Shows the help message and exit.

    • l, log [error, warning, info, debug]—Adjusts the log level.

    • no-color—Prints messages without colors.

    • bbecloudsetup—Enters operational parameters that align with a BBE Cloudsetup created cluster so that you do not have to interact with APM during the setup process (see the BBE Cloudsetup Installation Guide for cluster installation instructions).

      Note:

      Only use either the bbecloudsetup option or the template file-name option. You cannot use both options.

    • update—You will only be prompted for missing values during setup.

    • ssh host:port—A hostname or IP address of the cluster (any of the cluster’s nodes) and open port used for SSH access to the CLI.

      Note:

      Enabling SSH access requires the mgmt microservice to run in privileged mode.

    • secrets—Updates the keys, certificates, and secrets used by APM.

    • verbose—Provides a detailed description before each prompted question.

    • config config-file-path-name—The name of the initial configuration file that you want APM to use during startup.

      Note:

      You can use an initial configuration file to start and roll out APM. You use the configuration file through the --config config-file-path-name switch on the utility script’s setup command.

      When APM is started or rolled out, the configuration file that you supply during setup is used to initialize APM. If you do not supply a configuration file, APM starts with the factory defaults. If the BBE Event Collection and Visualization application is running on the cluster, the factory defaults include the bbe-ecav syslog server configuration.

      The supplied configuration file is stored on the jumphost’s context repository. This allows the configuration to be preserved across APM start and stop events. Commits to the initial configuration are not automatically saved to the persistent location on the jumphost. To update the configuration at the persistent location, use the utility script’s save-config command.

      Using the save-config command ensures that the latest configuration is used the next time that APM is started and rolled out. In order to restore the initial configuration back to its factory default, enter setup interactively and enter ^d to the startup config ... question.

      The action in the parenthesis changes to remove. Press Enter to accept the removal of the deployed configuration. APM reverts back to the factory default configuration after a stop and then rollout command sequence.

      When you change the initial configuration file using the utility script’s setup command, you must perform a stop and then rollout command sequence for the change to take effect.

    • template file-name—A YAML formatted file that contains a subset of the operational parameters file that is created during setup. The values that are entered in the template file are used automatically by the setup process. When you use the template option, you are not required to manually enter the information contained in the template file during the setup process. Table 3 describes the information that you can enter into the template configuration file.

      Note:

      Only use either the bbecloudsetup option or the template file-name option. You cannot use both options.

    • mandatory—Only asks required questions during setup.

    • optional—Only asks questions that are not required during setup.

    Table 3: Setup File Field Descriptions

    Field

    Description

    (Optional) Service account name

    The name of the Kubernetes service account used to bind certain operational privileges to the mgmt microservice. If a service account name is not provided, APM creates a service account named apm-svca during rollout.

    External registry address

    The external registry address is an FQDN that the container images are pushed to.

    Registry for k8s to pull from

    The transport address or FQDN:port that the container images are pulled from.

    (Optional) APMi secrets

    To secure the APMi (recommended),enter one of the following:

    • The name of a Kubernetes secret in the APM namespace that contains the TLS secret data (root Certificate Authority certificate, certificate, and private key)

    • Key files (root Certificate Authority certificate, certificate, and private key)
    Note:

    If a secret is provided, you will not be prompted for the Key files during installation.

    Startup config to mount into mgmt pod on rollout

    The configuration file to use for the initial configuration. If a configuration file is not provided a factory default configuration is used.

    (Optional) Address for operator's backup communication service (MGeo only)

    IP address for the operator's backup communication service. Enter an unused IP address from the backup subnet that the cluster nodes of each geography are connected to. This field is used in multiple geography setups only.

    Network load balancer APMi static IP address

    Static IP address for the APMi network load balancer.

    (Optional) APMi port

    The APMi port number (default is 20557).

    (Optional) Configuration archival server

    When you configure the Configuration archival server option, APM archives a copy of the updated configuration to an external server after each successful commit.

    To configure the server information where configuration file changes are archived, you must enter the following information:

    • ssh-key information. Provide information for one of the following:

      • The name of a Kubernetes Secret in the APM namespace that contains the SSH private key data.

      • The name of the SSH private key file.

      Note:

      If a secret name is supplied, you will not be prompted for the SSH private key file.

    • The Secure Copy Protocol (SCP) URL of the server where the configuration file will be archived.

      Note:

      The URL must use the following format: scp://user-login@server-fqdn:server-port/absolute-file-path (for example, scp://user@host1.mydomain.com:30443/home/user/configs/apm).

      Upon successful commit, an SCP transfer of the candidate configuration is transferred to the archival URL as a compressed file with the name:

      apm-identifier_YYYYMMDD_HHMMSS_juniper.conf.n.gz

      • apm-identifier is the external IP address of the APMi interface.

      • YYYYMMDD_HHMMSS is the time stamp in Coordinated Universal Time (UTC).

      • n is the number designation of the compressed configuration rollback file.

    (Optional) DBSync service type

    The apm multi-cluster status APM utility command collects the state to display from the DBSync microservice through a Kubernetes service. By default, the NodePort service is created for this purpose. If you select LoadBalancer, you are prompted for an external IP address and a MetalLB pool is created containing the supplied external IP address. The LoadBalancer service created at rollout is assigned the external IP address from the newly created MetalLB pool.

    Note: This field is used in multiple geography setups only.

    Preferred Active Cluster

    The name of the cluster (workload context) on which the management microservice should initially become active.

    Name of the storage class to use for APM config

    Storage class name to use (default is jnpr-bbe-storage).

    Amount of storage to reserve for APM config path

    Amount of storage to preserve.

    (Optional) Syslog Details

    If you want to export APM log information to an external syslog collector,enter the following syslog server information:

    • IP address or fully qualified domain name

    • Port number

    Syslog information is included in the generated factory default configuration file. If you did not use the generated factory default configuration file, and used your own initial configuration file, you must include the system syslog host stanza containing the connection details for the syslog server.

    (Optional) Operator backup channel TLS information (MGeo only)

    The operator backup channel is an alternate subnetwork between the two workload clusters. This subnetwork is used to facilitate health monitoring and for coordinating switchover if the inter-cluster subnetwork (that the Submariner tunnel traverses) is disconnected.

    Enter the following information for the operator backup channel:

    • To secure the operator backup channel (recommended), enter one of the following:

      • The name of a Kubernetes secret in the APM namespace that contains the TLS secret data (root Certificate Authority certificate, certificate, and private key)

      • Key files (root Certificate Authority certificate, certificate, and private key)

    • An IP address for the operator backup channel on each workload cluster. If an address is not provided, a backup channel will not be established. It is recommended that you provide backup addresses for each workload cluster. The backup channel ensures that the multiple geography deployment continues functioning in the event of a loss of the inter-cluster subnetwork.

    Note:

    If a secret is provided, you will not be prompted for the Key files during installation.

    (Optional) Initial APM configuration file

    The configuration file that is used at APM startup. You enter the file name when you use the config config-file-path-name option in the setup command.

    (Optional) Cluster storage-class name

    The name of the Kubernetes storage class to use for creating persistent volume claims (PVCs). The management microservice uses a PVC to record the configuration state. You enter the name of the Kubernetes storage class when you see the Name of the storage class to use for APM config prompt during setup.

    (Optional) Cluster storage size

    The amount of storage to reserve for APM the configuration path, in mebibytes (MiB).

    (Optional) Type of apm SSH service

    If SSH access to the mgmt microservice is specified (ssh <ip>:<port>), you must specify whether the service should be created as a node port (NodePort) service or a load balancer (LoadBalancer) service. If LoadBalancer is selected, a MetalLB pool is created containing the supplied external IP address. The load balancer service created at rollout is assigned the external IP address from the newly created MetalLB pool.

    How you enter the SSH information is different for single and multiple geography setups. See the following for your specific setup:

    • Single geography setup—You can provide an SSH address using the setup command (ssh option). If you provided an SSH address with the ssh option, the setup process prompts you for the service type:

      Type of apm SSH service

      -- LoadBalancer

      -- NodePort

    • Multiple geography setup—The SSH address can only be provided through the template file.

    Internal (Docker) registry transport address (fqdn:port)

    The internal registry transport address is the address from which the container images are pulled from during rollout. This address is typically different than the external registry address.

    (Optional) Number of provman worker processes

    The number of provman worker processes determines how simultaneous processes provman deploys to handle the entity workload. We suggest that you plan for 20 entities per process. Each process can consume a CPU core on the node it is running on. Therefore, the nodes in the cluster must have sufficient CPU cores to support the number of provman processes (plus any other workloads that may be running on a node).

    You can configure 1 to 10 worker process (default is 3).
  8. Verify the APM installation by running the apm version command.

    • context context-name—The Kubernetes context name of the cluster.

    • detail—Displays all available software versions.

Start APM in a Single Geography Setup

Use this procedure to configure and start APM in a single geography setup.

  1. Enter rollout to start the APM installation. You need to use the rollout command with sudo/as root. The rollout command validates that all the values needed for the new release are present and loads the new release container image to the registry. Use sudo -E apm rollout --context context-name to start APM services. For example:

    • context contex-name—The Kubernetes context name of the cluster.

    Note:

    By default, APM starts with the values that you provided during setup. Unless the configuration was saved, the initial configuration is what was entered during setup. All other persistent states (logs, database keys, and so on) are cleared.
  2. Enter apm status --context context-name [-o|--output json] [--detail] to verify that the APM services are up and running. For example:
    Note:

    Collect the logs for a service and contact the Juniper Networks Technical Assistance Center (JTAC) when either of the following occurs:

    • The service is not running.

    • The service’s uptime compared with other services indicates that it has restarted.

Prepare for APM Installation in a Multiple Geography Setup

Use the installation procedures in this section for an APM setup that consists of multiple APMs that are located in different geographical locations.

Before you begin, confirm that you meet the requirements for the APM installation (see Table 2).

Prerequisites

Before starting the APM installation, make sure that you have the following information:

For descriptions of the following information, see Table 3.

Required Information:

  • The cluster context names of the workload clusters.

    For example, your context output might look like the following:

  • Container registry details for each cluster:

    Note:

    You must collect the following information for all the clusters.

    • External registry address

    • External registry port number (usually 5000)

Optional Information:

  • APM initial configuration file. If a configuration file is not supplied, a basic configuration file is automatically generated.
  • Storage class name for persistent volume claim (PVC) creation (default is jnpr-bbe-storage).
  • PVC Size (default is 90 MiB).
  • Archival configuration details. This is required if you are planning to mirror a copy of the APM configuration to an external server.

    • Either the name of the SSH private key file or the name of the Kubernetes secret that is present in the jnpr-apm namespace containing the SSH private key.

    • The Secure Copy Protocol (SCP) URL of the server where the configuration file will be archived. An SCP URL takes the form of scp://user-login@server-fqdn:server-port/absolute-file-path (for example, scp://user@host1.mydomain.com:30443/home/user/configs/apm).

  • Syslog server details. This is required if you are planning to export APM logs to an external syslog collector.
    Note:

    If BBE Event Collection and Visualization is detected running on the target cluster, the address and port values of the ECAV deployment will be suggested as the default.

    • Syslog server address.

    • Sysylog server port number.

  • APMi Details—You can provide a specific IP address to use as the external load balancer IP address for the APMi on each workload cluster. If a specific address is not provided, APM attempts to allocate an external address from the network load balancer's default pool:

    • External IP address—Enter an unused IP address from a subnet that the cluster nodes and the entities are connected to.

    • Port (default is 20557)
    • TLS details. You will need one of the following:

      • None (insecure)

      • Either the key and certificate files, or the name of the Kubernetes secret that is present in the jnpr-apm namespace that contains the key and certificate information.

  • Inter-operator backup channel details (for multiple geography setups only)—It is recommended that you setup a backup channel over a different subnet other than the primary subnet on which the workload clusters connect (and the Submariner tunnel connects over).

    • External IP address—Enter an unused IP address from the backup subnet that the cluster nodes of each geography are connected to.

    • TLS details. You will need one of the following:

      • None (insecure)

      • Either the key and certificate files, or the name of the Kubernetes secret that is present in the jnpr-apm namespace that contains the key and certificate information.

  • Service account name—The name of the Kubernetes service account used to bind certain operational privileges to the mgmt microservice. If a service account name is not provided, APM creates a service account named apm-svca during rollout.

  • DBSync service type—The apm multi-cluster status APM utility command collects the state to display from the DBSync microservice through a Kubernetes service. By default, a node port service is created for this purpose. If you select LoadBalancer, you are prompted for an external IP address and a MetalLB pool is created containing the supplied external IP address. The LoadBalancer service created at rollout is assigned the external IP address from the newly created MetalLB pool.

  • Number of worker processes for the provman microservice (default is 3).

Install the APM Application (Multiple Geography Setup)

  1. Download the APM software package from Juniper Networks software download page to the jump host.

    APM is available as a compressed TAR (.tgz) file. The filename includes the release number as part of the name. The release number has the format: <Major>.<Minor>.<Maintenance>

    • major is the main release number of the product.
    • minor is the minor release number of the product.
    • maintainance is the revision number.
  2. Unpack the APM TAR (.tgz) file on the jump host by entering:
  3. Run the loader script after you unpack the TAR file.
  4. Use the sudo -E apm link command to link to the cluster. In preparation for running setup, the link command takes the list of workload cluster contexts and associates them to the loaded APM software package.

    • context multi-cluster-context-name—The context name for a multiple geography setup. The multi-cluster-context-name is a user-defined string applied to the link command that serves as a common reference for the two workload clusters.

    • workload-contexts workload-1-context-name workload-2-context-name—The two workload context names.

    • version software-release—The APM software version, as displayed from the apm_loader output.

    Figure 1 shows where the different contexts are located in a multiple cluster setup.

    Figure 1: Multiple Geography Cluster Multiple Geography Cluster
  5. When using a RHOCP cluster, you can interact with it after authenticating the OpenShift cluster and the three RHOCP clusters (management and two workload clusters) using the OpenShift CLI.

    For an example of the command to run, see the following:

  6. In order to push the APM container images, you must authenticate with the registry on each cluster in the multiple cluster setup. Authenticate with the registry by issuing a docker login as the system user (the system user entered in the BBE Cloudsetup configuration file) to the cluster's registry transport address (the FQDN supplied as the system address in the BBE Cloudsetup configuration file).

    For an example of the command to run, see the following:

  7. Run setup to configure your installation. The setup command does the following:

    • Collects information about the cluster environment such as container registry contact information, keys and certificates needed to secure external interfaces, persistent storage resources, and other information relevant to supporting APM features.

    • Establishes the operational parameters for the Kubernetes deployment.

      If you did not use the template file-name option with the setup command, you need to complete these prompts during the setup:

      • If you are using Rancher Kubernetes Engine2 or BBE Cloudsetup to create your cluster:

        • External registry address.

        • External registry port number.

      • If you are using a Red Hat OpenShift Container Platform cluster:

        • External registry (FQDN)

        • Internal (Docker) registry address

        • Internal (Docker) registry port number

      • Enter the following for each workload cluster when prompted:

        • Name of the cluster

        • Cluster registry address and port number

    Note:

    When running setup, you can interact with the setup process by entering ^d.

    If you want to change a value after entering it, enter ^d. After entering ^d, the value you previously entered is removed and the default value is used for the question. You can use the ^d operation for any setup questions that are optional, or for which a list of values can be provided.

    Following is an example setup command:

    Note:

    To set up CLI access through SSH in a multiple geography setup, you must use a template file (use the template option in the setup command). This enables you to configure the two workload cluster addresses.

    To configure SSH, add the following information (YAML formatted) for each workload cluster to the template file that you provide during the setup process:

    For more information regarding the SSH configuration in the template file, see the following:

    • ip-address—The IP address you use to manage the cluster from the jump host.

    • cluster-name—The name of the workload cluster as it appears in the output of the kubectl get clusters command.

    • available-port-value—If the NodePort option is entered in the type field, the port value must be a TCP port that is not used on any of the workload cluster's nodes. A best practice is to avoid ports that are already in use (like the often used SSH port 22), but below the ephemeral port range (port 49152 and higher). This avoids possible port contention with the node itself.

    • service-name—The name you want the created service to use. A best practice is to include the application name, the purpose, and the workload cluster in the name (for example, apm-ssh-workload1).

    The options that you can use with the setup command are listed in the following:

    • context multi-cluster-context-name—The context name for a multiple geography setup. The multi-cluster-context-name is a user-defined string applied to the link command that serves as a common reference for the two workload clusters.

    • h, help—Shows the help message and exit.

    • l, log [error, warning, info, debug]—Adjusts the log level.

    • no-color—Prints messages without colors.

    • update—You will only be prompted for missing values during setup.

    • secrets—Updates the keys, certificates, and secrets used by APM.

    • verbose—Provides a detailed description before each prompted question.

    • config file-name—The name of the initial configuration file that you want APM to use during startup.

    • template file-name—A YAML formatted file that contains a subset of the configuration file that is created during setup. The values that are entered in the template file are used automatically by the setup process. When you use the template option, you are not required to manually enter the information contained in the template file during the setup process. You should only use the template option when using Red Hat OpenShift Container Platform to create the cluster or when creating a multiple geography cluster. Table 3 describes the information that you need to enter into the template configuration file.

    • mandatory—Only asks required questions during setup.

    • optional—Only asks questions that are not required during setup.

  8. Verify the APM installation by running the apm version command.

    • context multi-cluster-context-name—The context name for a multiple geography setup. The multi-cluster-context-name is a user-defined string applied to the link command that serves as a common reference for the two workload clusters.

    • detail—Displays all available software versions.

Start APM in a Multiple Geography Setup

Use this procedure to configure and to start APM in a multiple geography setup.

  1. Enter rollout to start the APM installation. The APM utility allows you to roll out different software versions for all microservices that are part of your APM multiple geography setup. You need to use the rollout command with sudo as root. The rollout command also validates that all the values needed for the new release are present and loads the new release container images to the registry. For example:

    • context multi-cluster-context-name—The context name for a multiple geography setup. The multi-cluster-context-name is a user-defined string applied to the link command that serves as a common reference for the two workload clusters.

    • service service-name—The microservice name to rollout.

    • version software-release—The software release to rollout (defaults to the release that links to the cluster).

    Note:

    On the, first rollout service is not required. The service is used with the version to rollout (upgrade) specific versions of specific services.
    Note:

    By default, APM starts with the values that you provided during setup. Unless the configuration was saved, the initial configuration is what was provided during setup. All other persistent states (logs, database keys, and so on) are cleared.
  2. Enter apm status --context multi-cluster-context-name --detail to verify that the APM services are up and running. For example:
    Note:

    Collect the logs for a service and contact the Juniper Networks Technical Assistance Center (JTAC) when either of the following occurs:

    • The service is not running.

    • The service’s up time compared with other services indicates that it has restarted.