ASA Clustering (Spanned EtherChannel)

  • Post author:
  • Post category:ASA / Cisco

The Cisco ASA supports clustering feature to combine up to 16 appliances, each cluster unit actively forwards transit traffic (unlikely failover), acting as a single unit. Adjacent switches statelessly load balance traffic between available cluster members. A cluster internally compensates for asymmetric traffic, so different units may receive packets.

Modes

ASA clustering supports two modes:

Spanned EtherChannel (Cisco recommended)

  • Multiple physical interfaces of all cluster members bundled into a single cluster spanned EtherChannel.
  • All interfaces within the EtherChannel share the same cluster virtual IP and MAC address.
  • Connected switches see the cluster as a single logical unit and statelessly load-balance incoming traffic using the EtherChannel hashing algorithm.
  • Supports both routed and transparent firewall modes.

Individual Mode

  • Each member appears as a separate device.
  • Each unit uses a separate IP and MAC address on its data interfaces.
  • Bundle interfaces are supported but are local to the individual units.
  • Dynamic or Policy Based routing protocols load balance traffic across a cluster in individual mode.
  • Only routed firewall mode is supported

Unit Roles

Master and Slave Unit

  • One member of the cluster is elected as the master, all other units become slaves.
  • The first unit becomes the master and remains in that role until it reloads or fails.
  • The master unit provides the following functions:
    • Cluster configuration – configure of the cluster from the master
    • Main cluster IP address ownership – the master unit always receives, and traffic destined to the virtual IP address
    • Centralised connection handling – certain features and functions require processing on the master, such as IGMP, PIM or application inspected connections, such as TFTP. Slave units redirect these incoming packets to the master.

Flow Owner

  • A single cluster member must process all packets that belong to a single connection, ensuring symmetry.
  • Each cluster may own some flows
  • Typically, the unit which received the first packet for a connection becomes the owner.
  • If the original owner fails while a connection is active, another unit assume ownership of the flow.
  • A flow owner always maintains the full stateful connection entry.

Flow Director

  • The flow director always maintains the backup stateful information record.
  • This backup flow ownership allows another unit to recover the connection state and assume ownership if the original owner fails.
  • The flow owner periodically updates the flow director on the connection state.
  • The flow director provides the flow owner to the other cluster members.
  • Each cluster member knows which unit is the flow director for every possible transit connection.
  • Clustering uses a hash value for each flow with Source and Destination IP address, Source and Destination TCP/UDP ports.
  • The hash values are evenly divided among the cluster members.
  • Every unit in the cluster becomes the flow director for some connections.
  • All cluster units use the same function to determine the hash value, they can consistently identify the flow director.
  • The flow director maintains a stube connection entry with limited stateful information.

Flow Forwarder

  • As the flow owner must process the packets for a single connection, other units must forward the asymmetrically received packets to the correct owner.
  • The Flow Forwarder creates a forwarding stub entry for connections to avoid director queries.
  • This stub record points to the current owner.

Cluster Control Link

In both Spanned EtherChannel and Individual Mode a Cluster Control Link (CCL) interface is used to interconnect the ASA together, for control traffic and data traffic.

  • Control traffic includes Control unit election, configuration replication and health monitoring.
  • Data traffic includes state replication and connection ownership queries and data packet forwarding.

The CCL interfaces must be connected to a switch and isolated from other data traffic.

In a 2-member cluster, the ASA CCL must not be directly connected. If directly connected and one unit fails, the CCL fails, and the remaining health unit will fail.

A CCL can either be a single interface or a port-channel consisting of multiple interfaces bundled together (recommended).

Configuration

The following section will cover configuring ASA clustering using Spanned EtherChannel. The figure below represents the topology used.

Set the MTU value for the Cluster Control Link (CCL) to 1600, a prerequisite of this is to enable jumbo-frame reservation and optionally TCP MSS may need to be adjusted.

Clear the current configuration.

clear configure all

Configure the cluster interface mode to Spanned EtherChannel.

cluster interface-mode spanned

Configure jumbo-frame reservation and reboot. Post reboot, configure MTU 1600 for the CCL.

jumbo-frame reservation
WARNING: This command will take effect after the running-config is saved and the system has been rebooted. Command accepted.
INFO: Interface MTU should be increased to avoid fragmenting
      jumbo frames during transmit
ASA-1/PRI/master(config)# reload
Proceed with reload? [confirm]
!
mtu cluster 1600

Configure the data interfaces and bundle into Port Channels and no shutdown the CCL interface.

interface GigabitEthernet 0/0
 description OUTSIDE interface
 no shutdown
 channel-group 1 mode active
!
interface GigabitEthernet 0/1
 description INSIDE interface
 no shutdown
 channel-group 2 mode active
!
interface GigabitEthernet 0/2
 description Cluster interface
 no shutdown

Configure the Port Channels, the port channel must be configured with port-channel span-cluster when cluster interface-mode is spanned. Configure a unique MAC address per port-channel and the nameif, IP address and security-level.

interface port-channel 1
 port-channel span-cluster
 mac-address 1200.0000.0001
 nameif OUTSIDE
 ip address 192.168.178.254 255.255.255.0
 security-level 0
 no shutdown
!
interface port-channel 2
 port-channel span-cluster
 mac-address 1200.0000.0002
 nameif INSIDE
 ip address 192.168.10.3 255.255.255.0
 security-level 100
 no shutdown

Create the cluster group, define a key (must be the same across all units), define the local-unit ID, cluster interface and unique IP address and priority. Lastly enable the cluster.

cluster group LAB
 key Cisco1234
 local-unit LAB-MASTER
 cluster-interface GigabitEthernet 0/2 ip 192.0.2.1 255.255.255.0
 priority 1
 enable

The following commands will be defined automatically.

 health-check holdtime 3
 health-check data-interface auto-rejoin 3 5 2
 health-check cluster-interface auto-rejoin unlimited 5 1
 clacp system-mac auto system-priority 1

On each of the slave units, clear the current configuration.

clear configure all

Configure the data interface mode to Spanned EtherChannel.

cluster interface-mode spanned

No shutdown the CCL interface.

interface GigabitEthernet 0/2
 description Cluster interface
 no shutdown

Configure the cluster, define a unique IP address and enable the cluster.

cluster group LAB
 key Cisco1234
 local-unit LAB-SLAVE
 cluster-interface GigabitEthernet 0/2 ip 192.0.2.2 255.255.255.0
 priority 2
 enable

Once the cluster is enabled on the slave, the data interface configuration will be synchronised to the slave as well as the firewall configuration, including ACL, objects, policy-maps etc.

ASA-1/SEC/ClusterDisabled(cfg-cluster)# enable
Detected Cluster Master.
Beginning configuration replication from Master.
WARNING: Local user database is empty and there are still 'aaa' commands for 'LOCAL'.
INFO: Webvpn Cache is disabled by default on this release.
      Please refer to the documentation to enable WebVPN Cache using CLI or ASDM.
*** Output from config line 138, "webvpn"
INFO: Interface MTU should be increased to avoid fragmenting
      jumbo frames during transmit
*** Output from config line 192, "jumbo-frame reservation"
Cryptochecksum (changed): f88f7d08 1817a779 f9922ac8 604ff9ed
End configuration replication from Master.
Cluster unit SEC transitioned from DISABLED to SLAVE

Verification

Run the command show cluster info to configure the cluster members.

ASA-1/LAB-MASTER/master# show cluster info
Cluster LAB: On
    Interface mode: spanned
    This is "LAB-MASTER" in state MASTER
        ID        : 0
        Version   : 9.8(4)17
        Serial No.: FCH1721XXXX
        CCL IP    : 192.0.2.1
        CCL MAC   : f872.ea25.xxxx
        Last join : 12:26:49 UTC Jul 7 2022
        Last leave: N/A
Other members in the cluster:
    Unit "LAB-SLAVE" in state SLAVE
        ID        : 1
        Version   : 9.8(4)17
        Serial No.: FCH193XXXXX
        CCL IP    : 192.0.2.2
        CCL MAC   : 0462.735f.xxxx
        Last join : 12:30:51 UTC Jul 7 2022
        Last leave: N/A

Run the command show cluster info packet-distribution to determine the load distribution amongst the cluster members.

ASA-1/LAB-MASTER/master# show cluster info packet-distribution
Unit            Total Rcvd (pkt/sec)    Fwd (pkt/sec)   Locally Processed (%)
PRI                 14                                     9                   29
SEC                 30                                     1                   95

Run the command show conn on the master. From the output below the flag “y” confirms the connections on the master are a backup stub flow and the “Y” flag is a director stub flow. 

ASA-1/LAB-MASTER/master# show conn
9 in use, 19 most used
Cluster:
        fwd connections: 0 in use, 4 most used
        dir connections: 11 in use, 89 most used
        centralized connections: 0 in use, 0 most used
 
TCP OUTSIDE  192.0.66.88:443 INSIDE  192.168.10.250:1113, idle 0:00:07, bytes 0, flags  Y
TCP OUTSIDE  66.102.1.156:443 INSIDE  192.168.10.250:1103, idle 0:00:07, bytes 0, flags  Y
UDP OUTSIDE  142.250.187.227:443 INSIDE  192.168.10.250:56945, idle 0:01:05, bytes 0, flags -y
UDP OUTSIDE  216.58.212.206:443 INSIDE  192.168.10.250:58880, idle 0:01:05, bytes 0, flags -y
TCP OUTSIDE  3.94.97.242:443 INSIDE  192.168.10.250:6255, idle 0:00:31, bytes 0, flags  y
TCP OUTSIDE  170.72.231.161:443 INSIDE  192.168.10.250:1040, idle 0:00:14, bytes 0, flags  y
UDP OUTSIDE  172.217.16.228:443 INSIDE  192.168.10.250:59391, idle 0:01:05, bytes 0, flags -y
TCP OUTSIDE  13.224.222.75:443 INSIDE  192.168.10.250:1111, idle 0:00:21, bytes 0, flags  y
UDP OUTSIDE  192.168.5.100:161 INSIDE  192.168.10.250:60499, idle 0:00:34, bytes 0, flags -y
TCP OUTSIDE  20.54.37.64:443 INSIDE  192.168.10.250:1098, idle 0:12:05, bytes 0, flags  Y
TCP OUTSIDE  52.98.207.18:443 INSIDE  192.168.10.250:6214, idle 0:00:02, bytes 0, flags  y

Run the command show conn detail on the slave. Note in detailed mode, the output displays the ID of the LAB-MASTER (master) and LAB-SLAVE (slave). The connection flags on the slave of UIO, indicate UP, INBOUND data and OUTBOUND data, which confirms the SLAVE unit is the owner of the connection.

ASA-1/LAB-SLAVE/slave# show conn detail | begin ID
Cluster units to ID mappings:
  ID 0: LAB-MASTER
  ID 1: LAB-SLAVE
  ID 255: The default cluster member ID which indicates no ownership or affiliation
          with an existing cluster member

TCP OUTSIDE: 104.124.172.43/443 INSIDE: 192.168.10.250/1125,
    flags UIO , idle 1m14s, uptime 1m35s, timeout 1h0m, bytes 5765, cluster sent/rcvd bytes 0/5251, owners (1,255), xlate id 0x2aaac61cb680
  Traffic received at interface OUTSIDE
        Locally received: 0 (0 byte/s)
        From director/backup LAB-MASTER: 5251 (55 byte/s)
  Traffic received at interface INSIDE
        Locally received: 514 (5 byte/s)

TCP OUTSIDE: 3.94.97.242/443 INSIDE: 192.168.10.250/6255,
    flags UIO , idle 44s, uptime 7m28s, timeout 1h0m, bytes 9807, cluster sent/rcvd bytes 0/5526, owners (1,255), xlate id 0x2aaac61cb680
  Traffic received at interface OUTSIDE
        Locally received: 0 (0 byte/s)
        From director/backup LAB-MASTER: 5526 (12 byte/s)
  Traffic received at interface INSIDE
        Locally received: 4281 (9 byte/s)

Run the command show cluster access-list. From the output below we can determine the hit counts. 25 hits in total (aggregated result), 21 on the master (LAB-MASTER) and 4 on the slave (LAB-SLAVE). 

ASA-1/LAB-MASTER/master# show cluster access-list
hitcnt display order: cluster-wide aggregated result, LAB-MASTER, LAB-SLAVE
access-list cached ACL log flows: total 0, denied 0 (deny-flow-max 4096)
            alert-interval 300
access-list OUTSIDE_IN; 4 elements; name hash: 0xe01d8199
access-list OUTSIDE_IN line 1 extended permit icmp any any echo (hitcnt=0, 0, 0) 0x869bdf05
access-list OUTSIDE_IN line 2 extended permit icmp any any echo-reply (hitcnt=21, 21, 0) 0xc857b49e
access-list OUTSIDE_IN line 3 extended permit icmp any any unreachable (hitcnt=0, 0, 0) 0xec6c9a23
access-list OUTSIDE_IN line 4 extended permit icmp any any time-exceeded (hitcnt=45, 45, 0) 0x00c3b80d

For the full configuration of ASA Cluster Spanned EtherChannel, refer to the templates section of this website.

Tags: ,