Etherchannel between Linux (CentOS) and Cisco switches – Virtualbox how to

In my previous post about SPAN and RSPAN, one of my readers posted a comment saying that he had problems to configure an ether-channel between his CentOS server and a Cisco switch.
I don’t actually have a PC with two NICs, so I decided to do everything with VirtualBox as it’s integrated in GNS3.

Although is not my goal to describe you how to set up VirtualBox and install CentOS, I will still show you some important details to make your installation interact properly with GNS3.

Preliminary Configurations

VirtualBox

As a first step we need to add some NICs to our VirtualBox. To do so, start VirtualBox and click on File –> Preferences –> Network. Click on the “add host only network” icon on the right. In my example I want to configure an ether-channel with two links, therefore I’ll need only two NICs.

network-adapter

You’ll need Administrator’s rights, and accept the UAC prompt that will pop out, but is really straight forward. Once you have created a new virtual machine, right click on it, and select Settings –> Network.

Here we need to enable two Host only network adapters with cable disconnected. Also be sure that while selecting the NICs, you’ll select a different one for each virtual NIC.

network-adapter-1
network-adapter-2

On the VirtualBox side that’s about it. Now let’s check what we need to do on GNS3.

GNS3

After you have done creating the virtual machine, open GNS3 and click on Edit –> Preferences –> GUI Settings tab. In this tab you will need to select “Always use manual mode when adding links” as shown in the screen shot below.

gui-settings

Now click on VirtualBox –> VirtualBox Guest tab. At this point, click on Refresh VM List and you should see your virtual machine listed in the drop down VM List. Add a name, two NICs, and finally save and close.

virtual-box

Now, hopefully you have already completed the installation of CentOS, so we should be good to create our topology.

Drag and drop our EtherSwitch Router and the VirtualBox guest on the topology. Then connect port E1 of the host to port Fa1/0 on the switch, after this, connect port E2 of the host with port Fa1/1 on the switch. When ready, power up your gears!

ether-topo

Configure the Ether-channel

Configure CentOS side

To configure an ether-channel on CentOS (other linux distributions might slightly differ) follow those steps.

  • With the command ifconfig | grep Ether identify your ethernet NIC names. If you followed my guide we will have only eth0 and eth1
  • [root@localhost ~]# ifconfig | grep Ether
    eth0      Link encap:Ethernet  HWaddr 08:00:27:1D:3E:9D
    eth1      Link encap:Ethernet  HWaddr 08:00:27:C3:44:AD
    
  • Create the file /etc/modprobe.d/bondig.conf and add the line alias bond bonding
  • Create the file /etc/sysconfig/modules/bonding.modules with the following content:
  • #!/bin/bash
    modprobe bonding
    
  • Make bonding.modules executable with the command chmod +x /etc/sysconfig/modules/bonding.modules (those two steps are required to have the driver loaded on boot)
  • create the file /etc/sysconfig/network-scripts/ifcfg-bond0 with the following content: (bond0 is our port-channel interface with relative IP settings that can be changed as needed)
  • DEVICE=bond0
    ONBOOT=yes
    USERCTL=no
    IPADDR=10.1.1.2
    NETMASK=255.255.255.0
    NETWORK=10.1.1.0
    BROADCAST=10.1.1.255
    GATEWAY=10.1.1.1
    
  • Replace the entire config of the files /etc/sysconfig/network-scripts/ifcfg-eth0 and /etc/sysconfig/network-scripts/ifcfg-eth1 as follow: (make sure to backup the existing content if needed. Note that eth0 and eth1 are the interfaces we want to bundle together)
  • In the file ifcfg-eth0:
    DEVICE=eth0
    ONBOOT=yes
    MASTER=bond0
    SLAVE=yes
    USERCTL=no
    
    In the file ifcfg-eth1:
    DEVICE=eth1
    ONBOOT=yes
    MASTER=bond0
    SLAVE=yes
    USERCTL=no
    
  • load the driver using the command modprobe bonding
  • restart the network service using the command service network restart

To verify that everything is well setup, use the command ifconfig to show the interfaces details. What we are looking at are the highlighted parts of the following output. Note how the interface bond0 is listed as MASTER and with the IP configuration, while the interfaces eth0 and eth1 show as SLAVE with no IP.

[root@localhost /]# ifconfig
bond0     Link encap:Ethernet  HWaddr 08:00:27:C3:47:ED
          inet addr:10.1.1.2  Bcast:10.1.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:2699 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2737 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

eth0      Link encap:Ethernet  HWaddr 08:00:27:C3:47:ED
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:1347 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1369 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

eth1      Link encap:Ethernet  HWaddr 08:00:27:C3:47:ED
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:1352 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1368 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

Configure Cisco side

The Cisco side of the configuration shouldn’t be too difficult. We only need to keep in mind that the server will not be able to negotiate in any way with us, so a static ether-channel configuration is required. In my example i will create a Vlan 10 and put the Port-channel within this vlan.
Keep in mind that in GNS3 some basic switching features are emulated using a NM-16ESW= card, so the following commands to create Vlans vary from the one you find in a real switch.

R1-SW#vlan database
R1-SW(vlan)#vlan 10
VLAN 10 modified:
R1-SW(vlan)#exit
APPLY completed.
Exiting....
R1-SW#show vlan-switch

VLAN Name                             Status    Ports
---- -------------------------------- --------- -------------------------------
1    default                          active    Fa1/0, Fa1/1, Fa1/2, Fa1/3
                                                Fa1/4, Fa1/5, Fa1/6, Fa1/7
                                                Fa1/8, Fa1/9, Fa1/10, Fa1/11
                                                Fa1/12, Fa1/13, Fa1/14, Fa1/15
10   VLAN0010                         active
---- OMITTED ----

Once the Vlan has been created, is time to configure our ports. Fa1/0 and 1/1 will be configured to be part of an “always on” etherchannel and to belong to Vlan 10. The SVI10 will also be configured with ip address 10.1.1.1/24.

R1-SW#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R1-SW(config)#interface range fa1/0 - 1
R1-SW(config-if-range)#switchport mode access
R1-SW(config-if-range)#switchport access vlan 10
R1-SW(config-if-range)#channel-group 1 mode on
Creating a port-channel interface Port-channel1
*Mar  1 00:16:08.871: %EC-5-BUNDLE: Interface Fa1/0 joined port-channel Po1
*Mar  1 00:16:08.927: %EC-5-BUNDLE: Interface Fa1/1 joined port-channel Po1
*Mar  1 00:16:11.811: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel1, changed state to up

R1-SW(config-if)#int vlan 10
R1-SW(config-if)#ip add 10.1.1.
*Mar  1 00:20:05.095: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan10, changed state to up
R1-SW(config-if)#ip add 10.1.1.1 255.255.255.0
R1-SW(config-if)#^Z
R1-SW#
*Mar  1 00:20:15.475: %SYS-5-CONFIG_I: Configured from console by console
R1-SW#sh ip int brief | e down
Interface                  IP-Address      OK? Method Status                Protocol
FastEthernet1/0            unassigned      YES unset  up                    up
FastEthernet1/1            unassigned      YES unset  up                    up
Port-channel1              unassigned      YES unset  up                    up
Vlan10                     10.1.1.1        YES manual up                    up

R1-SW#sh vlan-switch

VLAN Name                             Status    Ports
---- -------------------------------- --------- -------------------------------
1    default                          active    Fa1/2, Fa1/3, Fa1/4, Fa1/5
                                                Fa1/6, Fa1/7, Fa1/8, Fa1/9
                                                Fa1/10, Fa1/11, Fa1/12, Fa1/13
                                                Fa1/14, Fa1/15
10   VLAN0010                         active    Po1
---- OMITTED ----

R1-SW#sh etherchannel 1 port-channel
                Port-channels in the group:
                ----------------------

Port-channel: Po1
------------

Age of the Port-channel   = 00d:00h:02m:24s
Logical slot/port   = 8/0           Number of ports = 2
GC                  = 0x00010001      HotStandBy port = null
Port state          = Port-channel Ag-Inuse

Ports in the Port-channel:

Index   Port   EC state
------+------+------------
  0     Fa1/0    on
  1     Fa1/1    on

Time since last port bundled:    00d:00h:02m:24s    Fa1/1

To verify that everything is working, let’s ping our CentOS server and see where our mac-address will be shown in the mac-address-table. Also we can check on the server itself how traffic is balanced.

R1-SW#ping 10.1.1.2

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.1.1.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 8/17/40 ms
R1-SW#sh mac-address-table
Destination Address  Address Type  VLAN  Destination Port
-------------------  ------------  ----  --------------------
c400.0308.0000          Self          1     Vlan1
c400.0308.0000          Self         10     Vlan10
0800.27c3.47ed          Dynamic      10     Po1

[root@localhost ~]# ifconfig
bond0     Link encap:Ethernet  HWaddr 08:00:27:C3:47:ED
          inet addr:10.1.1.2  Bcast:10.1.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:2699 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2725 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:307416 (300.2 KiB)  TX bytes:308730 (301.4 KiB)

eth0      Link encap:Ethernet  HWaddr 08:00:27:C3:47:ED
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:1347 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1363 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:153504 (149.9 KiB)  TX bytes:154418 (150.7 KiB)

eth1      Link encap:Ethernet  HWaddr 08:00:27:C3:47:ED
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:1352 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1362 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:153912 (150.3 KiB)  TX bytes:154312 (150.6 KiB)

Once more, the only thing that we need to remember when we configure an ether-channel with a server is that our switch needs to be set to mode on, and not try to negotiate at all. PaGP and LACP will not work. PaGP because is Cisco proprietary, LACP because servers simply don’t support it in most of the cases.

About these ads

, , , , ,

  1. #1 by data visualization on May 13, 2013 - 6:02 am

    I will right away seize your rss feed as I can not to find your e-mail subscription hyperlink or e-newsletter service. Do you have any? Please let me recognize in order that I may subscribe. Thanks.

    • #2 by anubisg1 on May 13, 2013 - 7:01 pm

      you can actually “follow” my blog, on the top-left of the page you have the follow button (if you have a wordpress account), else on the bottom-right
      :)

  2. #3 by mulberry bags sale on May 20, 2013 - 12:25 pm

    thanks, nice post.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 522 other followers

%d bloggers like this: