This week I finally organised a BGP from my ISP, and had a look at what my options were. I’d seen the Route-based QOS mini-HOWTO a while back, and it looked like it would work ok, but had a few problems. There’s no current way it to apply tc or iptables rules selectively based on a routing decision, or even on a route table. You can match on a route realm, however. The mini-HOWTO suggests copying your BGP routes into a separate table and into a realm at the same time, and then using tc and iptable’s realm matching code.

A quick aside: route realms are best described as a collection of routes. The decision as to which realm a route is placed is made by the local administrator, and each realm can contain routes from a mix of origins. Realms are used to allow administrators to perform bulk operations on large groups of routes in an easy manner. From the iproute command reference:

The main application of realms is the TC route classifier [7], where they are used to help assign packets to traffic classes, to account, police and schedule them according to this classification.

After a bit of digging, I found a link to a patch for quagga to provide route realms support. It’s even still maintained! After a bit of battling with autotools[1], and a bit of battling with linux capabilities[2], I had it up and running.

The route realms patch page covered off the BGP configuration I needed, and now I have a set of iptables counters for national, international and total traffic (for completeness). The only bit it doesn’t cover off is graphing, but we already have a set of perl scripts which pull information from interface totals or iptables FWMARK counters, so I modified that to pull from these counters as well, and set up RRD graphs. I was previously graphing interface totals out the external nic, and it’s interesting to note that the iptables “total” traffic, while adding up to the sum of national and international, does not correspond to the interface totals.

It’s worth pointing out that, as seen in iproute command reference, the rtacct tool will grab realm counts for you without needing iptables, so if you just want to something to graph things quickly, rtacct might do the job:

rtacct has a naive limit of 256 realms however, where as the actual implementation supports a 16 bit number, so if you have a large number of realms, or you autoclassify your inbound BGP into realms based on the AS number, you will have to use iptables only

I’m currently only accounting for traffic using this mechanism, but I can also do QOS on it – tc will match directly on realm tags, and any iptables based match systems you may have can be adapted to match on a realm as well.

[1] The realms patch touched configure.ac, which then required the autotools chain to rebuild everything, but it needed a very particular combination of autoconf and automake. Because it took me an hour or so to get this right, I’ll record it here:

autoheader and autoconf above are version 2.13. I have no idea why I had to run autoconf2.13 then autoconf2.50, but it seems that this actually worked.

[2] I initially tried building against quagga-0.98.6, because the quaggarealms patch site implied this was the “stable” verson, but it seems that quagga drops priviledges too soon. This works out fine if you have “capabilities” support in your kernel, which mine didn’t. They’ve changed this behaviour in 0.99.5, and incidentally this is the version in debian etch.

Quagga is composed of several daemons, one per routing protocol and another one called Zebra acting as the kernel routing manager. Each daemon has its own configuration file and terminal interface which can be accessed by telnet.
The vtysh tool is provided to configure the Quagga router from the localhost, in a unique interface.
See the Quagga tutorial for instructions on configuring the router.

Quagga works independently from the operational system (OS) over which it is installed. This is not the case for the open source Vyatta router or the commercial routers where the OS and the routing engine are built together.
With Vyatta routers, you can access the operational system (a modified version of the Debian Linux distribution) but even this is not necessary to configure the router.
With commercial routers like Cisco or Nortel, you can access only the router interface.

It must be emphasised that Quagga owns only routing capabililies and functionalities associated with it, such as access lists or route maps. It does not provide “non-routing” functionalities such as DHCP server, NTP server/client or ssh access but it is often possible to enable them on the operating system supporting your Quagga router.
See the Routers’ comparison to get an overview about the functionalities available with Cisco, Vyatty or Quagga routers.
The Quagga Command line Interface is very similar to the Cisco IOS software allows it to be configured very easily for those who are familiar with Cisco.

And finally, little story: the word ‘quagga’ comes from an extinct animal which was a subspecies of the plains zebra.

1. QUAGGA DAEMONS:

You have to activate the Quagga daemons matching the routing protocols you want to set on your router.
zebra:
bgpd:
ospfd:
ospf6d:
ripd:
ripngd:     Interface declaration and static routing
BGP routing protocol
OSPF routing protocol
OSPF IPv6 routing protocol
RIP v2 routing protocol
RIP Ipv6 routing protocol
In the example below, the zebra and IPv4 OSPF daemon have been activated.

#vim /etc/quagga/daemons
zebra=yes
bgpd=no
ospfd=yes
ospf6d=no
ripd=no
ripngd=no

Restart the Quagga service:

#/etc/init.d/quagga restart
Stopping Quagga daemons (prio:0): ospfd zebra (bgpd) (ripd) (ripngd) (ospf6d) (isisd).
Removing all routes made by zebra.
Nothing to flush.
Loading capability module if not yet done.
Starting Quagga daemons (prio:10): zebra ospfd.

You can check the Quagga daemons status:

#ps -ef | grep quagga
UID     PID     PPID     C     STIME     TTY     TIME     CMD
quagga     4632     1     0     22:25     ?     00:00:00     /usr/lib/quagga/ospfd –daemon
quagga     4636     1     0     22:25     ?     00:00:00     /usr/lib/quagga/zebra –daemon
If a Quagga daemon doesn’t stop properly, you can kill it manually and start the quagga service:

#kill -9 “UID number”
#/etc/init.d/quagga start

2. CONFIGURATION FILES: (/etc/quagga/*.conf files)

You must create a configuration file (even if it is empty) each time you activate a Quagga daemon.
Each daemon is associated with a specific file name:

zebra:
bgpd:
ospfd:
ospf6d:
ripd:
ripngd:     zebra.conf
bgpd.conf
ospfd.conf
ospf6d.conf
ripd.conf
ripngd.conf
To create the config files, copy the sample config files as follows:
In our example, as we activated the zebra and ospfd daemons; we need to create the zebra.conf and ospfd.conf files.

#cp /usr/share/doc/quagga/examples/zebra.conf.sample /etc/quagga/zebra.conf
#cp /usr/share/doc/quagga/examples/ospfd.conf.sample /etc/quagga/ospfd.conf
Another way to do it is to create two empty files called /etc/quagga/ospfd.conf and /etc/quagga/zebra.conf. But in this case you cannot telnet a daemon, you need to configure the telnet permissions with vtsh (see below).

Finally, give user and group ownership to respectively quagga and quaggavty to the files inside the /etc/quagga directory:

#chown quagga.quaggavty /etc/quagga/*.conf
#chmod 640 /etc/quagga/*.conf
Restart the Quagga service:

#/etc/init.d/quagga restart
Top of the page

3. DEBIAN.CONF FILE

By default, the Quagga daemons are listening only to the loopback interface 127.0.0.1. It means that you can telnet a daemon only on its loopback address 127.0.0.1 and thus cannot access it remotely.

If you want to telnet a Quagga daemon remotely you can, in the /etc/quagga/debian.conf file. Either indicate one or several IP addresses or remove the -A option meaning that you can telnet a daemon on any of its IP addresses.

Here are two examples:

The ospfd daemon is listening to the 127.0.0.1 and 192.168.1.104 IP addresses.
ospfd_options=” –daemon -A 127.0.0.1 192.168.1.104″

The zebra daemon is listening to all the Linux interfaces IP addresses. We recommend using this setting. If you want to filter who can access to your router, configure access lists on the router software.
zebra_options=” –daemon ”

Here is the recommended debian.conf file:

#vim /etc/quagga/debian.conf

# If this option is set the /etc/init.d/quagga script automatically loads
# the config via “vtysh -b” when the servers are started.
# Check /etc/pam.d/quagga if you intend to use “vtysh”!
#
vtysh_enable=yes
zebra_options=” –daemon -A ”
bgpd_options=” –daemon -A ”
ospfd_options=” –daemon -A ”
ospf6d_options=”–daemon -A ”
ripd_options=” –daemon -A ”
ripngd_options=”–daemon -A ”
isisd_options=” –daemon -A ”

The “vtysh_enable=yes” setting is required to access the Quaggga router via vtysh. (see vtysh section).

Restart the Quagga service

#/etc/init.d/quagga restart
Top of the page

4. VTYSH

As indicated in the Quagga introduction, you can access the daemons by telnetting their port number because each daemon has its own configuration file and terminal interface.

zebra:
ripd:
ripng:
ospfd:
bgpd:
ospf6d:         2601
2602
2603
2604
2605
2606
By instance, to access the ospfd daemon:

#telnet localhost 2604
As it’s not very practical to configure your router by telnetting its daemons separately, vtysh has been created to configure everything in one single interface.

To use vtysh, you must first create its configuration file as follows:

#cp /usr/share/doc/quagga/examples/vtysh.conf.sample /etc/quagga/vtysh.conf

/etc/quagga/vtysh.conf
!
! Sample
!
! service integrated-vtysh-config
hostname quagga-router
username root nopassword
!

Apply correct permissions and restart Quagga:

#chown quagga.quaggavty /etc/quagga/*.conf
#chmod 640 /etc/quagga/*.conf
#/etc/init.d/quagga restart
In the example above the “service integrated-vtysh-config” setting has been disabled (recommended). In this case, when you save the config under vtysh, it will be stored in separate files depending on the protocols you activated.
Below, an example where the Quagga configuration is saved under vtysh. (The zebra and ospfd daemons have been enabled.)

#vtysh
quagga-router#write
Configuration saved to /etc/quagga/zebra.conf
Configuration saved to /etc/quagga/ospfd.conf

If you activate “service integrated-vtysh-config”, the configuration under vtysh will be saved in one file called Quagga.conf in the /etc/quagga/ directory.
With this setting, when you access a daemon via telnet, the daemon will look first to the Quagga.conf file before looking for its own file. This means that, when you telnet a device, there can be a difference between what you see after the “show run” command and the content of the associated file, for example zebra.conf.

#vtysh
quagga-router#write
Configuration saved to /etc/quagga/Quagga.conf

It is recommended to disable “service integrated-vtysh-config” because if this setting is enabled and in case of a syntax error in the Quagga.conf file, this can lead to all your daemons being unable to start up. This will not be case when “service integrated-vtysh-config” is disabled because the configurations are stored in separate files.

Check that the default “vtysh_enable=yes” setting are configured in your /etc/quagga/debian.conf file. You can read the previous paragraph about the debian.conf file to get more information.

Then it’s useful to add the “VTYSH_PAGER=more” setting in your /etc/environment file, otherwise you will see an unfriendly “(END)” blinking in the left-down corner of the screen each time your enter a command and will need to press the “q” key to continue.

#echo VTYSH_PAGER=more > /etc/environnement
Log off and log on to enable the environment setting. You can now access the Quagga router with the vtysh command:

#vtysh
Hello, this is Quagga (version 0.99.6).
Copyright 1996-2005 Kunihiro Ishiguro, et al.

quagga-router#

If you want to run a Quagga command from the Linux shell:

#vtysh -c “command”
For instance, vtysh -c “show ip route” will display the Quagga routing table.

You can use ping and traceroute to perform connectivity checks from the vtysh prompt. Of course, these two programs need to be installed on the Linux machine. Ping is generally installed by default but traceroute often not.
To install traceroute:

#apt-get install traceroute
Top of the page

5. IP FORWARDING:

IP forwarding is required to transfer packets between the network interfaces of a Linux system.
See a picture of the Linux kernel routing.

#echo “1″ > /proc/sys/net/ipv4/ip_forward
The command above will add the “1″ value inside the /proc/sys/net/ipv4/ip_forward file and thus activate the IP forwarding.
If you want to keep the IP forwarding after a Linux reboot:

#echo “net.ipv4.ip_forward = 1″ >> /etc/sysctl.conf
It is possible to check the ip_forwarding status under the Quagga router:

#show ip forwarding
IP forwarding is on

In this case the IP forwarding is activated.

Top of the page.

6. SPEED/DUPLEX:

It is not possible to set the duplex and speed settings on the Quagga plateform. You have to configure them at the Linux level.
Use the interface configuration tutorial for assistance.

Here is a comparison of some characteristics you can find on four routers: the first two are the best open source routers available, Quagga and Vyatta and the remaining two are commercial devices and best-sellers from Cisco: a standard router from the 26xx family and a Layer Three Switch 3750.

Legend: