A nice way to do failover with bgp, ospf and hsrp

My setup is as follows:

Two 3550 switches, each connected to a different provider via BGP. I would like to have redundancy between the two switches/providers. Here's what I did:

1. Routing:
a) Each switch should have a route preference 190 towards the other switch. While BGP is up, we will have a default route [preference 20] towards the corresponding provider. If BGP is down, route will switch to the 190 preference route we just added. So we will go out via our second L3 switch.
b) Each router will be connected in both switches. I was thinking first to use bridge irb, but it looks more natural to use ospf. So I setup two IP ranges (172.16.1.0/24 [sw1] and 172.16.2.0/24 [sw2] ) to use with OSPF. Lets take for example one of the routers, a 2651XM. Fa0/0 is connected to port fa0/2 in SW1 and Fa0/1 is connected to port fa0/2 in SW2.

Below, an example config (we will discuss this below the sample config)
[sw1]

interface FastEthernet0/2
description to R01, fa0/0
switchport trunk encapsulation dot1q
switchport mode trunk

router ospf 1
network 172.16.1.0 0.0.0.255 area 0
network 172.16.2.0 0.0.0.255 area 0
network 129.x.x.0 0.0.0.255 area0
default-information originate always

interface vlan 100
ip address 129.x.x.11 netmask 255.255.255.0
standby 100 ip 129.x.x.1
standby 100 priority 255
standby 100 preempt

ip route 0.0.0.0 0.0.0.0 129.x.x.12 190


[sw2]

interface FastEthernet0/2
description to R01, fa0/1
switchport trunk encapsulation dot1q
switchport mode trunk

router ospf 1
network 172.16.1.0 0.0.0.255 area 0
network 172.16.2.0 0.0.0.255 area 0
network 129.x.x.0 0.0.0.255 area0
default-information originate always

interface vlan 100
ip address 129.x.x.12 netmask 255.255.255.0
standby 100 ip 129.x.x.1
standby 100 priority 101
standby 100 preempt


ip route 0.0.0.0 0.0.0.0 129.x.x.11 190



[r01]

interface FastEthernet0/0
description to SW1, fa0/2
no ip address

interface FastEthernet0/0.1
encapsulation dot1q 101
ip address 172.16.1.3 255.255.255.0

interface FastEthernet0/1
description to SW2, fa0/2
no ip address

interface FastEthernet0/1.1
encapsulation dot1q 102
ip address 172.16.2.3 255.255.255.0

interface Loopback0
ip address 129.x.x.54 255.255.255.255
ip ospf 1 area 0

router ospf 1
log-adjacency-changes
network 129.x.x.54 0.0.0.0 area 0
network 172.16.1.0 0.0.0.255 area 0

ip route 0.0.0.0 0.0.0.0 129.x.x.1

Now, how does this work?

- each router has a default route towards the other, preference 190 . As said above, when BGP fails on any of the switches, traffic goes out via the other switch.
- if both bgp sessions fail then.... well, thats just bad luck :)

How will my packets select a certain ISP?

- its gonna be due to HSRP protocol (hot standby router protocol). Notice on the second switch the 'standby 100 priority 101' statement. Default is 100, but I put it to 101 to make the config more self explanatory. The first switch has priority 255, which means in the HSRP group, the first switch will always be elected as default gateway.

What if HSRP fails?

- hsrp won't fail. If one of the switches fails, the switch still up will take the role as default gateway. (and use that switches bgp connection).


Other things: OSPF will always let know the switches on which interface they can find the 129.x.x.54 IP address. Check this out:

<------------------------------------------------------------>
sw2# sh ip route ospf
129.x.x.0/24 is variably subnetted, 11 subnets, 5 masks
O 129.x.x.54/32 [110/2] via 172.16.1.3, 3d01h, Vlan101
[110/2] via 172.16.2.3, 3d01h, Vlan102
<------------------------------------------------------------>

What else can I do?

- have each provider connect in each switch. In my config I could only afford provider A in switch 1 and provider B in switch 2. If you will be able to setup provider A & B in switch 1 and provider A & B in switch 2 as well, then this is going to be 100% redundant.

Any ideas, comments?

Dynamips / Dynagen on FreeBSD, connect to tap0

A real world example:

I needed to send calls directly to a router but didn't want to mess with production routers.
I had a FreeBSD box running so I decided to install dynamips and dynagen and put a IP2IP image on it.


How to do it:

1. fetch the image that you need from one of your routers (or from cisco's website).
2. install from /usr/ports dynagen and dynamips.
3. check out my config file and adapt to your needs:

[localhost]

[[3745]]
image = c3745......[cut here]
ram = 300
rom = 64
nvram = 300
mmap = true
cnfg = None
confreg = 0x2102

[[ROUTER IP2IP]]
model = 3745
console = 2000
f0/0 = NIO_gen_eth:tap0

then save and do as written below:

kldload if_tap
touch /dev/tap0
ifconfig tap0 up

kldload if_bridge
ifconfig bridge0 create
ifconfig bridge0 addm eth0 addm fxp0


ifconfig bridge0:

bridge0: flags=8843 metric 0 mtu 1500
ether 26:0a:67:8d:3e:3c
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: eth0 flags=143
member: fxp0 flags=143


Now I am able to reach the internet from my virtual router.

dynamips -H localhost &
dynagen ip2ip.cfg &


[root@cluster2 /home/dan/simulator]# dynagen ip2ip.cfg
Reading configuration file...

Shutdown in progress...
Shutdown completed.
*** Warning: Starting IP2IP with no idle-pc value
CPU0: carved JIT exec zone of 64 Mb into 2048 pages of 32 Kb.
C3745 instance 'IP2IP' (id 0):
VM Status : 0
RAM size : 300 Mb
NVRAM size : 300 Kb
IOS image : c3745-adventerprisek9_ivs-mz.124-25.bin

Loading ELF file 'c3745-adventerprisek9_ivs-mz.124-25.bin'...
ELF entry point: 0x80008000

C3745 'IP2IP': starting simulation (CPU0 PC=0xffffffffbfc00000), JIT enabled.
Network successfully loaded

Dynagen management console for Dynamips and Pemuwrapper 0.11.0
Copyright (c) 2005-2007 Greg Anuzelli, contributions Pavel Skovajsa

=>
=>
=>
[2]+ Stopped dynagen ip2ip.cfg
[root@cluster2 /home/dan/simulator]# telnet localhost 2000
Trying localhost...
Connected to localhost.
Escape character is '^]'.
Connected to Dynamips VM "IP2IP" (ID 0, type c3745) - Console port

############################################################################################################## [OK]

If you don't manage to connect to the internet like that, you could as well select the main interface:

f0/0 = NIO_gen_eth:fxp0

You should assign an ip address from the same range to f0/0. Worked for me.