bitwise

A nice way to do failover with bgp, ospf and hsrp

2009-12-07T15:05:00.000-08:00

My setup is as follows:

Two 3550 switches, each connected to a different provider via BGP. I would like to have redundancy between the two switches/providers. Here's what I did:

1. Routing:
a) Each switch should have a route preference 190 towards the other switch. While BGP is up, we will have a default route [preference 20] towards the corresponding provider. If BGP is down, route will switch to the 190 preference route we just added. So we will go out via our second L3 switch.
b) Each router will be connected in both switches. I was thinking first to use bridge irb, but it looks more natural to use ospf. So I setup two IP ranges (172.16.1.0/24 [sw1] and 172.16.2.0/24 [sw2] ) to use with OSPF. Lets take for example one of the routers, a 2651XM. Fa0/0 is connected to port fa0/2 in SW1 and Fa0/1 is connected to port fa0/2 in SW2.

Below, an example config (we will discuss this below the sample config)
[sw1]

interface FastEthernet0/2
description to R01, fa0/0
switchport trunk encapsulation dot1q
switchport mode trunk

router ospf 1
network 172.16.1.0 0.0.0.255 area 0
network 172.16.2.0 0.0.0.255 area 0
network 129.x.x.0 0.0.0.255 area0
default-information originate always

interface vlan 100
ip address 129.x.x.11 netmask 255.255.255.0
standby 100 ip 129.x.x.1
standby 100 priority 255
standby 100 preempt

ip route 0.0.0.0 0.0.0.0 129.x.x.12 190

[sw2]

interface FastEthernet0/2
description to R01, fa0/1
switchport trunk encapsulation dot1q
switchport mode trunk

router ospf 1
network 172.16.1.0 0.0.0.255 area 0
network 172.16.2.0 0.0.0.255 area 0
network 129.x.x.0 0.0.0.255 area0
default-information originate always

interface vlan 100
ip address 129.x.x.12 netmask 255.255.255.0
standby 100 ip 129.x.x.1
standby 100 priority 101
standby 100 preempt

ip route 0.0.0.0 0.0.0.0 129.x.x.11 190

[r01]

interface FastEthernet0/0
description to SW1, fa0/2
no ip address

interface FastEthernet0/0.1
encapsulation dot1q 101
ip address 172.16.1.3 255.255.255.0

interface FastEthernet0/1
description to SW2, fa0/2
no ip address

interface FastEthernet0/1.1
encapsulation dot1q 102
ip address 172.16.2.3 255.255.255.0

interface Loopback0
ip address 129.x.x.54 255.255.255.255
ip ospf 1 area 0

router ospf 1
log-adjacency-changes
network 129.x.x.54 0.0.0.0 area 0
network 172.16.1.0 0.0.0.255 area 0

ip route 0.0.0.0 0.0.0.0 129.x.x.1

Now, how does this work?

- each router has a default route towards the other, preference 190 . As said above, when BGP fails on any of the switches, traffic goes out via the other switch.
- if both bgp sessions fail then.... well, thats just bad luck :)

How will my packets select a certain ISP?

- its gonna be due to HSRP protocol (hot standby router protocol). Notice on the second switch the 'standby 100 priority 101' statement. Default is 100, but I put it to 101 to make the config more self explanatory. The first switch has priority 255, which means in the HSRP group, the first switch will always be elected as default gateway.

What if HSRP fails?

- hsrp won't fail. If one of the switches fails, the switch still up will take the role as default gateway. (and use that switches bgp connection).

Other things: OSPF will always let know the switches on which interface they can find the 129.x.x.54 IP address. Check this out:

<------------------------------------------------------------>
sw2# sh ip route ospf
129.x.x.0/24 is variably subnetted, 11 subnets, 5 masks
O 129.x.x.54/32 [110/2] via 172.16.1.3, 3d01h, Vlan101
[110/2] via 172.16.2.3, 3d01h, Vlan102
<------------------------------------------------------------>

What else can I do?

- have each provider connect in each switch. In my config I could only afford provider A in switch 1 and provider B in switch 2. If you will be able to setup provider A & B in switch 1 and provider A & B in switch 2 as well, then this is going to be 100% redundant.

Any ideas, comments?

Dynamips / Dynagen on FreeBSD, connect to tap0

2009-05-04T16:50:00.001-07:00

A real world example:

I needed to send calls directly to a router but didn't want to mess with production routers.
I had a FreeBSD box running so I decided to install dynamips and dynagen and put a IP2IP image on it.

How to do it:

1. fetch the image that you need from one of your routers (or from cisco's website).
2. install from /usr/ports dynagen and dynamips.
3. check out my config file and adapt to your needs:

[localhost]

[[3745]]
image = c3745......[cut here]
ram = 300
rom = 64
nvram = 300
mmap = true
cnfg = None
confreg = 0x2102

[[ROUTER IP2IP]]
model = 3745
console = 2000
f0/0 = NIO_gen_eth:tap0

then save and do as written below:

kldload if_tap
touch /dev/tap0
ifconfig tap0 up

kldload if_bridge
ifconfig bridge0 create
ifconfig bridge0 addm eth0 addm fxp0

ifconfig bridge0:

bridge0: flags=8843 metric 0 mtu 1500
ether 26:0a:67:8d:3e:3c
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: eth0 flags=143
member: fxp0 flags=143

Now I am able to reach the internet from my virtual router.

dynamips -H localhost &
dynagen ip2ip.cfg &

[root@cluster2 /home/dan/simulator]# dynagen ip2ip.cfg
Reading configuration file...

Shutdown in progress...
Shutdown completed.
*** Warning: Starting IP2IP with no idle-pc value
CPU0: carved JIT exec zone of 64 Mb into 2048 pages of 32 Kb.
C3745 instance 'IP2IP' (id 0):
VM Status : 0
RAM size : 300 Mb
NVRAM size : 300 Kb
IOS image : c3745-adventerprisek9_ivs-mz.124-25.bin

Loading ELF file 'c3745-adventerprisek9_ivs-mz.124-25.bin'...
ELF entry point: 0x80008000

C3745 'IP2IP': starting simulation (CPU0 PC=0xffffffffbfc00000), JIT enabled.
Network successfully loaded

Dynagen management console for Dynamips and Pemuwrapper 0.11.0
Copyright (c) 2005-2007 Greg Anuzelli, contributions Pavel Skovajsa

=>
=>
=>
[2]+ Stopped dynagen ip2ip.cfg
[root@cluster2 /home/dan/simulator]# telnet localhost 2000
Trying localhost...
Connected to localhost.
Escape character is '^]'.
Connected to Dynamips VM "IP2IP" (ID 0, type c3745) - Console port

############################################################################################################## [OK]

If you don't manage to connect to the internet like that, you could as well select the main interface:

f0/0 = NIO_gen_eth:fxp0

You should assign an ip address from the same range to f0/0. Worked for me.

Google Mail on multiple devices with pop3/imap

2008-07-24T15:58:00.000-07:00

Quick tip: instead of using username@gmail.com , use recent:username@gmail.com and you 'll be able to get all your messages on more than one device. Google Mail seems to 'mark' messages as read when you download them with POP3 (as the standard says it should), but does it in a funny way, so that you cannot download your messages with another computer/pda/etc later on (because they are marked as read, isn't it?). Well, on courier and other pop3/imap servers I was able to do it with no problems from 10 devices if i wanted to. So you can use recent:username@gmail.com and it'll work.

two different approaches on the same problem

2007-11-05T15:57:00.000-08:00

Let's presume your isp had the following config on your its side:

interface FastEthernet0/8
description Link to Dan - 1M
switchport access vlan 634
switchport trunk encapsulation dot1q
switchport trunk native vlan 635
switchport trunk allowed vlan 2,635
switchport mode trunk
no cdp enable
spanning-tree portfast
spanning-tree bpdufilter enable
spanning-tree bpduguard enable
spanning-tree guard none
service-policy input dan-1024

Why is it trunk? Because in between this switch and myself there is a radio bridge which is currently using vlan2 (for monitoring purpose). All of the sudden this config stops working after an IOS upgrade (says my ISP). (I was accessing vlan 635 on my side : switchport access vlan 634).

Since it was unlikely for my ISP to solve the problem (i don't know exactly what was wrong, they did not know either), here's the two approaches i found for this:

I had a 1800router with a NM8-ESW card

1. moved the cable from the radio unit to the NM-8ESW interface, put one of the fast
interfaces (fe0/0/1) to trunk mode, created vlan 634 (corresponding to
my vlan in the isp network), and after this changed the source
interface for tunnel1 from fast0/1 to vlan634 (which i configured with
ip address x.y.z.6/30). Was working, but had to try the next thing as well (see 2.)

2. put the cable from the radio unit back to fast0/1 , created a subinterface
on this (fast0/1.1) , assigned x.y.z.6/30 and encapsulation dot1q
634. Working again. Moved tunnel source to fa0/1.1

My relevant config:

ip sla monitor 1
type echo protocol ipIcmpEcho z.g.h.3
frequency 5
ip sla monitor schedule 1 life forever start-time now
ip sla monitor 2
type echo protocol ipIcmpEcho h.k.l.2
frequency 5
ip sla monitor schedule 2 life forever start-time now

track 123 rtr 2 reachability

!
interface Tunnel1
description gre pri tunnel
bandwidth 1024
ip unnumbered Loopback0
ip hold-time eigrp 31 60
ip tcp adjust-mss 1300
delay 40000
keepalive 33 3
tunnel source FastEthernet0/1.1
tunnel destination z.g.h.3

interface FastEthernet0/1.1
description radio connection x.y.z.6 to isp
bandwidth 1024
ip address x.y.z.6 255.255.255.252
encapsulation dot1q 634

the other tunnel doesnt matter, my other link was working fine.

Why I posted this? Because I need some advice. The last version is the version I'm using now, but I would like a second opinion on the two versions I've posted. Or maybe some other solution. Thanks in advance.

This summer's (minor) network upgrade.

2007-11-05T15:45:00.000-08:00

So, on Friday we started working on extending the data center. What we've done? Well, in order to accommodate the FPSO we really had to work a lot. Started at 20:00 on Friday and finished next day at 12:00. So about 16 hours. First thing done was cleaning the cabinets (ie throwing all the old cables and laying down new ones). Now what's so cool about this is that it will make life a hell lot easier since... well, i will later post a picture with the old cabinets so you'll get the big idea why now it's easier. Next, since the fiber leads were not in place yet (in the new trunking), we had to find some other leads and put them on top of the cabinets. Not a very elegant solution, but it worked. Next, since we Distribution and Access were collapsed into two Cisco 6509 switches doing HSRP. Since we had to accommodate lot of servers for the FPSO, we decided that the 6509s should be placed in the WAN part. So two stacks of 4x 3750 switches were set in place of this 6509s and .... well, to make a long story short , we switched from L2 to L3. Everything seems more organized right now, some parts have become more redundant (we had HSRP only on 6509 switches, now this moved to WAN part and everything became more redundant since everything has 2 or 3 fibers connecting here. Not to mention the new cabinets which each have 2 ups's , and since each router has two power supplies , each router is connected to two different upses. ) Tunnels are now terminating on a 3800 router (they were terminating into a 1800), and soon we'll change the raw internet router (now a 3620 which is barely over the minimum of what we need. not to say that with NBAR on place it used to choke before enabling NBAR on remote routers as well). Moved everything from Vlan1 to Vlan50 (at least for management purpose), made vlan400 for .187.0/24 class (Connecting the oil rigs ,remote locations and so on). Also Rami was kind enough to do some more overtime (I guess the guy spent about 24 hours here), and also upgraded the Call Manager so now it doesn't take like 15 seconds for each login/logoff.

The major network upgrade is not written (but took place). Kept me for almost 37 hours awake. Will post this at a later time.

idirect upgrade howto

2007-09-15T10:51:00.000-07:00

I run into this a couple of days ago. I had to upgrade a idirect router (upload a new options file), but i did not had the password for iSite anymore. Here's how i did it:

1. Reload the idirect router.
2. Connect to the router with a cisco console cable
3. Login with username root and password iDirect (defaults)
4. ps ax ; and kill all the falcon processes.
5. # cd /etc/idirect/falcon/
# ls

falcon.opt falcon.opt.old

#
6. # mv falcon.opt falcon.opt.old
7. set your computer's ip address to be whatever the router connected to the idirect
modem/router was.
8. start a webserver on your computer
9. on falcon router : wget http://your-computers-ip-address/name-of-new-conf-file.opt
10.reload falcon router

that's it. Too simple, but worth writing for the future.

How about multiple internet connections?

2007-09-15T08:42:00.000-07:00

Lets presume that one of your customers is buying two internet connections. One is from you, the other one is from the competition. Since he is not buying alot of bandwidth, he won't have much options regarding the usage of bgp, or some other routing protocol to switch between providers when one of his links is down. Here's a quick solution:

PS: this was a cisco 851

no service pad
service timestamps debug uptime
service timestamps log uptime
service password-encryption
!
hostname example-customer
!
boot-start-marker
boot-end-marker
!
logging buffered 65535 debugging
!
aaa new-model
!
!
aaa authentication password-prompt password:
aaa authentication username-prompt Username:
aaa authentication login default local
aaa authorization exec default local
!
aaa session-id common
!
resource policy
!
clock timezone GMT+1 1
ip subnet-zero
no ip source-route
ip cef
!
!
no ip dhcp use vrf connected
!
ip dhcp pool example-customer
network 192.168.2.0 255.255.255.0
default-router 192.168.2.1
dns-server
domain-name example-customer.com
lease 2
!
!
ip tcp path-mtu-discovery
no ip bootp server
ip domain name example-customer.com
ip name-server
ip name-server
ip sla 1
icmp-echo source-ip
request-data-size 500
threshold 2000
tag providers-rtr
frequency 45
ip sla schedule 1 start-time now
ip sla 2
icmp-echo source-ip
request-data-size 500
threshold 3000
tag providers-rtr
frequency 45
ip sla schedule 2 start-time now
!
!
!
!
!
!
track 1 rtr 1
!
track 2 rtr 2
!
!
!
!
!
interface FastEthernet0
description Link to Provider1
switchport access vlan 2
spanning-tree portfast
!
interface FastEthernet1
description Link to Provider2
switchport access vlan 3
spanning-tree portfast
!
interface FastEthernet2
!
interface FastEthernet3
!
interface FastEthernet4
description Local LAN
ip address 192.168.2.1 255.255.255.0
ip nat inside
ip virtual-reassembly
duplex auto
speed auto
!
interface Vlan1
no ip address
!
interface Vlan2
description Link to Provider1
ip address 255.255.255.248
ip nat outside
ip virtual-reassembly
!
interface Vlan3
description Link to Provider2
ip address 255.255.255.248
ip nat outside
ip virtual-reassembly
!
ip classless
ip route 0.0.0.0 0.0.0.0 track 1
ip route 0.0.0.0 0.0.0.0 10 track 2
!
no ip http server
no ip http secure-server
ip nat pool provider1 prefix-length 29
ip nat pool provider2 prefix-length 29
ip nat inside source route-map nat-provider1-out pool provider1 overload
ip nat inside source route-map nat-provider2-out pool provider2 overload
!
access-list 10 permit 192.168.0.0 0.0.0.255
access-list 11 permit
access-list 12 permit
access-list 20 permit 192.168.2.0 0.0.0.255

no cdp run
!
route-map nat-provider1-out permit 10
match ip next-hop 12
!
route-map nat-provider2-out permit 10
match ip next-hop 11
!
!
control-plane
!
banner motd ^C
welcome dude.
^C
!
line con 0
exec-timeout 30 0
no modem enable
line aux 0
line vty 0 4
access-class 160 in
exec-timeout 30 0
transport input telnet ssh
transport output telnet ssh
!
scheduler max-task-time 5000
end

I took out some parts of the config and changed it a bit from what it looks like on the router, but it should work without problems.

Exim + DomainKeys Howto.

2007-09-15T08:32:00.000-07:00

I ran into this problem a couple of months ago: all the mail one of my customers was sending ended up in bulk folder in Yahoo, Hotmail and other big free mail providers. While I was aware this will not solve the problem entirely, due to the fact that they open blindly any attachment in any email they receive (and thus get viruses which start sending emails/spam the next minute), I thought that I should also implement DomainKeys, since I was doing alot of maintenance work (like updating clamav, spamassasin and so on and so forth).

It took me a couple of days to finally get it working the right way due to different factors (dns, not being able to find a very comprehensive documentation, etc).

Here is what I came up with:

This is a quick tutorial on how to enable domainkeys (http://domainkeys.sourceforge.net/) on Exim on a FreeBSD server.

First, if you have not installed yet Exim, you have to install it. If you have installed it already, you have to recompile it.

The way to do it:

cd /usr/ports/mail/exim
ee Makefile    (or use joe / vi /whatever you like)

Search for the following lines:

# Enable DomainKeys support
#WITH_DOMAINKEYS=       yes

..and uncomment the "#WITH_DOMAINKEYS= yes".

Now you will have:

# Enable DomainKeys support
WITH_DOMAINKEYS=       yes

Save and exit the text editor.

Do the following:

make clean
make rmconfig
make
make FORCE_PKG_REGISTER=1 install  <-- if you already have exim installed. 
make install                       <-- if you don't have yet exim installed.

Copy & paste in your console:

cd /usr/local/etc/exim
mkdir dk
cd dk
openssl genrsa -out rsa.private 768
openssl rsa -in rsa.private -out rsa.public -pubout -outform PEM
cat rsa.public

After all this you will have a result which will look something like that:

Save whats between ---BEGIN PUBLIC KEY--- and ---END PUBLIC KEY--- for later use.

Edit with your favorite text editor /usr/local/etc/exim/configure

find the line which starts with "remote_smtp:" . This should be under the "begin transports" section of the file.

It looks like that:

remote_smtp:
  driver = smtp

Edit there and make it look like that:

remote_smtp:
  driver = smtp
  dk_selector = myselector    # you will need this later when you will alter your dns config
  dk_private_key = /usr/local/etc/exim/dk/rsa.private
  dk_canon = nofws

Save the file, exit and start/restart exim :

sh /usr/local/etc/rc.d/exim.sh restart

Go to /etc/namedb/

Find the file corresponding to your domain (look for it in named.conf and you will find the path to it).

Let's presume is /etc/namedb/pri/com/yourdomain.com. Edit this file, and just after/below the IN MX statement, add the following things:

_domainkey.yourdomain.com.       IN      TXT     "t=y; o=-"

myselector._domainkey.yourdomain.com.  IN      TXT     "k=rsa; t=y; p=MHwwDQYJKoZIhvcNAQEBBQADawAwaAJhAKJ2lzDLZ8XlVambQfMXn3LRGKOD5o6lMIgulclWjZwP56LRqdg5ZX15bhc/GsvW8xW/R5Sh1NnkJNyL/cqY1a+GzzL47t7EXzVc+nRLWT1kwTvFNGIoAUsFUq+J6+OprwIDAQAB"

Alter the serial (for example, if today is 28-aug-2007, make your serial look like 2007082800 or 2007082801, etc), save the file and reload named.

if your domain is something like customer.yourdomain.com, then the records will look like that:

_domainkey.customer.yourdomain.com.       IN      TXT     "t=y; o=-"

myselector._domainkey.customer.yourdomain.com.  IN      TXT     "k=rsa; t=y; p=MHwwDQYJKoZIhvcNAQEBBQADawAwaAJhAKJ2lzDLZ8XlVambQfMXn3LRGKOD5o6lMIgulclWjZwP56LRqdg5ZX15bhc/GsvW8xW/R5Sh1NnkJNyL/cqY1a+GzzL47t7EXzVc+nRLWT1kwTvFNGIoAUsFUq+J6+OprwIDAQAB"

Remember to also modify /etc/namedb/named.conf:

Add the following to your options { ... } section of named.conf

        check-names master ignore;

This will allow you to use _ (underscore).

You will have to edit and add that "check-names master ignore;" thing if you get the following error in your logs:

Aug 28 15:02:33 noc1 named[83277]: pri/com/yourdomain.com:15: myselector._domainkey.yourdomain.com: bad owner name (check-names)
Aug 28 15:02:33 noc1 named[83277]: zone yourdomain.com/IN: loading master file pri/com/yourdomain.com: bad owner name (check-names)

The long string after ....."k=rsa; t=y; p= is your public key which i said you should keep for later use.

To test send an e-mail to dk at dk.crynwr.com . You will receive about 5 messages back from different addresses with test results.

If any of them says test passed you should be ok. Send an e-mail to a yahoo.com e-mail address and check the headers.They should look like this:

From Dan Caescu Tue Aug 28 06:20:08 2007
Return-Path: 
Authentication-Results: mta233.mail.mud.yahoo.com  from=yourdomain.com; domainkeys=pass (ok)
Received: from x.x.x.x  (EHLO relay.yourdomain.com) (x.x.x.y)
  by mta233.mail.mud.yahoo.com with SMTP; Tue, 28 Aug 2007 08:16:56 -0700
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=myselector; d=yourdomain.com;

That should be all.

Good luck!

PS: I also posted this here: http://wiki.exim.org/DomainKeys You will also find there some other useful documentation.