Any routing/network/BGP experts on the list?

[matt-urrlRJtNKRMsHrnhXWJB8w at public.gmane.org (Matt Shields)]

On Thu, Oct 14, 2010 at 12:40 PM, Dan Ritter <dsr-mzpnVDyJpH4k7aNtvndDlA at public.gmane.org> wrote:

> On Thu, Oct 14, 2010 at 12:04:40PM -0400, Matt Shields wrote:
> > As the subject suggests, I'm looking for someone that's an
> > routing/networking/BGP experts.
> >
> > I have a question about BGP and AS Path Prepending.  I have 2
> datacenters,
> > each with their own ISP connection.  At both datacenters I have 2 routers
> > that are connected to the ISP with VRRP.  To provide redundancy
> (automatic
> > failover) to my main datacenter (everything is mirrored to my second
> one), I
> > have configured BGP on all my routers so that the second datacenter
> prepends
> > my AS 4x.  In theory, all traffic should go to my main datacenter because
> we
> > do not do any prepending there.
> >
> > To test this setup, we flipped the config so that we prepended 4x at the
> > main datacenter and removed prepending at the secondary.  After hours of
> > waiting traffic finally flipped over, but now when I change the
> prepending
> > back to the default way and have been waiting hours it hasn't flipped
> back.
> > Looking glass has shown the higher amount of prepends at the secondary
> > datacenter, but still prefers the secondary one.
> >
> > Last time I setup something like this was a few years ago and generally
> BGP
> > updates happened fairly quickly within minutes across the net and routing
> > was affected just as fast.  Anyone have any insight?
>
>
> Are they both connected to the same ISP?  Are you testing from
> multiple points? Are you advertising all your routes at both
> locations? How many peers? Do the peers apply filters?
> (Especially prefix-length filters?)
>
> -dsr-
>
> --
> http://tao.merseine.nu/~dsr/eula.html<http://tao.merseine.nu/%7Edsr/eula.html>is hereby incorporated by reference.
> You can't defend freedom by getting rid of it.
>


Two routers at one datacenter/ISP (two peers to same ISP), two routers at
second datacenter/ISP (two peers to same ISP).  x-connect between
datacenters and I peer DC1RTR1 with DC2RTR2 and DC1RTR2 with DC2RTR2.  No
filters.

What I found out from both ISP's is BGP has changed since the last time I
did this and some ISPs let you do prepending, some use what's called
"Community Preferences", some use both.  CP is a way of announcing to the
world how you want your announcements to be treated. The problem is just
because you configure your session to use prepending and your ISP
acknowledges it, doesn't mean upsteam providers from them will listen to
them.  If they use CP, they may choose to ignore your prepends and apply
their own rules since you didn't set any CP.  Luckily I finally found an
engineer at both ISPs that was able to give us the CP info we needed to make
our scenario work.

-matt