Web browsing performance problems

[Bob Tinkelman]

This is a follow-up message on the problem described in the
earlier posting (quoted below).


SUMMARY
-------

We believe that the problem has been solved and that web browsing
performance should now have returned to normal.

If you experience additional problems in this area, please report
them to us as soon as possible.



DETAILS
-------

ISPnet has, in addition to its customer connections, connections
to over 100 other Internet networks.  Some of these are our
upstream links, but most are "peers" with whom we have direct
connections, bypassing all upstreams.

One of these peers is Edgecast, a large Content Distribution
Network that, among other things, hosts web data for a large
number of companies.

Yesterday, we localized the performance problems to a strange
interaction between ISPnet and Edgecast at the NYIIX peering
switch in New York.

With help from Edgecast technical support, we were able to find
the problem and fix it.

There was an inconsistency in the way that Edgecast was bgp-
announcing some of their networks, in particular, ones in their
"any-cast group".  This inconsistency did not matter until after
a router configuration change made on the ISPnet routers during
this week's Tue-Wed overnight maintenance window.

Both ISPnet and Edgecast have made changes to avoid reoccurance
of this problem.  ISPnet has updated our router configurations
so as to be tolerant of this type of inconsistency from any peer. 
Edgecast has started bgp-announcing to ISPnet the anycast nets
which they had failed to announce previously.  Either of these
changes, alone, would have fixed the problem.  Both were done
as part of a "belt-and-suspenders approach".




ORIGINAL POSTING
----------------

> ISPnet received reports from multiple customers yesterday about
> problems accessing certain web sites.  For some sites, pages came
> up very slowly, and for other sites, not at all.

> We identified a common aspect to the problem pages, namely that
> they had content hosted by Edgecast, a large Content Delivery
> Network.

> ISPnet has a direct peering connection to Edgecast in New York
> which, as part of our troubleshooting, we shut down late last
> night, forcing traffice between ISPnet and Edgecast to take a
> slightly longer route, but without noticable improvement.

> We also opened a ticket with Edgecast, last night, requesting
> help.  They didn't have high level staff available to persue the
> problem until very recently (07:30 AM, Pacific).

> We are working on this problem and will send additional messages
> as we get more information.

> --
> Bob Tinkelman          <bob at tink.com>
> ISPnet, Inc.    http://www.ispnet.net
> +1 (718) 464-4747  office
> +1 (800) 806-NETS  toll free