cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
2888
Views
5
Helpful
7
Replies

How does the Internet Backbone Network Topology and Internet Service Provider Network Topology looks like?

effing_society
Level 1
Level 1

Hi all networking pros,

 

As you can see in the title, I was actually researching on the network topology of the internet backbone and also the internet service providers, but unfortunately I really couldn't find more information despite whole day of googling.

 

Now, I know that there is Internet Exchange Provider(IXP) that provides peering for all the Service Providers(SP), while the SP puts their router in the IXP hub to do peering with others SP, so it's like:

SP1 Border Router ---> IXP Switch (L2 or L3?) <--- SP2 Border Router

               l_______________(BGP)_________________l

 

So, does the design above reflects the real design of the Internet? Just a lot of routers being connected(peering) in the IXP hub? While the IXP has another link to another IXP in local and foreign regions?

 

Secondly the network topology of an ISP, I really couldn't find much information on the design of it as well, but I did designed one using the hierarchical design of CISCO as:

Border Router(BGP) <--- Core Router(High speed trunk connections to border routers) <--- Distribution Switch(Upstream to Core Router) <--- Access Router(Connected to a lot of Servers)

 

I wish that someone here can at least tell me am I right or wrong, if wrong how does the real life internet infrastructure being designed, craving for some real networking knowledges :D.

 

Thanks in advanced!

 

 

 

2 Accepted Solutions

Accepted Solutions

Joseph W. Doherty
Hall of Fame
Hall of Fame

". . . I was actually researching on the network topology of the internet backbone and also the internet service providers, but unfortunately I really couldn't find more information despite whole day of googling."

As an aside, that might be because there's not an "Internet backbone", per se.  Understand the Internet is a network of networks.

Internet service providers, as further described by Giuseppe, "peer" their networks with other Internet service providers, either privately or at "public" peering points.

As the most basic level, all Internet service providers share, in some manner, address blocks with other providers, generally permitting one Internet connected host being able to contact any other Internet connected host.

Besides the complexity of any one Internet service provider's physical network, you also have the complexity of how traffic is routed, or treated, between Internet service providers, for "financial" reasons.

Ignoring the latter, can not always be done because sometimes the physical topology is because of "financial" reasons, dealing with other Internet service providers.

That aside, often, I believe, many Internet service providers would consider how their network is designed "proprietary", which would be another reason why you cannot find much about it via googling.

View solution in original post

Hello @effing_society ,

I think you have found a nice presentation.

About your proposed topology is fine but the type of devices used should be high end routers not  C2900 series.

 

The backbone infrastructure of an ISP is organised in POPs .

How these POPs are interconnected can be considered confidential information as noted by @Joseph W. Doherty .

Most ISPs implement two indipendent forwarding planes so that if any fault happens on plane 1 end to end traffic between two POPs can be sent over the other plane plane 2.

However, there are cases where this is not used or ring topologies are used at some level ( near the access or customers) in parts of the network.

 

Hope to help

Giuseppe

 

View solution in original post

7 Replies 7

Giuseppe Larosa
Hall of Fame
Hall of Fame

Hello @effing_society ,

Internet Exchange Points play a role for providing places where peering with multiple other ISPs.

However, each ISP can have private peering links with other selected ISPs in addition to the presence in IXPs.

 

>> So, does the design above reflects the real design of the Internet? Just a lot of routers being connected(peering) in the IXP hub? While the IXP has another link to another IXP in local and foreign regions?

 

It is not so simple each provider can be present in different IXPs as far as I know IXPs do not provide transit to to other IXPs. The reason is scalability and redundancy an ISP cannot rely simply on its presence on one or more IXPs to build the internet connectivity.

 

So as I have written above in addition to be present in several IXPs an ISP can have multiple private peerings with other ISPs with different routing strategies.

 

About how the backbone of an ISP looks like the building blocks are called POP = Point of Presence.

A point of presence is formed by different routers and some multilayer switches.

Consiidering  the use of MPLS as typical in a modern ISP the POP have two  P routers connecting to the inner core with different links.

Within the POP there are edge devices called PE nodes where the customer connections are terminated.

Multiple services can be offered by an ISP so we can see edge devices like BRAS   for broadband aggregation that terminates forms of broadband access like ADSL, ADSL2+, VDSL, VDSL2.

Then there are broadband access based on metro ethernet services look for MEF. The concept is similar.

Each PE node connects to the two P nodes of the POP.

There can be small POPs , medium POPs and POPS of inner core.

Most providers use a design approach that allow them to have two separate forwarding planes so that end to  end communication between two POPs can happen either in one plane or the other.

In this way in case of issues it is possible to move the traffic on the other forwarding plane.

An ISP uses OSPF or IS-IS and it can use multiple domains of each protocols for scalability.

These IGP are used to publish the loopback addresses of each device that acts as PE or P node.

These loopback addresses are used as OSPF Router IDs. LDP router ids, RSVP TE RIDs and BGP RIDs.

All services are advertised over MP BGP multi protocol BGP using multiple address families for different services.

MPLS provides a framework for the forwarding plane that allows to go over the routing hop by hop paradigm and only edge devices of a service are aware of what they are sending and receiving all the internal nodes called P nodes simply perform the swap change of the external MPLS label that is going in the direction of the remote PE loopback address.

BGP peering for internet connectivity can be in the global routing table or in a dedicated L3 VPN.

MP BGP achieves scalability by using route reflector servers and / or BGP confederations.

 

Hope to help

Giuseppe

 

Hi Giuseppe, Effing here.

 

Thanks for your reply and I really appreciated it, sorry for replying in such a late time because I have been working on some research and work as well at the mean while.

 

Please take a look at the picture that I have attached below as for the network design for an ISP PoP based on my current understanding.

 

ISP Network Design

Core Network <--- Aggregation/Distribution Network <--- Edge Network 

 

I interpret them as below:

Core: Connects a lot of Aggregation/Distribution Network together and route them to each other to form "A really Tiny Portion of the Internet"

Aggregation/Distribution: Connects a lot of Edge Network such as the services that ISP offers, Customer Edge and links back to the Core Network to form "The Lower Layer of A really Tiny Portion of the Internet"

Edge: Formed up by a lot of users and servers, for example, web servers, file servers, enterprise routers and etc, the "Real Content of the Internet"

 

I wish to have more opinions for the network topology that I have created and perhaps more clarification on what are the devices should be in each of the networks that I have classified out.

Hello @effing_society ,

ISP POPs are deployed at different levels of hierarchy

 

small POPs connecting to regional POPs connecting to inner core POPs.

 

the ISP edge devices are only the PE or customer facing routers , enteprise routers are considered customer equipment CE nodes.

The same happens for web servers of any type of hosts unless the ISP is providing hosting services they are not part of the ISP network.

 

POPs are interconnected via multiple links for load balancing. scalability and redundancy reasons. So a single link between POPs is not correct.

 

About devices to be used on different POPs they can be:

ASR 1000 or ME switches or ASR 900 series for low end PE nodes.

ASR 9000 for medium POPS and for BRAS based on IOS XR.

in the inner core Cisco provides CRS-3 or better IOS XR carrier class routers.

 

https://www.cisco.com/c/en/us/products/routers/index.html#~products

 

service provider section

Hope to help

Giuseppe

 

 

 

effing_society
Level 1
Level 1

hi @Giuseppe Larosa ,

 

I have thought a lot in the afternoon, and based on the reference link below, I have created a small ISP PoP network(without data center) as the picture in attachment, feel free to let me know what do you think! If you think that there is some mandatory modules that needed to be there, do let me know as well!

 

Thank you! It's been fun learning from you.

 

Reference:

https://www.itu.int/en/ITU-D/Regional-Presence/AsiaPacific/SiteAssets/Pages/Events/2019/ITU-ASP-CoE-Training-on-/Session11%2612_ServiceProvider_Network_Design.pdf

 

Effing

Hello @effing_society ,

I think you have found a nice presentation.

About your proposed topology is fine but the type of devices used should be high end routers not  C2900 series.

 

The backbone infrastructure of an ISP is organised in POPs .

How these POPs are interconnected can be considered confidential information as noted by @Joseph W. Doherty .

Most ISPs implement two indipendent forwarding planes so that if any fault happens on plane 1 end to end traffic between two POPs can be sent over the other plane plane 2.

However, there are cases where this is not used or ring topologies are used at some level ( near the access or customers) in parts of the network.

 

Hope to help

Giuseppe

 

Joseph W. Doherty
Hall of Fame
Hall of Fame

". . . I was actually researching on the network topology of the internet backbone and also the internet service providers, but unfortunately I really couldn't find more information despite whole day of googling."

As an aside, that might be because there's not an "Internet backbone", per se.  Understand the Internet is a network of networks.

Internet service providers, as further described by Giuseppe, "peer" their networks with other Internet service providers, either privately or at "public" peering points.

As the most basic level, all Internet service providers share, in some manner, address blocks with other providers, generally permitting one Internet connected host being able to contact any other Internet connected host.

Besides the complexity of any one Internet service provider's physical network, you also have the complexity of how traffic is routed, or treated, between Internet service providers, for "financial" reasons.

Ignoring the latter, can not always be done because sometimes the physical topology is because of "financial" reasons, dealing with other Internet service providers.

That aside, often, I believe, many Internet service providers would consider how their network is designed "proprietary", which would be another reason why you cannot find much about it via googling.

effing_society
Level 1
Level 1

@Joseph W. Doherty @Giuseppe Larosa 

 

I kinda have the idea now on the things, thank you 2 really much, you guys have been really really helpful!

 

Stay safe and healthy!

 

Kind regards,

Effing

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: