multiple vlan from multiple ports connecting to router through a single interface

[MATT ALLEN]

I recently received a Catalyst 3850-24t-s with IP Base license to configure to work with some network upgrades we are wanting to do. I work for a small ISP and we are a FTTH network using Trident7 chassis from Pace networks. the connections from the chassis are sent from a switch module that provides connection for the entire chassis through a single ethernet connection. To decrease our broadcast domain we are needing to break the connection to the chassis into several VLAN, one per card in the chassis. in doing this I need to send 18 vlan per chassis to the Catalyst switch which should be no problem. the problem i am running into is connecting all the interfaces that feed the chassis to a single interface that connects to the router and also be able to use a single DHCP server. i have made a simple network diagram to help better understand what I am talking about. Any help would be greatly appreciated. thank you in advance.

Matt 

[John Blakley]

Matt,

I'm not familiar with the Trident 7 chassis. Are hosts going to be connected here? Where is your L3 interface being housed? Is that on the 3850?

If the L3 svi is on the 3850, do you have a dedicated svi for every vlan that you've created? The L2 stuff is easy, but the routing piece may be a little tricky depending on what's routing. If you could use one vlan to demonstrate how a host should be getting an address, that may help... (like booting a host on vlan 301, default gateway would be this, the default gateway is on the switch/router, etc.)

HTH,

John

[MATT ALLEN]

The Trident 7 Chassis is basically a large layer 2 fiber switch. it has cards that have PONs on them that the subscriber connects to through fiber, and in the center is a switch module that connects to the switch. What we are working on is making each card a separate VLAN to decrease the size of the broadcast domain. 

I do not have any dedicated SVI set up for each VLAN yet but do have all the VLAN in the router and configured to their appropriate port. the connections to the switch for testing purposes will be 

1/0/2 is the connection to the router 192.168.2.254

1/0/3 is the connection to the chassis VLAN 301-318

1/0/5 is the connection to the chassis VLAN 401-408

1/0/7 is the connection to the chassis VLAN 501-518

1/0/9 is the connection to the chassis VLAN 601-607

1/0/12 is the connection to the DHCP server for all customers

This is my first run in with a layer 3 switch and still have some learning to do.

[MATT ALLEN]

I have been doing some more testing today and I have assigned vlan 302 a SVI and have turned the connection to the router 1/0/2 into a layer 3 port

interface vlan 302 192.168.1.1 255.255.255.0

interface 1/0/2 no switchport

interface 1/0/2 ip address 192.168.2.250 255.255.255.0

the IP address of the router that is plugged into 1/0/2 is 192.168.2.254

I have not done anything with the DHCP server yet so I am using a static IP in the laptop I am using trying to ping the gateway but, still no results or throughput.  Are there any configurations that need to be done on the router that connects to 1/0/2?

[paul driver]

Hello

It seems from your posts that a possible solution would be using the L3 switch:

Basic setup would be:
 

1) on the L3 switch assgin L3 SVI interfaces ( switch virtual interfaces) for all you vlans and create the L2 vlan also

int vlan xx
ip address x.x.x.x y.y.y.y

vlan xxx
exit

2) enable Ip routing on the switch

conf t
ip routing



3) For the router connection  assign a specific vlan on a access port or a L3 interface to connect to the router


int x/x
descriptioLink to router
switchport host
switchport acces vlan xx

or

int x/x
descriptioLink to router
no switchport
Ip address x.x.x.x.
 

4) For the dhcp server assign a specific vlan on a access port
 

int x/x
descriptioLink to dhcp server
switchport host
switchport acces vlan xx

5) provide a default route toward your router

ip route 0.0.0.0 0.0.0.0 x.x.x.x ( this is the router Lan facing ip address)

res

Paul



 

[Richard Burts]

I am not clear about the original environment and would appreciate some clarification. If the Trident is essentially a large layer 2 switch and users connect to it, then what are they communicating with? Is it logical to assume that there is some device that they communicate with? Is that communication layer 3 oriented or is it layer 2 oriented?

If the central device that is communicating with users is layer 2 oriented then the device and the user connection communicate directly (the central device has learned the MAC address of the user and can send directly to that MAC). That will no longer be the case when you create separate VLANs. If the communication from central device to user connection is layer 3 oriented then separate VLANs and routing between VLANs is feasible (though it seems it will complicate the data flow since a packet from the user to the central device will now go from Trident user VLAN to the 3850 to be routed and then sent back to the Trident to be delivered to the central device). So the traffic flows over the link from Trident to 3850 will traverse that link multiple times for each packet.

So can you provide some clarification about the nature of the communication within the Trident?

HTH

Rick

[MATT ALLEN]

Richard, we are an ISP and do not have any users that communicate with each other. the only communication that is going on is allowing Subscribers with internet access. the trident is a layer 2 device that communicates with an optical network terminal (ONT) at the customers residence. the configuration we are going for is each card in the trident will be on a separate VLAN, meaning all the ONT connected to that card will communicate over the VLAN configured for that card. I have already done some work with a layer 2 switch and proved that data will flow from ONT to trident to layer 2 switch to router but that was only one VLAN at a time. in order to have multiple VLAN on the same interface going to the router we needed a layer 3 switch to route the traffic. Hope this clears things up for ya.

Matt 

[Richard Burts]

Matt

I appreciate your explanation, though I still am a bit confused. My fundamental question still amounts to this: if you have a bunch of things (users, or subscribers, or ONTs, or whatever) that have been communicating with something at layer 2 in one big broadcast domain, and then you divide that single broadcast domain into multiple VLANs will all those end points still communicate with the thing that need to communicate with now that the communication needs to be based on layer 3 instead of layer 2?

If you have tested this and it works, then that is good news and I do not necessarily need to understand how it works. It just seemed to be a point of possible concern from my perspective.

HTH

Rick

[Jon Marshall]

Matt

It may be a terminology thing but I would be asking the same as Rick.

To put it another way.

When you tested with the gateway on the router and it worked how did you test it ie. did you have an end client with a 192.168.2.x IP using a default gateway IP of 192.168.2.254 ?

If so when you tested using vlan 302 the client has to have an IP from the vlan 302 subnet and the gateway has to be the vlan 302 SVI IP address on your L3 switch. 

Just in case it's not clear an SVI is the L3 vlan interface on your switch ie. "interface vlan <x>".

Did you try this ?

Edit - note also that when you use the 3850 to do the routing for the vlans your router now needs to know how to get to those IP subnets ie. your L3 switch has a default route to the router IP of the L3 link and for each vlan/IP subnet on the L3 switch your router needs a route pointing to the L3 switch end.

For the router to learn of the IP subnets on the 3850 you can either use a routing protocol between the L3 switch and router or use static routes, up to you.

For DHCP you need to add this to all the SVIs on the 3850 -

"ip helper-address <DHCP server IP>"

except the SVI for the vlan the DHCP server is in.

Finally you may want to use acls on the SVIs to stop end customers from sending traffic to each other.

All of the above assumes that end clients are using IPs from the vlan IP subnets.

If they aren't then I am in the same position as Rick ie. confused :-)

Jon

[MATT ALLEN]

I see what you are saying about the default gateway and i see my error there. I need to use the SVI IP of VLAN 302 as the default gateway for devices attached to VLAN 302. I will Fix that and test it but, I can see where this will cause me some problems with issuing IP information to the customer. I will have around 60 VLANs and I have 15 class C IP. to issue different default gateway to customers in different VLANs could turn into a mess. 

[Jon Marshall]

If you are using DHCP you just assign the default gateway that way.

And if you are using class C private IP addressing there should be no limit on how many subnets you have ie. you can have one per vlan unless you are saying you are using those subnets already ?

Jon

[MATT ALLEN]

we are using the IP addresses already as one big pool for our customers. we were hoping to be able to move the customers to the VLANs a little at a time because it would be a difficult task to re-provision thousands of customer devices in a single maintenance window. From what I am understanding it may not be possible to do a little at a time since all the devices now are in one big untagged broadcast domain. to have both the untagged and the routed vlans going to the same interface may be a problem. especially when we start breaking the DHCP pool into subnets for the vlans. If I am understanding you correctly. 

[Jon Marshall]

The main problem you have is that currently all customers use the same default gateway and that IP address resolves to a mac address ie. the routers physical interface.

If you move the IP to an SVI on your L3 switch then all the customers will have the wrong mac address in their arp tables which would break connectivity.

There may be a way around it especially if the current IP subnet is not going to be reused with smaller subnet masks.

It really depends on how you see the migration happening and how your current setup works ie. do all customers use DHCP currently to get an IP address ?

Jon

[MATT ALLEN]

I am still having problems connecting from end user device to router.

I have changed my configuration to this

VLAN 302 ip address 192.168.2.1 255.255.255.248

interface 1/0/3 acces port vlan 302 (connected to Trident 7)

interface 1/0/2 no switchport IP 192.168.2.253 255.255.255.248  (connected to router)

default gateway 192.168.2.254 (IP address of router port)

router I am using for testing is a cisco 3725

the one that will be used when the switch is put into service is a ASR 1001-x but can't test with that one because it is in production. 

I don't have a DHCP server connected yet so i am using a static IP 

the IP I have assigned the end user is 192.168.2.2 mask 255.255.255.248 gateway 192.168.2.1 (IP of the VLAN interface)

I am still unable to ping the router from the end user. 

Any thoughts Rick, Jon 

[Richard Burts]

Matt

What you have described creates one subnet for vlan 302 connecting to the Trident 7 and a separate subnet for connecting to the router. This can work if the default gateway of devices connected through Trident 7 have their default gateway set to the 192.168.2.1 address of the switch and if the switch has ip routing enabled. Can you verify that the devices connected through Trident 7 have 192.168.2.1 as their default gateway? And can you verify that ip routing is enabled on the switch?

As a side note the addressing and subnet mask that you are using for vlan 302 will allow 5 devices in the subnet (in addition to the address used by the switch). Is this consistent with how vlan 302 is being used on Trident 7?

HTH

Rick