cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
3430
Views
0
Helpful
13
Replies

How do I load balance TFTP between two servers and a client on the same subnet?

pjwhitby_2
Level 1
Level 1

Hi,

I have trawled through several documents and tried umpteen different configs, all to no avail. I have a PXE boot client trying to access a boot file via TFTP from a couple of TFTP servers on the same VLAN/subnet. For HA purposes I want to load balance the two TFTP servers.

Config is currently;

=====

probe icmp ICMP_PROBE

  description icmp probe for default gateway tracking

  interval 5

  passdetect interval 15

rserver host server1

  description Server1

  ip address 10.0.0.1

  inservice

rserver host server2

  description Server 2

  ip address 10.0.0.2

  inservice

serverfarm host serverfarm_01

  description servers used

  probe ICMP_PROBE

  rserver server1

    inservice

  rserver server2

    inservice

class-map match-all L4_VIP_TFTP

  10 match virtual-address 10.0.0.10 udp eq 69

policy-map type loadbalance first-match L7_TFTP

  class class-default

    serverfarm serverfarm_01

policy-map multi-match L4_LB_VIP_POLICY

  class L4_VIP_TFTP

    loadbalance vip inservice

    loadbalance policy L7_TFTP

    loadbalance vip icmp-reply active

nat dynamic 1 vlan 200

interface vlan 200

  ip address 10.0.0.250 255.255.255.0

  nat-pool 1 10.0.0.241 10.0.0.243 netmask 255.255.255.255 pat

  service-policy input L4_LB_VIP_POLICY

  no shutdown

ip route 0.0.0.0 0.0.0.0 10.0.0.254

=====
I have read the doco by Ivan Kovacevic amongst many others but as my clients and servers are on the same subnet, the config doesnt work.
Can anybody point me in the right direction please. The devices are ACE 4710 running A3(2.3).
Thanks
13 Replies 13

pablo.nxh
Level 3
Level 3

Hello there,

Have you tried using the same IP (10.0.0.10) for NATing the response back from your TFTP server?

Regardless of the one-armed mode you still need the dummy (all zero) virtual to match the UDP random port sent from your servers.

If you haven't tried with that kind of NAT let me know and I'll drop you a config sample

Tnx

__ __

Pablo

Hi Pablo,

I knew it would something on the NAT as I can see the connections in 'show connection'. If you have a sample config for this example that would be awesome.

Thanks alot,

Hi,

Try simply changing the nat-pool or create a new one but using the same IP as the VIP. (The decision depends whether you have other protocols

being balanced besides TFTP),

  class L4_VIP_TFTP

    loadbalance vip inservice

    loadbalance policy L7_TFTP

    loadbalance vip icmp-reply active

    nat dynamic 2 vlan 200

interface vlan 200

  ip address 10.0.0.250 255.255.255.0

  nat-pool 1 10.0.0.241 10.0.0.243 netmask 255.255.255.255 pat

  nat-pool 2 10.0.0.10 10.0.0.10 netmask 255.255.255.255

  service-policy input L4_LB_VIP_POLICY

  no shutdown

HTH

__ __

Pablo

Hi pablo,

thanks for the config, but it didnt work, here is my current config,

=====

probe icmp ICMP_PROBE

  description icmp probe for default gateway tracking

  interval 5

  passdetect interval 15

rserver host server1

  description Server1

  ip address 10.0.0.1

  inservice

rserver host server2

  description Server 2

  ip address 10.0.0.2

  inservice

serverfarm host serverfarm_01

  description servers used

  probe ICMP_PROBE

  rserver server1

    inservice

  rserver server2

    inservice

class-map match-all L4_VIP_TFTP

  10 match virtual-address 10.0.0.10 udp eq 69

policy-map type loadbalance first-match L7_TFTP

  class class-default

    serverfarm serverfarm_01

policy-map multi-match L4_LB_VIP_POLICY

  class L4_VIP_TFTP

    loadbalance vip inservice

    loadbalance policy L7_TFTP

    loadbalance vip icmp-reply active

    nat dynamic 2 vlan 200

interface vlan 200

  ip address 10.0.0.250 255.255.255.0

  nat-pool 1 10.0.0.241 10.0.0.243 netmask 255.255.255.255 pat

  nat-pool 2 10.0.0.10 10.0.0.10 netmask 255.255.255.255 pat

  service-policy input L4_LB_VIP_POLICY

  no shutdown

ip route 0.0.0.0 0.0.0.0 10.0.0.254

=====
I have the following in the 'show conn  detail' output
conn-id    np dir proto vlan source                destination           state
----------+--+---+-----+----+---------------------+---------------------+------+
43455      1  in  UDP   212  10.0.0.200:52007   10.0.0.10:69       --
          [ idle time   : 00:00:02,   byte count  : 79         ]
          [ elapsed time: 00:00:02,   packet count: 1          ]
1847713    1  out UDP   212  10.0.0.2:69        10.0.0.10:52007    --
          [ conn in reuse pool : FALSE]
          [ idle time   : 00:00:02,   byte count  : 0          ]
          [ elapsed time: 00:00:02,   packet count: 0          ]
and then the connection times out after approx 3 minutes with no increase in the latter connections byte or packet count.
I am guesssing the packet or request contains the clients real IP address and the server is trying to talk directly to the client rather than the LB VIP?
Can anyone suggest the next step, I am running out of ideas?
Thanks

Hi,

Oh I thought the dummy virtual was already in place, please configure the highlighted portion.

class-map match-any TFTP    
   match virtual-address 0.0.0.0 0.0.0.0 udp eq any


policy-map multi-match L4_LB_VIP_POLICY

  class L4_VIP_TFTP

    loadbalance vip inservice

    loadbalance policy L7_TFTP

    loadbalance vip icmp-reply active

    nat dynamic 2 vlan 200

  class TFTP
   nat dynamic 2 vlan 20
0

* I don't see the ACL permitting traffic into VLAN 200, did you remove it from the example?

HTH
__ __
Pablo



Hi pablo,

thanks for sticking with this one its really appreciated, but it still isnt working. I did have the ip any any for access into the vlan, heres is the current config

Theres is now a second VIP 10.0.0.201 as I am running through this case with Cisco TAC and I wanted to keep both yourself on this thread and the Cisco TAC engineer active, if I get an answer from either method I will let everybody know.

and sorry, the VLAN is VLAN212 not VLAN200.

====

access-list ALL line 10 extended permit ip any any

probe icmp ICMP_PROBE

  interval 5

  passdetect interval 15

rserver host server1

  description Server1

  ip address 10.0.0.1

  inservice

rserver host server2

  description Server2

  ip address 10.0.0.2

  inservice

serverfarm host serverfarm_01

  description servers used

  probe ICMP_PROBE

  rserver server1

    inservice

  rserver server2

    inservice

class-map match-all L4_VIP_TFTP

  10 match virtual-address 10.0.0.10 udp eq 69

class-map match-all L4_VIP_TFTP2

  10 match virtual-address 10.0.0.201 any

class-map match-any TFTP

  10 match virtual-address 0.0.0.0 0.0.0.0 udp any

policy-map type loadbalance first-match L7_TFTP

  class class-default

    serverfarm serverfarm_01

policy-map type loadbalance first-match L7_TFTP2

  class class-default

    serverfarm serverfarm_01

policy-map multi-match L4_LB_VIP_POLICY

  class L4_VIP_TFTP

    loadbalance vip inservice

    loadbalance policy L7_TFTP

    loadbalance vip icmp-reply active

    nat dynamic 2 vlan 212

  class L4_VIP_TFTP2

    loadbalance vip inservice

    loadbalance policy L7_TFTP2

    loadbalance vip icmp-reply active

    nat dynamic 3 vlan 212

  class TFTP

    nat dynamic 2 vlan 212

interface vlan 212

  ip address 10.0.0.250 255.255.255.0

  access-group input ALL

  nat-pool 1 10.0.0.241 10.0.0.243 netmask 255.255.255.255 pat

  nat-pool 2 10.0.0.10 10.0.0.10 netmask 255.255.255.255

  nat-pool 3 10.0.0.201 10.0.0.201 netmask 255.255.255.255

  service-policy input L4_LB_VIP_POLICY

  no shutdown

ip route 0.0.0.0 0.0.0.0 10.0.0.254

====

Please feel free to ask any questions, I am on BNE time.

Thanks

Paul

Hi pablo,

Do you have any thoughts or configs I can try please.

If anybody else has any ideas, please feel free to jump in, just to remind everybody.

I have two TFTP servers that will be issuing a boot file (ardbp32.bin) to PXE booting clients. The two TFTP servers reside within the same VLAN as the PXE booting clients and in order to supply a degree of HA, I want to place the two TFTP servers (.1 and .2) behind a VIP on a ACE4710 (.10). When we distribute the TFTP server address via DHCP (option 66) I want to send the VIP address (.10) and allow the ACE4710 to load balance the TFTP conversation.

Thanks to all,

Paul

Try using the following configuration:

Note: Please make sure to configure also a udp probe to probe udp port 69, in case the application is down.

You need to configure a management policy on the interface when using a UDP probe.

That is because, when port 69 on the server will be unreachable, the server will send an ICMP unreachable.

ACE will consider a udp probe as "failed" only when it sees ICMP unreachable.

Without a management policy-map, the ICMP unreachable message will be dropped.

Also, add an ICMP probe to the rserver because udp probe will not be enough when the physical interface will be down.

That is because UDP is a connection-less protocol. To consider a UDP probe successfull, ACE need to see NO answer from the server in respose to the probe.

The ACE will not see any answer from the server when the interface is down and thus, will consider the probe as "sucessful".

With ICMP probe attached to the rserver, you also test the reachability of the server and not only the UDP port.

Here is the configuration (of course, you can chage the names of the of the objects to the name you are using if you want) :

access-list ALL line 10 extended permit ip any any

probe udp TFTP

  port 69

  interval 5

  passdetect interval 15

probe icmp ICMP_PROBE

  interval 5

  passdetect interval 15

rserver host TFTP_1

  ip address 10.0.0.1

  probe TFTP

  probe ICMP_PROBE

  inservice

rserver host TFTP_2

  ip address 10.0.0.2

  probe TFTP

  probe ICMP_PROBE

  inservice

serverfarm host TFTP-SFARM

  rserver TFTP_1

    inservice

  rserver TFTP_2

    inservice

sticky ip-netmask 255.255.255.255 address source TFTP-STICKY

  timeout 10

  replicate sticky

  serverfarm TFTP-SFARM

class-map type management match-any MANAGE

  2 match protocol icmp any

class-map match-all NAT

  2 match virtual-address 0.0.0.0 0.0.0.0 udp any

class-map match-all TFTP

  2 match virtual-address 10.0.0.10 udp eq 69

policy-map type management first-match MANAGE

  class MANAGE

    permit

policy-map type loadbalance first-match ROUTE

  class class-default

    forward

policy-map type loadbalance first-match TFTP-POL

  class class-default

    sticky-serverfarm TFTP-STICKY

policy-map multi-match TFTP-MULTI

  class TFTP

    loadbalance vip inservice

    loadbalance policy TFTP-POL

    nat dynamic 1 vlan 212

  class NAT

    loadbalance vip inservice

    loadbalance policy ROUTE

    nat dynamic 2 vlan 212

interface vlan 212

  ip address 10.0.0.250 255.255.255.0

  no normalization

  access-group input ALL

  nat-pool 1 10.0.0.241 10.0.0.243 netmask 255.255.255.0 pat

  nat-pool 2 10.0.0.10 10.0.0.10 netmask 255.255.255.0 pat

  service-policy input TFTP-MULTI

  service-policy input MANAGE

  no shutdown

Let me know how it goes.

Good luck!

I'm back from leave so I can now reply,

thanks for your response Ehud but that didnt work either.

I have another test config from TAC that I am going to try and I will then re-visit your solution and try again but I wanted to re-post this (bump it) to see if anybody out these has successfully used a Cisco ACE to load balance TFTP?

Thanks

Paul

sorry, I still don't have a solution but I wasn't able to work on the issue last week.

anybody any thoughts?

Hi pjwhitby,

I'm having the same problem here.. So, I get the solution?

hey Wallace,

sorry, we never got this running and we worked in a work around, although I cannot remember what it was, I think we went with a round robin DNS solution, but I would have to check.

Paul

All-

  You can not loadbalance TFTP one-armed (servers and clients on the same vlan).  Here is why-

  The way tftp works, the client sends a UDP packet with a random source port destine to the server on port 69.  The server responds with source port 69, however, instead of using the client port as a destination like most protocols, TFTP generates a random port number to send data traffic back to the client.

  ACE tracks connections via MAC addresses, IP's and port numbers, and ingress/egress interfaces on a connection.  When we send the packet out of the interface to the server, we are expecting a response back from to the clients port.  To ACE, the port randomization the server is doing looks like a brand new UDP connection coming from the server.

As relates to loadbalancing TFTP - The client is talking to the VIP IP and expects to recieve a response from that IP.  The server is talking to the client IP and is communicating with ACE.  ACE has to catch the server's traffic destine to the client and modify the source IP from the server back to the VIP.  When the server replies to the client connection with a new port, you can catch it with a class map that matches UDP source port 69 and source-nat the packet to the VIP IP.   That is what was being done above via this configuration:

class-map match-any TFTP

  10 match virtual-address 0.0.0.0 0.0.0.0 udp any

class NAT

    nat dynamic 2 vlan 212

However - with one-armed configurations, the gateway on the server is not ACE.  That means, the client sends a packet to the VIP, the ACE forwards it to the server IP, the server sends its response back to the client.  At that point, the client is confused because it was talking to the VIP IP, but recieved a response from the Server IP.  To mitigate that, you need to configure source NAT on the connection going from ACE to the server.  That exact part is what makes it impossible to loadbalance TFTP one-armed. 

Think about this:

When you configure source nat, many clients use one single IP.  Source ports are changed to track which client belongs to which connection on the backend. (since all clients are going to use the same single IP between the ace and the server, how else are you going to track the connection?)

So, when the server is going to reply to the NAT pool IP and the destination port is going to be random - How exactly would the ACE know what client in the NAT table is supposed to recieve that packet?  It can't. 

The only possible way to get this to work is configure policy based routing on the gateway to force any source port 69 traffic from the server back to the ACE so that ace can Un-Nat the traffic with a standard tftp loadbalancing configuration.  In that manner, you are replacing the function of source nat by routing.  You just need to ensure that the packet goes into the interface that ACE egressed the first udp packet to the server on.

I know that this is complicated to understand by reading, if you would like a set of wireshark traces to suppliment this, email me: chrhiggi@cisco.com.  I have both a standard TFTP connection as well as a single trace showing both sides of TFTP being loadbalanced.

Regards,

Chris Higgins 

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: