RFC 2544 Recommendation ?

[duahimanshu816]

Hello Experts,

I'm interested in RFC 2544 testing of any product . some vendors provide online throughput figures for their VMs and hardware and mention details like server specifications, memory, CPU and all.  

my question is what are  recommendations for conducting RFC 2544 tests for throughput. For instance, should there be a delay between each iteration? For example, when testing different frame sizes like 64, 128, 512, and 1528 bytes, should there be a pause between each frame size, such as waiting for a few seconds after the 64-byte test before proceeding to the next frame size? If such delays are recommended, what is the suggested duration for these pauses? How cisco does this test? what are the best practice to do the RFC2544 for network devices like Switch, Router, Firewall, hardware base and software base.  

[Ramblin Tech]

Test sets from Spirent or Ixia have RFC2544 testing integrated into their application suites, just configure the ports and hit the start button (Cisco does this). If you will not be using one of these test sets, but testing manually with some traffic generator, you can get good guidance on running RFC2544 tests from… RFC2544, particularly sections 23 & 24. 

[Ramblin Tech]

See RFC 2544 Sections 23 & 24 for recommended test duration and pauses between tests; I recommend reading the entire document.

23. Trial description

A particular test consists of multiple trials. Each trial returns
one piece of information, for example the loss rate at a particular
input frame rate. Each trial consists of a number of phases:

a) If the DUT is a router, send the routing update to the "input"
port and pause two seconds to be sure that the routing has settled.

b) Send the "learning frames" to the "output" port and wait 2
seconds to be sure that the learning has settled. Bridge learning
frames are frames with source addresses that are the same as the
destination addresses used by the test frames. Learning frames for
other protocols are used to prime the address resolution tables in
the DUT. The formats of the learning frame that should be used are
shown in the Test Frame Formats document.

c) Run the test trial.

d) Wait for two seconds for any residual frames to be received.

e) Wait for at least five seconds for the DUT to restabilize.

24. Trial duration

The aim of these tests is to determine the rate continuously
supportable by the DUT. The actual duration of the test trials must
be a compromise between this aim and the duration of the benchmarking
test suite. The duration of the test portion of each trial SHOULD be
at least 60 seconds. The tests that involve some form of "binary
search", for example the throughput test, to determine the exact
result MAY use a shorter trial duration to minimize the length of the
search procedure, but the final determination SHOULD be made with
full length trials.

[guruvignesh]

Hello duahimanshu816

For conducting RFC 2544 tests for throughput:

1. Test various frame sizes (start small, then increase).
2. A brief pause between iterations helps stabilize the network.
3. Test each frame size for around 60 seconds.
4. Use specialized testing tools or equipment.
5. Configure the device appropriately and analyze results accurately.
6. Comply with RFC 2544 guidelines for testing.

[duahimanshu816]

Hello Guruvignesh, 

It's common to wait a few seconds between different frame sizes (how much delay 10 sec, or 1 min), secondly what about within the same frame size? For example, if we're using a 128-frame size and initially sending a load of 50% for one gig, and if the test fails, we reduce the load by 50% and start sending traffic again. We continue this process, increasing and decreasing the load until we achieve 0% packet frame loss. The time difference between the first load and the second load is typically 5 to 10 seconds. Is this also a common practice?

[Joseph W. Doherty]

"How cisco does this test?"

Unclear how Cisco does their own performance tests, although they sometimes do mention RFC 2544, see attachment as a Cisco example.

Cisco also appears to use a 3rd party, e.g. Miercom, to validate the performance of various Cisco products.

The Miercom test results, allowed to be public by Cisco, often include a "How we did it" (the testing) section, which you might find of some interest, if you haven't already seen such reports.  For an ISR 4K example.