cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1887
Views
0
Helpful
2
Replies

TREX: Multiple Trex Instances Running on the same ports crashing

givohra
Cisco Employee
Cisco Employee

My goal is to send more than 983K client/server hosts as currently supported by Trex. I tried to create 2 trex instances which uses the same ports but a diff set of traffic yaml which has diff src/ip addresses range so that eventually on the same set of port we can send ~2M client & ~2M server ips but as soon as in instantiate the second trex instance, both instances start to crash as below. Any suggestions please ?

 

WATCHDOG: task 'Trex DP core 1' has not responded for more than 1.24747 seconds - timeout is 1 seconds

*** traceback follows ***

1 0x55de57431c1a ./_t-rex-64(+0x164c1a) [0x55de57431c1a]
2 0x7f36d40075e0 /lib64/libpthread.so.0(+0xf5e0) [0x7f36d40075e0]
3 0x55de5769ffea rte_delay_us_block + 106
4 0x55de57414cee CCoreEthIF::send_burst(CCorePerPort*, unsigned short, CVirtualIFPerSideStats*) + 158
5 0x55de57414d69 CCoreEthIF::flush_tx_queue() + 73
6 0x55de57401774 CNodeGenerator::handle_maintenance(CFlowGenListPerThread*) + 100
7 0x55de57402ecc CNodeGenerator::handle_flow_sync(CGenNode*, CFlowGenListPerThread*, bool&) + 92
8 0x55de574032f8 CNodeGenerator::handle_slow_messages(unsigned char, CGenNode*, CFlowGenListPerThread*, bool) + 184
9 0x55de573cabff int CNodeGenerator::flush_file_realtime<24, false>(double, double, CFlowGenListPerThread*, double&) + 1103
10 0x55de575bbd9b TrexDpCoreStf::start_stf() + 1083
11 0x55de575bc7c9 TrexDpCoreStf::start_scheduler() + 9
12 0x55de57573dc9 TrexDpCore::start() + 57
13 0x55de573f8f33 CFlowGenListPerThread::start(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, CPreviewMode&) + 115
14 0x55de5741c556 CGlobalTRex::run_in_core(unsigned char) + 326
15 0x55de57423eed ./_t-rex-64(+0x156eed) [0x55de57423eed]
16 0x55de576b64dd eal_thread_loop + 477
17 0x7f36d3fffe25 /lib64/libpthread.so.0(+0x7e25) [0x7f36d3fffe25]
18 0x7f36d307934d clone + 109


*** addr2line information follows ***

??:0
??:0
??:0
??:0
??:0
??:0
??:0
??:0
??:0
??:0
??:0
??:0
??:0
??:0
??:0
??:0
??:0
??:0


./t-rex-64: line 80: 30305 Aborted ./_$(basename $0) $INPUT_ARGS $EXTRA_INPUT_ARGS
[root@atqa-trex4 latest]#

2 Replies 2

givohra
Cisco Employee
Cisco Employee

I found the workaround to add more clients & servers hosts by adding multiple generator clients & servers but i am still curious to find the reason of the crash i saw before.

I'm also facing this problem, though I already have configured multiple client & server generator IP pool.