• State Resolved
  • Locked Locked
  • Replies 8 replies
  • Subscribers 100 subscribers
  • Views 1045 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Jumbo frames support on TI Integra

Henk Jan Ober
Prodigy 80 points

Hi,

I did some testing today with the ethernet running at 1000/full and using MTU size larger than the default 1500. With my PC setup using an Intel ethernet card (e1000e driver) on Linux I can set the MTU size to 9000 (anything higher will give an error because it is not supported) and I can reduce the system load and improve throughput of my GigEVision application which is basically the camera sending a constant stream of UDP packets.

With the TI Integra I seem to be able to set any MTU size using 'ifconfig eth0 mtu 10000' gives no indication of errors and ifconfig eth0 shows the MTU size happily set to 10000.

When I set the MTU it to 9000 and start my streaming application the EVM hangs after it starts to receive data (before I can type ifconfig to see packet statistics) and if I turn it off and on again it gives an endless list of crash reports at startup before the kernel is fully initialized (the camera is still sending jumbo frames at this point).

My Question: What is the maximum MTU size supported on TI Integra ? Is it 9000 like the Intel ethernet cards or just the 1500 default value for 100mbit ethernet?

I have attached the crash report below:

Pid: 5, comm:             events/0                                                                           
CPU: 0    Not tainted  (2.6.34 #1)                                                                           
PC is at emac_adjust_link+0xa8/0xac                                                                          
LR is at release_console_sem+0x198/0x1ac                                                                     
pc : [<c01ee01c>]    lr : [<c00537dc>]    psr: 60000013                                                      
sp : c7033f48  ip : c7033e68  fp : c7033f5c                                                                  
r10: 00000000  r9 : 00000000  r8 : c7048128                                                                  
r7 : 00000020  r6 : c7048000  r5 : 60000013  r4 : c70ecae0                                                   
r3 : 60000093  r2 : c03d3e00  r1 : 00003590  r0 : 00000001                                                   
Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment kernel                                          
Control: 10c5387d  Table: 87318019  DAC: 00000017                                                            
[<c0032edc>] (show_regs+0x0/0x50) from [<c004fe1c>] (__schedule_bug+0x4c/0x60)                               
 r5:00000002 r4:c7033f00                                                                                     
[<c004fdd0>] (__schedule_bug+0x0/0x60) from [<c02d9528>] (schedule+0x4c/0x2b8)                               
 r5:00000002 r4:ffff8f34                                                                                     
[<c02d94dc>] (schedule+0x0/0x2b8) from [<c02d9a58>] (schedule_timeout+0x158/0x190)                           
[<c02d9900>] (schedule_timeout+0x0/0x190) from [<c02d9ab8>] (schedule_timeout_uninterruptible+0x28/0x2c)     
 r6:00000001 r5:c721c140 r4:c70ecaf4                                                                         
[<c02d9a90>] (schedule_timeout_uninterruptible+0x0/0x2c) from [<c005d1e0>] (msleep+0x1c/0x28)                
[<c005d1c4>] (msleep+0x0/0x28) from [<c01eec54>] (emac_poll+0x55c/0x660)                                     
[<c01ee6f8>] (emac_poll+0x0/0x660) from [<c025dc4c>] (net_rx_action+0x58/0x154)                              
[<c025dbf4>] (net_rx_action+0x0/0x154) from [<c0057fdc>] (__do_softirq+0x7c/0x100)                           
[<c0057f60>] (__do_softirq+0x0/0x100) from [<c00580a8>] (irq_exit+0x48/0x50)                                 
[<c0058060>] (irq_exit+0x0/0x50) from [<c0031078>] (asm_do_IRQ+0x78/0x94)                                    
[<c0031000>] (asm_do_IRQ+0x0/0x94) from [<c02dae34>] (__irq_svc+0x34/0xa0)                                   
Exception stack(0xc7033f00 to 0xc7033f48)                                                                    
3f00: 00000001 00003590 c03d3e00 60000093 c70ecae0 60000013 c7048000 00000020                                
3f20: c7048128 00000000 00000000 c7033f5c c7033e68 c7033f48 c00537dc c01ee01c                                
3f40: 60000013 ffffffff                                                                                      
 r5:fa200000 r4:ffffffff                                                                                     
[<c01edf74>] (emac_adjust_link+0x0/0xac) from [<c01eae98>] (phy_state_machine+0x450/0x4b0)                   
 r5:00000000 r4:c70480fc                                                                                     
[<c01eaa48>] (phy_state_machine+0x0/0x4b0) from [<c0063a10>] (worker_thread+0x100/0x178)                     
 r9:00000000 r8:c7000628 r7:c7032000 r6:c7000620 r5:c7024c00                                                 
r4:c01eaa48                                                                                                  
[<c0063910>] (worker_thread+0x0/0x178) from [<c0066ce0>] (kthread+0x88/0x90)                                 
 r8:00000000 r7:c0063910 r6:c7000620 r5:c7027f20 r4:c7033fcc                                                 
[<c0066c58>] (kthread+0x0/0x90) from [<c0055fec>] (do_exit+0x0/0x5bc)                                        
 r7:00000000 r6:00000000 r5:00000000 r4:00000000                                                             
BUG: scheduling while atomic: events/0/5/0x00000100                                                          
Modules linked in: ipv6                                                                                      
                                                                                                             
Pid: 5, comm:             events/0                                                                           
CPU: 0    Not tainted  (2.6.34 #1)                                                                           
PC is at emac_adjust_link+0xa8/0xac                                                                          
LR is at release_console_sem+0x198/0x1ac                                                                     
pc : [<c01ee01c>]    lr : [<c00537dc>]    psr: 60000013                                                      
sp : c7033f48  ip : c7033e68  fp : c7033f5c                                                                  
r10: 00000000  r9 : 00000000  r8 : c7048128                                                                  
r7 : 00000020  r6 : c7048000  r5 : 60000013  r4 : c70ecae0                                                   
r3 : 60000093  r2 : c03d3e00  r1 : 00003590  r0 : 00000001                                                   
Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment kernel                                          
Control: 10c5387d  Table: 87318019  DAC: 00000017                                                            
[<c0032edc>] (show_regs+0x0/0x50) from [<c004fe1c>] (__schedule_bug+0x4c/0x60)                               
 r5:00000002 r4:c7033f00                                                                                     
[<c004fdd0>] (__schedule_bug+0x0/0x60) from [<c02d9528>] (schedule+0x4c/0x2b8)                               
 r5:00000002 r4:ffff8f4a                                                                                     
[<c02d94dc>] (schedule+0x0/0x2b8) from [<c02d9a58>] (schedule_timeout+0x158/0x190)                           
[<c02d9900>] (schedule_timeout+0x0/0x190) from [<c02d9ab8>] (schedule_timeout_uninterruptible+0x28/0x2c)     
 r6:00000001 r5:c721c140 r4:c70ecaf4                                                                         
[<c02d9a90>] (schedule_timeout_uninterruptible+0x0/0x2c) from [<c005d1e0>] (msleep+0x1c/0x28)                
[<c005d1c4>] (msleep+0x0/0x28) from [<c01eec54>] (emac_poll+0x55c/0x660)                                     
[<c01ee6f8>] (emac_poll+0x0/0x660) from [<c025dc4c>] (net_rx_action+0x58/0x154)                              
[<c025dbf4>] (net_rx_action+0x0/0x154) from [<c0057fdc>] (__do_softirq+0x7c/0x100)                           
[<c0057f60>] (__do_softirq+0x0/0x100) from [<c00580a8>] (irq_exit+0x48/0x50)                                 
[<c0058060>] (irq_exit+0x0/0x50) from [<c0031078>] (asm_do_IRQ+0x78/0x94)                                    
[<c0031000>] (asm_do_IRQ+0x0/0x94) from [<c02dae34>] (__irq_svc+0x34/0xa0)                                   
Exception stack(0xc7033f00 to 0xc7033f48)                                                                    
3f00: 00000001 00003590 c03d3e00 60000093 c70ecae0 60000013 c7048000 00000020                                
3f20: c7048128 00000000 00000000 c7033f5c c7033e68 c7033f48 c00537dc c01ee01c                                
3f40: 60000013 ffffffff                                                                                      
 r5:fa200000 r4:ffffffff                                                                                     
[<c01edf74>] (emac_adjust_link+0x0/0xac) from [<c01eae98>] (phy_state_machine+0x450/0x4b0)                   
 r5:00000000 r4:c70480fc                                                                                     
[<c01eaa48>] (phy_state_machine+0x0/0x4b0) from [<c0063a10>] (worker_thread+0x100/0x178)                     
 r9:00000000 r8:c7000628 r7:c7032000 r6:c7000620 r5:c7024c00                                                 
r4:c01eaa48                                                                                                  
[<c0063910>] (worker_thread+0x0/0x178) from [<c0066ce0>] (kthread+0x88/0x90)                                 
 r8:00000000 r7:c0063910 r6:c7000620 r5:c7027f20 r4:c7033fcc                                                 
[<c0066c58>] (kthread+0x0/0x90) from [<c0055fec>] (do_exit+0x0/0x5bc)                                        
 r7:00000000 r6:00000000 r5:00000000 r4:00000000                                                             
BUG: scheduling while atomic: events/0/5/0x00000100                                                          
Modules linked in: ipv6                                                                                      

  • 0
    TI__Mastermind 25055 points

    Hi,

    The EMAC can send and receive frames up to 2**16 - 1. However, the RX_MAXLEN register needs to be increased to the max frame size in order to allow jumbo frames.

    Regards,
    Marc

  • 0 in reply to MarcPyne
    Prodigy 80 points

    Hi Marc,

    I have compiled a custom kernel with the EMAC_DEF_MAX_FRAME_SIZE set to (9000 + 14 + 4 + 4) instead of (1500 + 14 + 4 + 4) in davinci_emac.c and I can now stream jumbo frames through GigEVision just fine. This setup will work fine for our application, thank you for the information!

    Two related follow up question though.

    1) With most network cards you can use the 'ifconfig eth0 mtu <x>' interface to change the maximum frame size dynamically. Does TI have any plans to change the hardcoded frame size to be controlled dynamically by the ifconfig interface ?

    2) With the frame size set to 1500 when the Ethernet controller receives frames larger than 1500 instead of dropping them with a frame error the kernel crashes (see my original report). Is this a known bug? It would be bad if someone can crash the device by just sending jumbo frames to it with the default kernel.

     

     

  • 0 in reply to Henk Jan Ober
    TI__Expert 8250 points

    Hi

    On some earlier platforms, there are restrictions on the Max.Packet size(the same driver is re-used in this case).  Also, in supporting higher frame sizes, we have seen that we often run in to memory allocation failures at run time(though this can be worked around by pre-allocating memory).

     

    I definitely do agree that supporting configuration of MTU size through "'ifconfig eth0 mtu <x>" is the right step forward. We look at this for our future releases.

     

    Thanks for you inputs! (will look into crash dump analysis)

    Regards

    Sriram

  • 0 in reply to Srirama Govindarajan
    Prodigy 80 points

    Hi,

    I have updated my kernel to the one from TI816X-Linux-PSP-04.00.00.10 which is Linux 2.6.37 instead of Linux 2.6.34 from PSP 4.00.00.07 and now the kernel does NOT crash when it starts up while getting flooded with jumbo frame packets. It just gets very slow with 99% sirq and ifconfig reporting all packets (correctly) as overruns and frame.

    Unfortunately the patch of setting "EMAC_DEF_MAX_FRAME_SIZE        (9000 + 14 + 4 + 4)" in the davinci_emac.c does not work any longer in 2.6.37. All packets simply get reported as overrun and frame. I did notice that this driver uses 'cpdma' instead of CPU processing so maybe this has something to do with it.

    Any suggestions on making jumbo frames working on this version ?

    One of the reasons I upgraded to this version is that CPU time reporting per process is working in this kernel version. Unfortunately with 1500 byte packets my GigEVision application tops out at 100% CPU when processing around 25Mbyte/second of data (which is the worst case data rate required) so I am looking at any means to improve this.

  • 0 in reply to Henk Jan Ober
    TI__Expert 8250 points

    Hi

    If you are limited by the throughput can you enable the interrupt pacing feature supported by the HW - this should improve the throughput and relieve CPU loading as well.

     

    You can do so, with the ethtool utility as follows:

    # ethtool -C eth0 rx-usecs 250

     

    I will look into the jumbo frame part in the meanwhile

     

    Regards

    Sriram

  • 0 in reply to Srirama Govindarajan
    Prodigy 80 points

    Hi,

    Your suggestion made an incredible difference in performance. I went from 99% CPU worst case to 53% CPU worst case! Especially the load from software irqs went down (from 71% to 15%).

    Thanks ! This really helps to validate this platform as a suitable candidate for our application.

    Henk Jan

  • 0 in reply to Henk Jan Ober
    Prodigy 10 points

    Hi,

    I have one question:

    did jumbo frames finally work with kernel 2.6.37?

    and if yes, what changes needed to the source code apart from changing the EMAC_DEF_MAX_FRAME_SIZE?

     

    thank you in advance

  • 0 in reply to Konstantinos Karampogias
    TI__Mastermind 25055 points

    Hi,

    I will forward this question, but in the future I recommend starting new forum posts with new questions rather than adding them to older closed forum threads.

    Regards,
    Marc