AM2732: LWIP: Issue with big package receiving

Add: On the 2732 side, a TCP server is established to receive data. The TCP client sends dozens of 30KB packets, which has a high probability of causing the TCP server to hang. Currently, reducing the transmission frequency or size can reduce this probability.

Hi Gmsy, Chris

I am looking into this issue.

Regards,

Shaunak

Hi Chris & Gmsy, 

Sorry for the late response, the steps to increase the capacity of the client or server are:

1) In MCU_PLUS_SDK/source/networking/lwip/lwip-config/am273x/lwippools.h, increase the number of elements, similar to the screenshot attached:

2) The memory regions in the linker.cmd might also have to be modified accordingly to accommodate the increase number of elements in LWIP_POOL

3) Rebuild the libraries, rebuild the example and it should work.

Regards,

Shaunak

Our cfg is 

LWIP_MALLOC_MEMPOOL(80, 128)
LWIP_MALLOC_MEMPOOL(80, 1568)
LWIP_MALLOC_MEMPOOL(10, 4096)

The problem is the same. We don't have enough memory to increase further. Could you please help us see if you have the same problem based on this configuration?

Hi Gmsy,

Can you please share your linker.cmd file? I wanted to check the placement of LwIP Pbufs in the memory.

Regards,

Shaunak

r5f_linker_enet.zip

Hi Gmsy,

I see that you store LwIP Pbufs in MSS L2 region and you have 50.176 KB of space in MSS L2 region.

Any specific region for not using DSS L3 for Pbufs?

Also, at maximum, MSS_L2 region can be of 960 KB. You can try increasing the MSS L2 length or use DSS L3 which can be stretched to 3.5625 MB

Regards,

Shaunak

Hi Shaunak,

Our MSS_ L2 only has 7.5KB left, DSS_ The L3 algorithm is in use. As discussed last time, we have increased the number of network MEMPOOLs to 80, but the phenomenon still persists. Is it possible that the problem is due to other reasons?

Hi Gmsy,

Even though you have increased the number of Instances to 80, if memory is not available for Pbufs to be allocated, it won't work, which I think is the case here, To dig deeper into the issue, you might want to enable the LWIP debug logs from source/networking/lwip/lwip-config/am273x/lwipopts.h

Then rebuild the library and the project, the Debug logs will help you identify the problem.

Note: Rebuilding project through CCS does not rebuild the lwip library.

Regards,

~Shaunak

Hi Gymsy/Chris,

Can you please try to enable LWIP_DEBUG logs from source/networking/lwip/lwip-config/am273x/lwipopts.h. It would be a bit difficult to root cause the issue without error logs.

Because of memory limitations we have in this case, it is highly possible that we are running out of memory while receiving packets, thus the NETBUF will be NULL pointer, which might be giving a memory access related error, in turn closing the connection (this explains why the ping won't work).

Regards,
Shaunak

Shaunak,

I would like to enable the debug log, but there are a lot of them in header file. I tried below setting but didn't see any new print. Would you pls kindly advise if I miss anything or I need enable all?

#define LWIP_DEBUG 1

#ifdef LWIP_DEBUG

#define LWIP_DBG_MIN_LEVEL 0
#define PPP_DEBUG LWIP_DBG_OFF
#define MEM_DEBUG LWIP_DBG_ON
#define MEMP_DEBUG LWIP_DBG_ON
#define PBUF_DEBUG LWIP_DBG_ON
#define API_LIB_DEBUG LWIP_DBG_OFF
#define API_MSG_DEBUG LWIP_DBG_OFF
#define TCPIP_DEBUG LWIP_DBG_OFF
#define NETIF_DEBUG LWIP_DBG_OFF
#define SOCKETS_DEBUG LWIP_DBG_OFF
#define DNS_DEBUG LWIP_DBG_OFF
#define AUTOIP_DEBUG LWIP_DBG_OFF
#define DHCP_DEBUG LWIP_DBG_OFF
#define IP_DEBUG LWIP_DBG_OFF
#define IP_REASS_DEBUG LWIP_DBG_OFF
#define ICMP_DEBUG LWIP_DBG_OFF
#define IGMP_DEBUG LWIP_DBG_OFF
#define UDP_DEBUG LWIP_DBG_OFF
#define TCP_DEBUG LWIP_DBG_ON
#define TCP_INPUT_DEBUG LWIP_DBG_ON
#define TCP_OUTPUT_DEBUG LWIP_DBG_OFF
#define TCP_RTO_DEBUG LWIP_DBG_OFF
#define TCP_CWND_DEBUG LWIP_DBG_OFF
#define TCP_WND_DEBUG LWIP_DBG_OFF
#define TCP_FR_DEBUG LWIP_DBG_OFF
#define TCP_QLEN_DEBUG LWIP_DBG_OFF
#define TCP_RST_DEBUG LWIP_DBG_OFF
#endif

#define LWIP_DBG_TYPES_ON (LWIP_DBG_ON|LWIP_DBG_TRACE|LWIP_DBG_STATE|LWIP_DBG_FRESH|LWIP_DBG_HALT)

I reproduced customer’s issue on AM2732EVM (mcu_plus_sdk_am273x_08_04_00_17) and see the length return by netbuf_data call is abnormal (too big) when issue happens. Do you have more comment on which debug I need to enable for further check?

netbuf_data(buf, &data, &len);
printf("len=%d\n",len);  

len=1460

len=1460

len=1460

len=1460

len=1460

len=1460

len=1460

len=1460

len=28005

Thanks,

Chris 

Hi Chris,

Just making sure, you rebuild the LwIP Libraries after the above change and then clean and rebuild the example right?

Regards,
Shaunak

Shaunak,

Must clean and rebuild lib instead of incremental build after above change?

I ran below command, and doesn't run clean.

gmake -s libs PROFILE=debug

Thanks,

Chris

Hi Chris,

The above command will work and build the lib, please make sure you rebuild the example as well again. The above abnormal length seems to be because of some corrupted packet, which again seems to be some memory limitation issue, as the standard netbuf size would be 1460 only.

1. Can the customer use TCM memory or SBL memory since they have a memory limitation here.

2. Also, before you print the length of the netbuf, can you also check if the netbuf is NULL. 

Regards,
Shaunak

Shaunak,

I used different cmd which uses L3 for LWIP buffer. Pls check below.

https://e2e.ti.com/cfs-file/__key/communityserver-discussions-components-files/908/4645.linker.cmd

I changed lwippools.h to increase the number of elements as you suggested.

I enabled all debug log than I can see debug info, but the test case can't receive any eth package then hung. 

I will try to check if the netbuf is NULL. I still need your suggestion on which debug info need to turn on.

Thanks,

Chris