NDK, TFTP for File Transfer Host-EVM

Hi Rizuan,

Which version of the NDK are you using?

As a first suggestion, I would recommend to get your TFTP transfer working in a "non OpenMP" example, such as the standard NDK helloWorld or Client example that comes with your MCSDK.  This is not necessary, but just a suggestion as this would eliminate bringing in unnecessary complexities into the app.

Now, the problem that you are seeing sounds like the NDK is running out of memory and so there's a dynamic memory allocation that's failing.

The way to fix this is dependent on the NDK version you are using.  Once you respond with that version let's take it from there.

Another thing, does your application have the telnet console available?  If so, can you telnet into the board and run the command "mem"?

If not, you could add a call to dump the memory allocation information (same thing that "mem" command does) near the point of failure:

_mmCheck( MMCHECK_MAP, &printf );

Steve

Hi Steve,

Thank you for the reply. Unfortunately I can't do as you suggested today because I don't have access to CCS today. Will try those later.

For my NDK version, it is ndk_2_21_01_38.

I was surprised when received the error and I thought the memory allocation was from the 'buffer' which is static. Thanks for your clarification.

Thanks Steve,

Rizuan

Rizuan,

RIZUAN said:

I was surprised when received the error and I thought the memory allocation was from the 'buffer' which is static

The TFTP code contains a call to allocate memory; this is the call that appears to be failing.  So it's separate from your buffer.

RIZUAN said:

For my NDK version, it is ndk_2_21_01_38

Ok, great.  You will need to increase the number of pages available in the NDK's scratch memory.

One more thing to verify before trying the following steps.  Can you please verify that your *.cfg file has a setting like:

Global.enableCodeGeneration = false;

If so, then you should follow these steps:

1. Copy the file ti/ndk/os/mem_data.c into your project

2. Add the following define to your project options (C compiler options):

-D_NDK_MIN_PAGE_SIZE

This will "unlock" the scratch memory settings.

3. Add the following additional defines to your C compiler options (see screen shot below).  Note that below I'm increasing the value for RAW_PAGE_COUNT so that there are more memory pages available for the NDK's mmAlloc() calls:

-DRAW_PAGE_SIZE=3072

-DRAW_PAGE_COUNT=32

4. Rebuild your application

If you run the "mem" command from telnet (or add the call to _mmCheck as mentioned above) you will see that the total available memory has increased from 3072 * 16 = 49152 bytes to 3072 * 32 = 98304 bytes.

Steve

Hi Steve,

I recreated the project using the empty RTSC project, and do as you suggested before. The project is as attached. When I called the _mmCheck(MMCHECK_MAP, &printf);, I received the following output:

[C66xx_0] TFTP Reported Error: -3

[C66xx_0]

[C66xx_0]    0:48              0:96              0:128             0:256         

[C66xx_0]    0:512             0:1536            0:3072        

[C66xx_0] (0/0 mmAlloc: 1/1/0, mmBulk: 0/0/0)

(I defined MMCHECK_MAP as 0, is this right?) I even tried to change the value of RAW_PAGE_COUNT to 64 and it gave me the same output.

4670.TFTPtest.zip

Rizuan,

It looks like your application is not configured correctly for SYS/BIOS (your *.cfg file does not bring in proper BIOS modules, main() does not call BIOS_start(), etc.).  The above output is not correct probably because the NDK is not even running.

I recommend that you start with the NDK helloWorld example that ships with your MCSDK.

Once that is working, I would then take that project and then add in your code for TFTP into it.  Maybe even starting with a smaller TFTP transfer, then once that is working, make the size bigger until you get up to what you need.

Steve

Hi Steve,

Thanks again for the support and patience. In my first project that I submitted, it was based on the Hello World OpenMP example. It ran even without calling to BIOS_start(). From the project, I include the call to the NtTftpRecv() function and I thought it would work. Then I create the new project and copy from the original one. On top of that, on each core I added the ..\..\emulation\boards\evmc6678l\gel\evmc6678l.gel file.

If I need to call the BIOS_start(), does this mean that I need to call the NtTftpRecv() from a task? Can OpenMP run in a task?

Regarding the HelloWorld on NDK, I read that the example has been moved to NSP package. From the NSP (1_10_00_03), there are no example for C6678, only for EVM6678, and OMAPL138. Can you suggest me how to get the example for the C6678 on NDK HelloWorld?

I'm sorry, I'm quite new with CCS and your help so far has improved my knowledge about it and thanks for your patience.

Best regards,

Rizuan

Rizuan,

RIZUAN said:

Thanks again for the support and patience. In my first project that I submitted, it was based on the Hello World OpenMP example. It ran even without calling to BIOS_start(). From the project, I include the call to the NtTftpRecv() function and I thought it would work. Then I create the new project and copy from the original one. On top of that, on each core I added the ..\..\emulation\boards\evmc6678l\gel\evmc6678l.gel file.

If I need to call the BIOS_start(), does this mean that I need to call the NtTftpRecv() from a task? Can OpenMP run in a task?

Ok.  I'm not too familiar with OpenMP and we may have to get someone else to help you with that.

RIZUAN said:

Regarding the HelloWorld on NDK, I read that the example has been moved to NSP package. From the NSP (1_10_00_03), there are no example for C6678, only for EVM6678, and OMAPL138. Can you suggest me how to get the example for the C6678 on NDK HelloWorld?

Yes, the MCSDK in actuality "is" and NSP because it has the Ethernet driver and examples ... the NDK uses "NSP" generically ... sorry for the confusion.

I'm not sure which version of the MCSDK you are using, but I see the helloWorld example on my machine here:

C:\ti\mcsdk_2_00_09_21\examples\ndk\helloWorld\evmc6678l

I think you would learn a lot by starting with this hello example, which is just a single core example of the NDK using SYS/BIOS.  You may also want to have a look at the client example, which has TFTP built into its Telnet console:

C:\ti\mcsdk_2_00_09_21\examples\ndk\client\evmc6678l

Once you're familiar with the NDK and BIOS more, then I would suggest changing one of the above examples to make your TFTP transfer.

Then once you have that working, the next step would be to integrate it into your OpenMP application.

RIZUAN said:

I'm sorry, I'm quite new with CCS and your help so far has improved my knowledge about it and thanks for your patience.

Don't worry about it, that's why we have these forums! :)

Steve

Hi Steve,

Here am I again. You are right, it is essential to run the example, at least the client_evmc6678l. This is the project example 7115.client_evmc6678l.zip but I've added several lines as previous to call the NtTftpRecv function. However, I received the return value -4, something about internal socket error. My guess is I'm putting those codes at the wrong line. Maybe my understanding of the example is not good.

I have a basic knowledge about DHCP (using it as client on desktop computer, not running it as server), thus I configure my board to use the static IP (192.168.2.100) as in the example. The host is 192.168.2.101 with subnet mask 255.255.255.0. However, I believe the error message is not about my tftp server but something about the code. When I run the code on core0, I received the following output:

[C66xx_0] QMSS successfully initialized

[C66xx_0] CPPI successfully initialized

[C66xx_0] PA successfully initialized

[C66xx_0]

[C66xx_0] TCP/IP Stack Example Client

[C66xx_0] PASS successfully initialized

[C66xx_0] Ethernet subsystem successfully initialized

[C66xx_0] Ethernet eventId : 48 and vectId (Interrupt) : 7

[C66xx_0] Registration of the EMAC Successful, waiting for link up ..

[C66xx_0] Service Status: Telnet   : Enabled  :          : 000

[C66xx_0] Service Status: HTTP     : Enabled  :          : 000

Can you advice me on where to call the NtTftpRecv function? From my understanding, it should be after calling the NC_NetStart. I've tried to put it inside the NetworkOpen function but the same error occur. In the attached project, I've putted the code lines after the do while for the NC_NetStart() that I believe is non-reachable.

I've used your previous suggestion to increase the memory.

/*Some addition*/

I'm using MCSDK 2.1.0.2 Beta and CGT 7.4.1. I don't know why I couldn't compile with the CGT 7.4.0 Beta.

Thanks, Steve,

Rizuan

Rizuan,

You should do all of your networking work inside a task thread.  Task threads can be created either statically (as part of the SYS/BIOS configuration *.cfg file) or dynamically at run time.

For example, you could use the NDK's dynamic task create API "TaskCreate" to create a new task thread at run time from the NetworkOpen() function.

Your corresponding task function would then contain the sockets code (TFTP code).  Note that before calling any sockets APIs you must have called fdOpenSession() and and end with fdCloseSession().

Please refer to the NDK API guide (spru524h.pdf) for detailed info on TaskCreate().

Steve

Steve,

5265.client_evmc6678l.zip

I use TaskCreate in NetworkOpen() right after local server creation for my function, RecvTFTP. Inside this function, as you adviced, I called the fdOpenSession() before using the NtTftpRecv(). Before that, I tried to create socket for tftp by using socket() and set some of its option through setsockopt() before call the NtTftpRecv(). I'm still receive the -4 error as before.

My guess is I'm doing wrong on the socket configuration. Could you please point me the right way? I tried to find any thread discussing about socket configuration for tftp but couldn't find anything. I tried to change the parameter according to the spru524h.

Kind regards,

Rizuan

Rizuan,

Let's come back to the NDK client example.  Can you run this example again?

The client example has a Telnet terminal built into it.  Once it's up and running, you can telnet into the board using the IP address printed out by the example.

Once inside, can you try the following command to see if you're able to successfully transfer a file?

>tftp

[TFTP Command]

Called to retrieve a file from a TFTP server.

tftp x.x.x.x myfile  - Retrieve 'myfile' from IP address
tftp hostname myfile - Resolve 'hostname' and retrieve 'myfile'

>

If that works, then you can see the code for this in the file:

ti/ndk/tools/console/contftp.c

Steve

Hi Steve,

Thanks for the reply. I am now able to receive a file (with size around 134 MBytes) through the code and the telnet. The problem was from the IP address. It should be 1694673088 for 192.168.2.101 and not 3232236133 as in the project that I submit before.

Do you know how to increase the speed of transfer? I've tried to change the DATA_SIZE in tftpinc.h by increasing the SEGSIZE in tftp.h from 512 to 1024 but it cause error. Currently the speed is around 467 kbytes/sec and it is very slow.

For the benefits of the others, I will post the project later.

Thanks Steve,

Rizuan

Hi Rizuan,

That transfer speed can depend on a number of factors, including your network hardware setup and where the server is that you are actually transferring the file from.  But these may be out of your control.

To maximize TCP transfer speeds in the NDK, you should increase the size of the TCP send and receive buffers.  The default for these is 8K but you can increase them up to 64K.

I believe you said that your configuration has "Global.enableCodeGeneration = false" so then you will need to make this change at run time in the NDK thread as follows:

    // TCP Transmit buffer size
    rc = 65536;
    CfgAddEntry( hCfg, CFGTAG_IP, CFGITEM_IP_SOCKTCPTXBUF,
                 CFG_ADDMODE_UNIQUE, sizeof(uint), (UINT8 *)&rc, 0 );

    // TCP Receive buffer size (copy mode)
    rc = 65536;
    CfgAddEntry( hCfg, CFGTAG_IP, CFGITEM_IP_SOCKTCPRXBUF,
                 CFG_ADDMODE_UNIQUE, sizeof(uint), (UINT8 *)&rc, 0 );

    // TCP Receive limit (non-copy mode)
    rc = 65536;
    CfgAddEntry( hCfg, CFGTAG_IP, CFGITEM_IP_SOCKTCPRXLIMIT,
                 CFG_ADDMODE_UNIQUE, sizeof(uint), (UINT8 *)&rc, 0 );

Furthermore, you should inspect where your NDK buffers are being placed. You'll want to place these buffers into the fastest memory available to you (internal memory if possible).  You can see where they're being placed in your *.map file.

This should increase your performance.

Steve

Hi Steve,

Those lines are already in the task. I just change the value from 8192 to 65536. Unfortunately there is no speed improvement.

Is there any other way? How about increasing the blocksize. Currently the value of blocksize is 512.

Note: I'm currently using MCSDK 2.1.0.3. My PC connect directly to the board via gigabit ethernet connection (shown from windows task manager). Current speed is about 400 kbytes/sec.

Kind regards,

Rizuan

Rizuan,

It certainly could be due to some bottleneck like this.  But again, it could also be due to network conditions.  We need to eliminate the variables ...

Is the FTP server you are trying to transfer from on the LAN?

Can you try another test?  Can you transfer a data buffer of the same size from that FTP server to your NDK board?  Basically, make the same data transfer but just using a TCP socket with the send() API (to eliminate the TFTP variable).

If the server is not on your LAN, and you try the same test but using a machine on your LAN?  What data rates do you get?  It may also be helpful to make the above test as well on such a set up.

For example, when I test TFTP, I used a free app that runs on my PC called "OpenTFTPServer".  My network set up was such that my PC and test board (running NDK) were on the same LAN, connected to the same switch.

I'd recommend starting with that simple set up first and see what you get.  Because if the TFTP server you are working with now has multiple hops in the connection, or if that server is overloaded, you could be seeing slow performance due to congestion in the server or at any of the routers along the route to that server.

If you just make a transfer from your PC to your 6678 board with the two connected to the same switch, you can eliminate a lot of other possibilities.

Steve

Hi Steve,

I connect the board directly to my PC, and it is a gigabit ethernet connection according to what is detected in Task Manager in Windows 7. My PC host the tftp server, and I've tried 2 different sofware; TFTP32 and WinAgents TFTP Server. Both gives the same speed. I believe, server is not the factor.

I've tried using the WinAgents TFTP Server and use the TFTP32 as tftp client, and the rate is high even by only using the default blocksize, 512. I believe this doesn't count because both sofwares are on the same machine.

Well, I will try to connect another PC directly to the tftp server PC to see its transfer rate. I will also try the send() API. According to other thread (don't remember which), sending data from the board to PC will be faster that the other way around. There is also due to version of MCSDK and NDK. To eliminate this, I'm using the recently released MCSDK.

Thanks Steve, have a nice weekend.

Rizuan

Hi Steve,

I've connected 2 PC at each end of an ethernet cable. One is running tftp32 as server and the other one as client. By setting the block size as 512, it takes only 145s to transfer 234881024 bytes of data. The transfer rate is around 1.4Mbytes/sec (yes, it is in bytes/sec as reported from the tftp32 software)

Whereas for the board, it takes 482s to transfer (receive) the same size of data at around 0.4Mbytes/sec by using the NtTftpRecv().

In both cases, gigabit connection was used. Clearly the slow transfer rate is from the board and not network.

I've changed the transmit and receive buffer for TCP and UDP to 65535 but nothing change in the transfer rate. Even at 145s to transfer the data it is still slow. I am wondering whether we can change the block size from the default value of 512? I've included the tftp.c, tftp.h and tftpinc.h from ndk\nettools\tftp folder into my project and change the value of SEGSIZE from 512 to 1024 but it caused an error during transfer.

Kind regards,

Rizuan

Hi Rizuan,

RIZUAN said:

I've included the tftp.c, tftp.h and tftpinc.h from ndk\nettools\tftp folder into my project and change the value of SEGSIZE from 512 to 1024 but it caused an error during transfer.

Is your project linking in the IPv6 libraries? I'm guessing that it is.  You can check in the *.xdl file that's generated in  your project.

If you're using IPv6 libraries can you check that you have the following defined for your project?

_INCLUDE_IPv6_CODE

Please make sure this is defined in your project.

RIZUAN said:

change the value of SEGSIZE from 512 to 1024 but it caused an error during transfer.

I checked the code quickly and I didn't see why this wouldn't work.  maybe your trick of copying the files into your project isn't working as it seems (some mismatch of lengths...).  But that's just a guess.  One way to ensure that's not the case is to rebuild the nettools library with your TFTP changes, remove the TFTP files from your project and rebuild your application.

Steve

Hello Steve,

Thanks for the reply.

Steven Connell said:

Is your project linking in the IPv6 libraries?

No. I just use IPv4 and I didn't define the _INCLUDE_IPv6_CODE

Steven Connell said:

One way to ensure that's not the case is to rebuild the nettools library with your TFTP changes, remove the TFTP files from your project and rebuild your application.

Done that. It produces the same result. That mean, copying the files is actually worked. Actually, when I change the value of SEGSIZE to 1024, 2048, only 512 bytes are retrieved eventhough the return value is 1 (according to SPRU524h, 1 indicates the file was received and copied into the buffer, 0 indicates the files was received but too large for the buffer).

Rizuan

Hi Steve,

I've done some debugging on the tftp.c. When I change SEGSIZE from 512 to 1024, the recvfrom() still read only 512+4 (4 is tftp header I believe). That is why only 512 bytes are read, when only partial block are received, then the transmission stop because 512 < 1024 + 4. See tftp.c line 386.

Maybe tftp.c is fixed only for SEGSIZE = 512? I hope this is not correct.

Rizuan

Rizuan,

I think you may need to set the socket option "MSG_WAITALL"

Can you try changing the receive call to use this flag?

    BytesRead  = recvfrom( pTftp->Sock, ReadBuffer, DATA_SIZE, MSG_WAITALL,
                           (struct sockaddr *)&pTftp->tmpaddr, &addrLength );

Steve

Hi Steve,

Thanks very much for your help. I've just tried that. I do that and change the SEGSIZE to 1024 and recompile the NDK library. Only 512 bytes have been received. Similar if I just copy the tftp.c, tftp.h and tftpinc.h to my project folder and introduce the changes.

Now I tried to change my network adapter to another one and it is not because of it. Really hope this issue can be addressed.

Rizuan