ATA File System File Navigation Issues

Benjamin Michaud

Expert 1825 points

Other Parts Discussed in Thread: TMS320C5515

Hi,

I'm currently working on a project which utilizes a SD card to store files on.

I have been working with the ATA File System provided by TI, but have found it's behavior to be unpredictable.

Being able to create, navigate, and modify files using the ATA File System has proved problematic.

Below are some of the tests I've conducted with the file system and the results.

Performed the following Test:

Opened a file, wrote 512 bytes, rewind, read 512 bytes, close.

Reopened the file, wrote 44 bytes, rewind, read 512 bytes, close.

The filed opened with no issue in Windows, and the data was correct.

Performed the following Test:

Opened a file, wrote 512 bytes, wrote 512 bytes, rewind, read 512 bytes, close.

Reopened the file, wrote 44 bytes, rewind, read 512 bytes, close.

The file was corrupted when opened in Windows.

I have found some work arounds for the issues I've discovered.

For instance, if a file is closed but the file pointer isn't at EOF, then the file will be reported corrupted by Windows when you try to open it.

Work around, put a seek(file, 0xFFFFFFFF) to make sure the file pointer is at EOF before closing.

Additional problem discovered, if a file pointer is at EOF, seek(file, 0xFFFFFFFF) shouldn't be called, instead just close the file, otherwise the file will be reported corrupted by Windows when you try to open it.

Has anybody else seen behavior similar to this?

Also, I based all my tests off of the CSL 3.04 CSL_MMCSD_SdCardFSExtExample project.

Thanks,

Ben

Performed the following Test:

Opened a file, wrote 512 bytes, rewind, read 512 bytes, close.

Reopened the file, wrote 44 bytes, rewind, read 512 bytes, close.

The filed opened with no issue in Windows, and the data was correct.

Performed the following Test:

Opened a file, wrote 512 bytes, wrote 512 bytes, rewind, read 512 bytes, close.

Reopened the file, wrote 44 bytes, rewind, read 512 bytes, close.

The file was corrupted when opened in Windows.

over 9 years ago

0 Steve Tsang over 9 years ago

TI__Mastermind 23785 points

Thanks for reporting. Once it's been verified, it will be reported for future enhancement.
Regards.

0 Steve Rapinchuk over 9 years ago

Prodigy 100 points

Benjamin,

Although I don't remember the exact details, we also had similar problems with the TI SD library. We ended up purchasing the following FAT32 library from Applied Signal Processing, and have been quite happy with it. (It does have some limitations, like 8.3 filenames.)

appliedsignalprocessing.com/fatcfhd.htm

One thing that many people don't realize is that SD cards (and other flash-based drives such as SSD and CF) perform periodic garbage collection.

http://en.wikipedia.org/wiki/Write_amplification#Garbage_collection

During this time, the SD card will be busy; you won't be able to read from or write to it. Although this is documented in the SD spec, it seems that not many people know about it. It varies from card to card, but it seems to happen once every 1 to 10 minutes. Although this busy time is usually less than 250 ms, on some SD cards I've observed it to be as long as 700 ms. (The SD card spec says it can be busy for up to 500 ms; apparently card manufacturers ignore this.) This poses problems when trying to write real-time data like audio when you don't have enough RAM to buffer the data while the card is busy. We did find out that we can reduce the occurrence and duration of the busy time by always writing sequentially on a sector (512 byte) boundary. Longer writes are better. We were originally writing a 44 byte wave header, followed by writes that were 8K in length. Because of the 44 byte offset, the first and last sector of this 8K write would cause the garbage collection to erase sectors. By incorporating the 44 byte header into the first 8K write, we significantly reduced the occurrence and duration of the card being busy.

I hope this information is helpful. I would recommend enough RAM to buffer at least 1 second of data. You can do it with less, but you are going to spend a lot of time tweaking your code to get it to work.

Steve

0 Steve Rapinchuk over 9 years ago in reply to Steve Rapinchuk

Prodigy 100 points

I found this in the SD Spec. This is how card manufacturers get away with long busy times.

SD Specifications

Part 1
Physical Layer Simplified Specification
Version 4.10
January 22, 2013

4.6.2.2 Write

Application Notes:

It is strongly recommended for hosts to implement more than 500ms timeout value even if the card indicates the 250ms maximum busy length.

0 Benjamin Michaud over 9 years ago in reply to Steve Rapinchuk

Expert 1825 points

Hi Steve,

Thank you for the information.

We started looking in other directions and are testing a file system found at the following link:

http://elm-chan.org/fsw/ff/00index_e.html

It is working great when staying in 8.3 filename format, but is giving issues when trying long file names.

Unfortunately we need to support long file names in this project, otherwise that file system you have a link to would be great especially considering it was designed to run on products from TI.

We spoke to CMX about using their FAT32 file system modules, and their engineers believe there will be considerable work to get the file system to work on the TMS320C5515 because it does not have byte level primitive data types.

We just have to keep working with it until we find an acceptable solution.

Thanks!

Ben

0 Steve Rapinchuk over 9 years ago in reply to Benjamin Michaud

Prodigy 100 points

Ben,

We were also looking at RTFS from EBS:

http://ebsembeddedsoftware.com/NewWebSite/AboutRtfs.html

It's got a lot more features (including long filenames and exFAT support), but it hadn't been ported to the C5000 platform. (They were willing to port it as they already supported some other TI processors.) We decided on the ASP product since it was already ported and tested.

Please update this thread when you find a solution. I may have need for a C5000 supported file system with long filename support in the future.

0 Benjamin Michaud over 9 years ago in reply to Steve Rapinchuk

Expert 1825 points

Steve,

Not a problem, I will let you know the outcome either way.
The guys over at EmbCode seem willing to work with us in regards to getting their FatFS up and running on the C5515, which I would assume could trickle over to any C55xx.
Here's their link:
www.embcode.com/.../EcFAT
We just don't know which direction we're taking yet.

Ben

0 Benjamin Michaud over 9 years ago in reply to Steve Rapinchuk

Expert 1825 points

Hi Steve,

It looks like we might move forward with the FatFs from ChaN:

www.embcode.com/.../EcFAT

However, like said on the webpage, care must be taken if the data types are not the size they are stated to be.

So essentially in our C55xx world, anything declared as BYTE in the FatFs module is now suspect since it is 16 bits wide.

I found an issue in sum_sfn() where the fact that the variable sum was declared as a BYTE led to an issue.

I needed to modify it to properly treat sum as a BYTE (essentially just bitwise anding it with 0x00FF (sum & 0x00FF)).

There might be other areas in the code where the same issues persist.

Therefore I'm debating declaring a structure of bits

typedef struct

{

unsigned int useless_upper_8_bits : 8;

unsigned int actual_byte : 8;

}REAL_BYTE;

and then changing everything, but that will take some time I think given the size of the code.

I'm thinking problems like this will persist with the C55xx line when software from 3rd parties is used as in the case where we spoke to CMX.

When we spoke to CMX about the microcontroller we're using, and no native 8 bit data type available, they said that would be a big problem for their FAT FS module.

Good luck,

Ben

0 SMART over 8 years ago in reply to Benjamin Michaud

Intellectual 860 points

Hi ,

How to use long filename in TI C5515 (SD card)?

Processors

Processors forum

ATA File System File Navigation Issues