FAT32 library for MMC/SD cards

[marina.petrovic]

Hi,

I believe I already answered you.
viewtopic.php?f=88&t=54595&p=214296#p214296

Best regards,
Marina

[borris]

I apologize for asking this question, but I need some additional help on swap file creation and use with the FAT32 library.

For example:

System will download a unknown data size. How could I use swap files to speed up write operations?

Can I create a "huge" swap file? When I close the swap file, how do I convert whatever was saved to the swap into a normal file?

Unfortunately, there isn't any decent example that I could find anywhere.

Any help would be greatly appreciated.

[marina.petrovic]

Hi,

The most important thing, that you need to know when you want to use swap file is that you have part of the memory
(with certain consecutive sectors) that FAT can not change any more.

When you create swap file you will get the number of sectors for the swap file,
and that can be 1 sector if you want ("huge" swap file) or any other number of sectors.

Then you can use: Mmc_Read_Sector() and Mmc_Write_Sector() functions
(the function reads/writes one sector from MMC card).

When you finish, with FAT32_Open() you can create new file, use Mmc_Read_Sector()
to read from swap file and then write in the created file.
Function reads from the swap file and do the necessary formatting.

Also, you can take a look at the example that is provided with FAT32 Library,
there you have section that demonstrate creation of a swap file.

Best regards,
Marina

[Dany]

I apologize for asking this question, but I need some additional help on swap file creation and use with the FAT32 library.
For example:
System will download a unknown data size. How could I use swap files to speed up write operations?
Can I create a "huge" swap file? When I close the swap file, how do I convert whatever was saved to the swap into a normal file?
Unfortunately, there isn't any decent example that I could find anywhere.

Hi, perhaps this can help:

A SwapFile is always created as a file of consecutive physical sectors. This means that one can read and write the file with the raw read/write sector commands of the medium (e.g. mmc card) involved if one knows the physical start sector of the file and the length of the file in sectors.

The writing and reading to the swapfile is done sector per sector in a random access manner. Since the native medium read/write routines are used here the speed is maximal.

Some remarks:
- The file should be created with a size that should be big enough for its entire lifespan. The size can not made bigger afterwards without losing the “swap file” criterion (consecutive sectors),
- You will have to remember the start sector and the number of sectors somewhere (e.g. eeprom) if you want to continue to use the file as a swap file,
- The file can however also be accessed by the normal Fat16/32 routines. It is then handled as a normal Fat16/32 file. All routines work as with any other file, BUT:
- Extending the file with the normal Fat32 routines is also possible, but there is a chance that, after doing so, the file will be no “SwapFile” any more: the added sectors are most probably not consecutive any more.

the above quote is an extract from http://www.rosseeld.be/DRO/PIC/Fat32_2.pdf (does NOT describe the mE Fat32 library, but some of its content (like the swapfile description) complies with the mE Fat32 behaviour).

[borris]

Dany,

I appreciate the response and your attention to detail. What I am struggling with is probably something simple. I understand the creation of a swap file and what it represents, but what I am confused about the FAT32_WriteSwap() function found in the FAT32 library. They are poorly documented and lack any real example.

Here is the fundamental that I'm confused with:

The FAT32_WriteSwap function is described as:

int8 FAT32_WriteSwap(short fHandle, char *wrBuf, uint16 len);

It doesn't mention anything in the description about sector writes.

When I write to the swap, say 1023 bytes. How do I read from the swap and place it into a regular file? 1023 isn't quite 2 sectors so....how would you write a partial sector? I know that question shows my lack of experience with FAT but I believe if someone where to offer up an example, people would gain alot from it.

[Dany]

Dany,

I appreciate the response and your attention to detail. What I am struggling with is probably something simple. I understand the creation of a swap file and what it represents, but what I am confused about the FAT32_WriteSwap() function found in the FAT32 library. They are poorly documented and lack any real example.

Here is the fundamental that I'm confused with:

The FAT32_WriteSwap function is described as:

int8 FAT32_WriteSwap(short fHandle, char *wrBuf, uint16 len);

It doesn't mention anything in the description about sector writes.

When I write to the swap, say 1023 bytes. How do I read from the swap and place it into a regular file? 1023 isn't quite 2 sectors so....how would you write a partial sector? I know that question shows my lack of experience with FAT but I believe if someone where to offer up an example, people would gain alot from it.

Indeed, I do not understand it any more, the help is rather confusing. It seems the file must be opened as a regular file to be able to use the MakeSwap, ReadSwap, SeekSwap and WriteSwap routines.

Anyway, the sector by sector reading of a swapfile is "normally" no business of the FAT library, one uses simply the read/write sector routines of e.g. the mmc library. Of course one needs to know the startsector and the number of sectors of the swapfile.

But... it is possible that mE made read/write sector of a swapfile easier with the "ReadSwap", "WriteSwap" and "SeekSwap", but that is not clear at all from the helpfile. The helpfile suggest that those 3 routines act the same as the "normal" (non swap) ones...

I think the help file is messed up (or at least a bit unclear)...

[marina.petrovic]

Hi,

Difference between Write_Sector() function and FAT32_WriteSwap() function is that FAT32_WriteSwap() have a pointer that will be set to the place where you earlier have stop writing and you can use that function when you want to consecutive write in the swap.

Precisely, you open some file, then write something and close the file, function will "remember" how far did you get with writing and when you re-open the file, function will start writing from the remembered position (first it reads and then write from file and because of that it's little bit slower).

Advantage is that you can write arbitrary length without thinking.

Dany has the right, file must be opened as a regular file in order to use FAT32_WriteSwap() function.

"Common" Mmc_Write_Sector() function, use the buffer that you already have and just write in the sector,
because of that it is the fastest way to write into the sector.

As for Help file, we will try to add some additional information regarding swap functions, in order to be more understandable.

Thank you for your suggestions and interest, we really appreciate it.

Best regards,
Marina

[Dany]

Hi,

Difference between Write_Sector() function and FAT32_WriteSwap() function is that FAT32_WriteSwap() have a pointer that will be set to the place where you earlier have stop writing and you can use that function when you want to consecutive write in the swap.

Precisely, you open some file, then write something and close the file, function will "remember" how far did you get with writing and when you re-open the file, function will start writing from the remembered position (first it reads and then write from file and because of that it's little bit slower).

Advantage is that you can write arbitrary length without thinking.

Dany has the right, file must be opened as a regular file in order to use FAT32_WriteSwap() function.

"Common" Mmc_Write_Sector() function, use the buffer that you already have and just write in the sector,
because of that it is the fastest way to write into the sector.

As for Help file, we will try to add some additional information regarding swap functions, in order to be more understandable.

Thanks Marina.

But... it is possible that mE made read/write sector of a swapfile easier with the "ReadSwap", "WriteSwap" and "SeekSwap", but that is not clear at all from the helpfile. The helpfile suggest that those 3 routines act the same as the "normal" (non swap) ones...

Could you explain then the difference between the Fat32_Read/Write/SeekSwap and the normal Fat32_Read/Write/Seek functions? According the help (and your description) they yield the same result. Or are the ...swap functions more efficient due to the fact that they know the file sectors will be consecutive?

[borris]

I've attached a test project to this post. In it is my attempt to make some sense of:

FAT32_WriteSwap and FAT32_ReadSwap etc. Everything "should" work as to the help file.

If you have some time, please check out the code and let me know where I'm going wrong. Now that I'm just about bald, I sure would like some help on this one.

[borris]

Success!

Silly mistake. Ok, so I have a functioning project. Problem was I was opening the swap file for writing, then trying to read from it. You must, apparently, close the swap file and re-open it in read mode.

The questions remaining are:

If the library keeps track of the sector and make it seamless for reading and writing, why is one of the arguments to the makeswap function a starting sector? Why would you need that information if the library is keep track of that for you?

If anyone would like to see a functional example, one is attached.

[marina.petrovic]

Hi,

Difference between the Fat32_Read/Write/SeekSwap and the normal Fat32_Read/Write/Seek functions is that
Fat32_Read/Write/Seek functions read/writes requested number of bytes to the currently opened file and
Fat32_Read/Write/SeekSwap read/writes requested number of bytes to the currently opened swap file
and yes, swap functions will be more efficient due to the fact that they "know" the file sectors will be consecutive.

Thank you very much borris for sharing the code with our users.
It will certainly be of great help.

As regards of starting sector in MakeSwap function (if you think on location of the first sector of the swap file),
it is related to the MakeSwap function and only represent the location of the first sector of the swap file that you want to make.

Best regards,
Marina

[borris]

I've been enjoying the FAT32 library and am reporting that I have good performance.

One question remains.

I've noticed a couple of issues:

Question - Is there a problem if FAT32_Close is called more than once on the same file handle?

Issue - I've noticed that if the system is on and I remove the SD Card, if I re-insert, even though I have a ISR that re-initializes the card. The only way I can get the card to be recognized is to remove again and press reset. Even if I remove all power from the board, without removing the card and re-inserting, it get a -3 error (Media Error ). How can I insure a reliable initialization everytime?

[marina.petrovic]

Hi,

It is not the problem that FAT32_Close() is called more than once on the same file handle,
but pay attention FAT32_Close() requires that file pointed by fHandle needs to be opened.
If the file selected is not opened (or due to any other error), the routine will return -1.

You need to insert the card into MMC-SD module prior usage, if you remove the SD card and re-insert the card again,
you need to reset your hardware, it is the most secure way to now that SD card will be recognized.

Best regards,
Marina

[Yves32]

My own needs and wishes are modest, concerning FAT32

If I could get only Mmc_FAT32_Init() and FAT32_Read_Sector, I would be very happy. Because, in principle, I could read anything on the card.
I work with Basic32.

Is it something I could get at some time? Actually, I live with the FAT limitations, only for prototyping.
At some point in the near future, I will need the real thing.

[marina.petrovic]

Hi,

Did you try the example provided with FAT32 Library?
Can you tell me which hardware and microcontroller you use?

Also, you need to make sure that SD card is properly formatted (to FAT32 file system) prior testing it on the example
and insert the card into MMC-SD module prior usage.

Best regards,
Marina