Microsoft File Systems

First, don't use them if you can since they do not support many common Linux features as links, and permission features. If you can not avoid to use Microsoft Filesystems under Linux, two options are possible:

  1. Linux has read access only Windows has r/w access then use NTFS. https://www.tuxera.com/

  2. If Linux has to write as well use FAT32, however also FAT32 has its issues.

  3. FAT12 and FAT12 might be interesting to use on micro controller projects.

FAT

Using FAT

If possible do not use FAT under Linux. It is simply too old an very limited.

The problem with FAT32 is still the old DOS issue with the limit of 8 character filenames and 3 character extensions. To exceed those limits there are certain incompatibilities between the different flavors of FAT16 and FAT32 (Win95 or WinNT). Usually you do not notice it, but when suddenly filenames appear in lower or uppercase letters that you have such a problem. File synchronizing tools are sensitive to that and probably drive you crazy especially when the filenames are getting converted from one format into an other.

Mounting a FAT32 partition could look as follows:

mount -t vfat /dev/exthd1 \

/mnt/exthd1 -o \

iocharset=iso8859-15,codepage=850,utf8=true,shortname=mixed,umask=022

See:

/usr/src/linux/Documentation/filesystems/vfat.txt

When creating a fat system with fdisk then you will see there are many different variants. Hard disks shipped are usually W95 FAT32 (LBA) set this is id c.

Memory sticks partitioned formatted with XP are id b Win95 FAT32. Other choices are id 6 FAT16.

Under Gentoo there is the useflag fat to get support for it and emerge dosfstools

mkfs.vfat /dev/sdg1 to format the disk

fatlabel /dev/sdg1 HELLO to change label of the disk

fatlabel /dev/sdg1 to see the change

fsck.vfat /dev/sdg1 to check it

If it is going really bad then testdisk from https://www.cgsecurity.org/wiki/TestDisk is a utility that runs on different operating systems and can restore deleted files and lost partitions. It is good to use testdisk > Advanced > Image Creation to create a image.dd file first (or make even two copies) of the corrupt filesystem and then use testdisk image.dd to try repairing it. testdisk comes with photorec that allows recovering photos and other files from a corrupt filesystem without the hassle of restoring the filesystem. So photorec image.dd might do what it is desired.

FAT internals

FAT12 and FAT16 might be still of interest to be used on micro controller projects, to connect such small micro controllers to Memory devices as SD cards. For a library check http://elm-chan.org/fsw/ff/00index_e.html Please note that SD technology requires companies developing SD products to join their organization https://www.sdcard.org/home/.

FAT (File Allocation Table) has been developed for disk drives. Disk drives contain centric tracks, every track spans over sectors, therefore sectors per track is a key figure. A sector contains typically 512Byte of data. The term sector might be confusing, since a geometrical sector holds as many sectors as the disk track has. Additionally a hard disk can contain multiple magnetic disk inside and each disk usually holds data at both sides (this results in number of heads of the disk).

The disk driver software abstracts this physical arrangement by separating physical sectors from logical sectors. For the boot sector (number 0), the physical sector corresponds to the logical sector as during boot not much software runs.

To not risk to position the heads between successive sector reads and to not have to deal on a high level with sector numbers, the sectors are grouped into clusters. Clusters have a fix length that is a multiple sector size. Cluster size can be something between 2KB and 32KB. Clusters can be chained together to hold more data. Such cluster chains might become physically split apart. De-fragmentation of a disk then moves them back in sequence.

Note

Blocks is a synonym for Clusters

There is therefore a size limit of the disk that depends on

Size of the sectors (usually 512Byte)

Number of sectors in a cluster (1..128)

Bits in the File Allocation Table to hold the clusters (block) number (FAT12 holds 12 bits and therefore can address just 4096 clusters, FAT16 holds 16bits ....)

The File Allocation Table (FAT) holds the cluster numbers, where the clusters hold files content or directory list

To put everything necessary onto the disk, the disk has first to be partitioned simple disk as floppies have just one partition), after that in DOS terms a Volume is created. The volume is usually called FAT, but FAT is the File Allocation Table and is just a part of it. The volume consist of:

  1. Boot sector

  2. One or more File Allocation Tables

  3. Root Directory

  4. The clusters containing the data

The boot sector contains more then the boot routine, it contains the key parameters of the FAT system as: Number of sectors per volume, Number of sectors per cluster, ...

The File Allocation Table is contains all clusters that form a file. The entries of the FAT point to other entries of the FAT until the complete file is covered. The number of the FAT entry is directly linked to the cluster number that holds the data. So looking on the next FAT referenced FAT entry means also looking at its cluster and therefore to the data inside the sectors. Since the first two entries of the FAT are reserved (held the media descriptor), the 2 has to be subtracted from the FAT to get the corresponding cluster number.

The size of data to be accessed is limited and since memory devices become bigger, FAT needed to be revised and new FAT versions got created. In FAT12 the FAT entry consists of 12 bits and creates therefore a limit of having maximum 4096 clusters. It got expanded to FAT16 where FAT entry consists of 16 bits and creates therefore a limit of having maximum 65536 clusters. Note a file consists of at least one cluster.

Since a mess in the FAT would result in not finding the data anymore, a FAT file system usually creates one or more copies of the File Allocation Table to have redundancy.

The Root Directory finally makes the link between cluster number and filename and directory. An entry has a size of 32 Byte and is also the reason why FAT was limited to 8 character filenames and 3 character extension.

NTFS

Be aware when writing to a NTFS file system some Linux drivers might damage it badly.

However there the 3rd attempt to write to NTFS the 3rd generation ntfs driver is available and works fine. So why bother with historic fat32?

Fuse from the kernel source might be too old for ntfs3g, instead:

echo "<=sys-fs/fuse-2.6.0 ~x86" >> /etc/portage/package.keywords

emerge ntfs3g it emerges also the kernel module fuse.

/etc/fuse.conf is missing after the emerge, therefore it takes default values and regular users can not mount the ntfs partition.

echo "user_allow_other" >> /etc/fuse.conf enables this.

modprobe fuse to load the driver and for the next boot

echo "fuse" >> /etc/modules.autoload.d/kernel-2.6

Now you can manually mount the partition as regular user (verify that you have write access to the mounting point /mnt/sda1).

ntfs-3g /dev/exthd1 /mnt/exthd1 or

ntfs-3g /dev/exthd1 /mnt/exthd1 -o locale=de_CH,umask=022

Or mount as user 1000 group users with rwxr-xr-x rights:

ntfs-3g /dev/exthd1 /mnt/exthd1 -o locale=de_CH,uid=1000,gid=100,umask=022

To unmount it type

fusermount -u /mnt/exthd1

To be put in the/etc/fstab :

/dev/exthd1 /mnt/exthd1 ntfs-3g noauto,user,rw 0 0

it mounts with the kde icon, but with errors and I can not unmount. Probably it is too much for kde to mount with ntfs-3g and unmount with fusermount.

Simple scripts to mount/unmount:

      echo "ntfs-3g /dev/exthd1 /mnt/exthd1 -o \
locale=de_CH,umask=022" >> /usr/bin/mexthd
echo "fusermount -u /mnt/exthd1" >> /usr/bin/uexthd

Graphical Python script to do the same:

Example 7.3. GUI mount NTFS

#!/usr/bin/python
from os import *
from tkMessageBox import *
from Tkinter import *

def mount():
#If you want to synchronize a nfts with a ext2 then
#uncomment the line below to make all ext2 stuff
#having the same rights
#system("chmod -R 755 /home/<your username and directory>")
 popchild=popen4("ntfs-3g /dev/exthd1 /mnt/exthd1 -o locale=de_CH,umask=022")  
 childresp=popchild[1].read()
 if len(childresp)==0:
   showinfo("mount","Successfully mounted")
 else:  
   showinfo("mount",childresp)
 return 0

def umount():
 popchild=popen4("fusermount -u /mnt/exthd1")
 childresp=popchild[1].read()
 if len(childresp)==0:
   showinfo("mount","Successfully mounted")
 else:  
   showinfo("mount",childresp)
 return 0

window=Tk()
window.title('Ntfs3g')
window.mountbutton=Button(master=window,text='mount NTFS',command=mount)
window.mountbutton.pack()
window.umountbutton=Button(master=window,text='umount NTFS',command=umount)
window.umountbutton.pack()
window.mainloop()

Be aware that you safely remove your USB HD on your Windows computer (icon in the bottom), so no unfinished Windows journaling data is being created, otherwise the script above can not mount the NTFS partition and a Windows mount umount has to be repeated.

If you synchronize hard disks be aware about the following: Since not all permission features are supported on the NTFS partition, make sure that all files of the non NTFS partition have the NTFS default permission values.

The package emerge ntfsprogs contains a lot of tools to deal with ntfs:

mkftfs /dev/<my disk> to format ntfs

ntfsfix /dev/<my disk> to force windows to check it

Fix permissions

Since Microsoft filesystem do not support the Linux permissions, default permissions is set by the Linux system. This can cause troubles when synchronizing files (e.g. using unison) since the permissions might differ between Linux disk (e.g. Hard disk) and Microsoft disk (e.g. FAT USB stick). Instead of changing the default setting of the system and apply those also to all kind of disks, a dedicated configuration can be done:

  1. Create a udev rule to get a dedicated name for the memorystick that is recognized using its serial number:

                SUBSYSTEMS=="usb", KERNEL=="sd*", ATTRS{serial}=="200512326309FCF28507", NAME="memstick%n"
  2. Add an entry to fstab to /media and if required create the /media/<disklabel> mounting point manually:

                /dev/memstick1/media/<disklabel> auto noauto, user, rw, umask=022 0 0

Linurs Servernest startpage