Wikipedia

exFAT (Extended File Allocation Table) is a Microsoft file system introduced in 2006 optimized for flash memory such as USB flash drives and SD cards.[5] It is proprietary and Microsoft owns patents on several elements of its design.[2]

exFAT can be used where the NTFS file system is not a feasible solution (due to data structure overhead), yet the file size limit of the standard FAT32 file system (i.e. 4 GiB) remains in those scenarios.

exFAT has been adopted by the SD Card Association as the default file system for SDXC cards larger than 32 GiB.

Contents

History

exFAT was first introduced in late 2006 as part of Windows CE 6.0, an embedded Windows operating system. Most of the vendors signing on for licenses of exFAT are either for embedded systems or device manufacturers that produce media that will be preformatted with exFAT. The entire File Allocation Table (FAT) family, exFAT included, is used for embedded systems because it is lightweight and is better suited for solutions that have low memory and low power requirements, and can be implemented in firmware.

Features

The specifications, features, and requirements of the exFAT file system include these:

exFAT allows individual files larger than 4 GiB, facilitating long continuous recording of HD video which can exceed the 4 GiB limit in less than an hour. Current digital cameras using FAT32 will break the video files into multiple segments of approximately 2 or 4 GiB. With the increase of capacity and the increase of data being transferred, the write operation needs to be made more efficient. SDXC cards, running at UHS-I have a minimum guaranteed write speed of 10 MBps and exFAT plays a factor in achieving that throughput through the reduction of the file system overhead in cluster allocation. This is achieved through the introduction of a cluster bitmap and elimination (or reduction) of writes to the FAT. A single bit in the directory record indicates that the file is contiguous, telling the exFAT driver to ignore the FAT. This optimization is analogous to an extent in other file systems, except that it only applies to whole files, as opposed to contiguous parts of files.

Adoption

exFAT is supported in Windows XP and Windows Server 2003 with update KB955704,[1] Windows Embedded CE 6.0, Windows Vista with Service Pack 1,[16] Windows Server 2008,[17] Windows 7, Windows 8, Windows Server 2008 R2 (except Windows Server 2008 Server Core), Windows 10, and Mac OS X starting from 10.6.5.[18][19]

Companies can integrate exFAT into a specific group of consumer devices, including cameras, camcorders, and digital photo frames for a flat fee. Mobile phones, PCs, and networks have a different volume pricing model.[5]

exFAT is supported in a number of media devices such as modern flat panel TVs,[20] media centers, and portable media players.[21]

Some vendors of flash media, including USB pen drives, compact flash (CF) and solid state drives (SSD) are shipping from the factory with some of their high capacity media pre-formatted with the exFAT file system. For example, Sandisk ships their 256 GB CF cards as exFAT.

Microsoft has entered into licensing agreements with BlackBerry,[22][23] Panasonic, Sanyo, Sony, Canon, Aspen Avionics,[24] Audiovox, Continental, Harman, LG Automotive[25] and BMW.[26]

Mac OS X Snow Leopard 10.6.5 and later can create, read, write, verify, and repair exFAT file systems.[18][19]

Reimplementations

A FUSE-based implementation named fuse-exfat, or exfat-fuse[3], with read/write support is available for FreeBSD and multiple Linux distributions.[27][28][29][30] A kernel implementation has also been released, written by Samsung.[31] It was initially released on GitHub unintentionally,[32] and later released officially by Samsung in compliance with the GPL.[33][4] None of the solutions can become an official part of Linux due to the patent encumbered status of the exFAT file system.[citation needed] An implementation called exFATFileSystem, based on fuse-exfat, is available for AmigaOS 4.1.[34]

Proprietary read/write solutions licensed and derived from the Microsoft exFAT implementation are available for Android,[35] Linux, and other operating systems from Paragon Software Group and Tuxera.

XCFiles (from Datalight) is a proprietary, full-featured implementation, intended to be portable to 32-bit systems.[36] Rtfs (from EBS Embedded Software) is a full-featured implementation for embedded devices.[37]

Two experimental, unofficial solutions are available for DOS. The loadable USBEXFAT driver requires Panasonic's USB stack for DOS and only works with USB storage devices; the open-source EXFAT executable is an exFAT file system reader, and requires the HX DOS extender to work.[38] There are no native exFAT real-mode DOS drivers, which would allow usage of, or booting from exFAT volumes.

Technical specialities

File name lookup

Like NTFS and HFS+, exFAT is a proprietary file system. Microsoft asserts that exFAT is covered by US Patent 8583708,[39] awarded on November 27, 2013, and US Patent 8321439,[40] Quick File Name Lookup Using Name Hash, which is an algorithm used in exFAT to speed up file searches. Microsoft had also applied for, and so far in some cases, received additional US patents on separate components that are used within exFAT. Since Microsoft has not officially released the entire exFAT specification, the Name Hash patent application is a key document in providing internal details in the understanding of the file system, since those details were revealed in Appendix A of the application.

File and cluster pre-allocation

Like NTFS, exFAT can pre-allocate disk space for a file by just marking arbitrary space on disk as 'allocated'. For each file, exFAT uses two separate 64-bit length fields in the directory: the Valid Data Length (VDL) which indicates the real size of the file, and the physical data length.

To provide improvement in the allocation of cluster storage for a new file, Microsoft incorporated a method to pre-allocate contiguous clusters and bypass the use of updating the FAT table and on December 10, 2013 the US patent office granted patent US8606830.[41] One feature of exFAT (used in the exFAT implementation within embedded systems) provides atomic transactions for the multiple steps of updating the file system metadata. The feature, called Transaction Safe FAT, or TexFAT, was granted a patent by the US patent office under US7613738 on November 3, 2009.[42]

Directory file set

exFAT and the rest of the FAT family of file systems does not use indexes for file names, unlike NTFS which uses B-trees for file searching. When a file is accessed, the directory must be sequentially searched until a match is found. For file names shorter than 16 characters in length, one file name record is required but the entire file is represented by three 32-byte directory records. This is called a directory file set, and a 256 MiB sub-directory can hold up to 2,796,202 file sets. (If files have longer names, this number will decrease but this is the maximum based on the minimum three-record file set.) To help improve the sequential searching of the directories (including the root) a hash value of the file name is derived for each file and stored in the directory record. When searching for a file, the file name is first converted to upper case using the upcase table (file names are case insensitive) and then hashed using a proprietary patented algorithm into a 16-bit (2 byte) hash value. Each record in the directory is searched by comparing the hash value. When a match is found, the file names are compared to ensure that the proper file was located in case of hash collisions. This improves performance because only 2 bytes have to be compared for each file. This significantly reduces the CPU cycles because most file names are more than 2 characters (bytes) in size and virtually every comparison is performed on only 2 bytes at a time until the intended file is located.

Metadata and checksums

exFAT introduces metadata integrity through the use of checksums. There are three checksums currently in use. The Volume Boot Record (VBR) is a 12 sector region that contains the boot records, BIOS Parameter Block (BPB), OEM parameters and the checksum sector. (There are two VBR type regions, the main VBR and the backup VBR.) The checksum sector is a checksum of the previous 11 sectors, with the exception of three bytes in the boot sector (Flags and percent used). This provides integrity of the VBR by determining if the VBR was modified. The most common cause could be a boot sector virus, but this would also catch any other corruption to the VBR. A second checksum is used for the upcase table. This is a static table and should never change. Any corruption in the table could prevent files from being located because this table is used to convert the filenames to upper case when searching to locate a file. The third checksum is in the directory file sets. Multiple directory records are used to define a single file and this is called a file set. This file set has metadata including the file name, time stamps, attributes, address of first cluster location of the data, file lengths, and the file name. A checksum is taken over the entire file set and a mismatch would occur if the directory file set was accidentally or maliciously changed. When the file system is mounted, and the integrity check is conducted, these hashes are verified. Mounting also includes comparison of the version of the exFAT file system by the driver to make sure the driver is compatible with the file system it is trying to mount, and to make sure that none of the required directory records are missing (for example, the directory record for the upcase table and Allocation Bitmap are required and the file system can't run if they are missing). If any of these checks fail, the file system should not be mounted, although in certain cases it may mount read-only.

The file system provides extensibility through template based metadata definitions using generic layouts and generic patterns.[43][44]

Other implementations

ExFAT is a proprietary file system and Microsoft has not released a specification for it. This lack of documentation hampered the development of free and open-source drivers for exFAT. Accordingly, exFAT support was effectively limited to Microsoft's own products and those of Microsoft's licensees, which in turn inhibited exFAT's adoption as universal exchange format.

Interoperability requires that certain results be achieved in a particular, predefined way (an algorithm). For exFAT, this potentially requires every implementation to perform certain procedures in exactly the same way as Microsoft's implementation. Some of the procedures used by Microsoft's implementation are patented, and these patents are owned by Microsoft.[45] A license to use these algorithms can be purchased from Microsoft,[46][45] and some companies – including Apple, Google and Samsung – have done so.[47] However, in the open-source ecosystem, there is no financial incentive for open-source vendors to purchase a license for these algorithms.

Regardless of whether open-source or not (including Samsung's leaked kernel driver source that was initially fraudulently rebadged as GPL-licensed),[48][49] according to Microsoft "a license is required in order to implement exFAT and use it in a product or device."[45] Unlicensed distribution of an exFAT driver would make the distributor liable for financial damages if the driver is found to have violated Microsoft's patents.[50][51] While the patents may not be enforceable, this can only be determined through a legal process, which is expensive and time consuming. It may also be possible to achieve the intended results without infringing Microsoft's patents.cf. [52]

The patents used by exFAT appears on a list of Microsoft's "Android-killer patents" published in June 2014 by the Government of China.[53]

See also

Notes

1. ^ a b Although Microsoft published a different value in KB955704, the file size is in bytes and is stored as a 64-bit number. The largest theoretical file size would be 16 EiB−1 byte, the same as in NTFS. However, since the true theoretical maximum volume size under the current specification cannot exceed 128 PiB, a file can never reach that file length. Another reason for the limit is that current IDE/ATA disk addressing is LBA-48, and uses a 48 bit block address to address a sector. A 512 byte sector size is represented by 29, which makes the maximum addressable file system 29 × 248 = 257, which is 128 PiB. In other words, the 128 PiB limit on the architecture is a hardware restriction. This scenario does not include AF (4k sectors) and exFAT is limited to 128 PiB regardless of sector size based on the specification.
2. ^ The theoretical maximum volume size is defined by up to 232 − 11 possible clusters with up to 225 − 1 bytes per cluster = ca. 128 PiB. The size is currently also limited by the LBA48 addressing scheme, as with a 512 byte sector size, only 248 × 512 = 257 bytes = 128 PiB can be addressed.
3. ^ This limit applies because the maximum directory size is 256 MiB.
4. ^ a b 268,304,373 files = 228 − 11 reserved clusters - 131,072, the minimum number of 64 kiB clusters occupied for the 268,435,445 directory entries (á 32 bytes) without VFAT LFNs, which are required for 268,435,445 files with sizes between 1 and 65,535 bytes. With VFATs, the 131,072 number must be multiplied by 21 (worst case), which would result in 265,682,933 files instead.

References

‏Source en.m.wikipedia.org