Jun 23
Extended file attributes on your filesystem, how to enable and use them
Many modern filesystems provide the user with a facility to store arbitrary key/value-pairs along with files, so-called metadata.
Wikipedia's definition:
Extended file attributes is a file system feature that enables users to associate computer files with metadata not interpreted by the filesystem, whereas regular attributes have a purpose strictly defined by the filesystem (such as permissions or records of creation and modification times). Unlike forks, which can usually be as large as the maximum file size, extended attributes are usually limited in size to a value significantly smaller than the maximum file size. Typical uses can be storing the author of a document, the character encoding of a plain-text document, or a checksum.
Refer to the complete article for an investigation how different operating systems deal with them.
Wikipedia also provides a helpful file system comparison table.
Enabling extended attributes on Linux
Go to /etc/fstab and add "user_xattr" to the options section of the line regarding the file-system you'd like to enable extended attributes on. Such a line might look like:
/dev/sda1 / ext4 errors=remount-ro,user_xattr 0 1
Using extended file attributes
As XFS on Irix was one of the first filesystems to support extended attributes, there is a set of "legacy commands" to set/get xattribs:
xattr
getxattr - lgetxattr, fgetxattr
setxattr - lgetxattr, fgetxattr
listxattr - llistxattr, flistxattr
removexattr - lremovexattr, fremovexattr
To use xattribs on Linux, you need to install the attr tool with sudo apt-get install attr. The commands then are:
Usage: attr [-LRSq] -s attrname [-V attrvalue] pathname # set value attr [-LRSq] -g attrname pathname # get value attr [-LRSq] -r attrname pathname # remove attr attr [-LRq] -l pathname # list attrs -s reads a value from stdin and -g writes a value to stdout
If you mount NTFS, which does support extended attributes, under Linux, I can assure you that the ntfs-3g driver properly supports xattribs, trough the unix commands and for examples on moves, even between different filesystems. So you can rely on xattribs remaining intact, for example when you move a file between ext3 and NTFS.
It might be helpful to point out that each of the above commands behaves a bit differerntly. attr does automatically prepend the "user." part in fron of attribute names, so if you would set "user.id" it would actually write the "user.user.id" attribute. And after operations it returns the number of bytes written. The getfattr and setfattr commands do not prepend the "user." bit and expect you to do it. And they also don't return the amount of data written. So make sure you test each command and what it expects, as these APIs might differ from platform to platform and might change over time.
Sidecar files, etc.
Despite Apple's HFS supports xattr, you sometimes see the .DS_Store file, a file which stores metadata about directories. (Wikipedia)
Perl
Working with Perl, some modules from cpan help you wrangling xattribs:
File::ExtAttr used by File::Attributes::Extended, for example. Seems quite mature.
File::Attributes tries to provide a unique API for xattribs on various systems and filesystems. If File::Attributes::Extended is installed, it uses it to store "real" xattribs, otherwise it falls back to Fie::Attribute::Simple which stores extended attributes in sidecar files named with filename plus the .attributes suffix. It uses this sidecar files also on filesystems which don't allow xattribs, like FAT.
Linux::UserXAttr "This module implements a very thin layer around the extended attributes syscalls on Linux".
Note: Installing File::ExtAttr might throw the error "Can't build and link to 'attr'". In this case you need to install the attr devel package, which is named libattr-devel or libattr1-dev on Ubuntu.
CLI tools support
Be aware that common unix command line tools have varying behaviour regarding xattribs:
- cp by default will not copy over xattribs to new/copied files. You need to add the -a (archive) flag.
- mv does copy over xaatribs, even across file system boundaries.