Metadata Workarounds on Tiger

Tiger introduced functions for file metadata in the BSD layer (see man setxattr for more info).

File metadata is something most programmers have wet dreams about, and naturally I started to use these new functions in TextMate 1.1b12. After this I got several reports of kernel panics when saving files to AFP mounted volumes (personal file sharing).

Yesterday I got the chance to play with an AFP mounted volume myself, and there are two problems with setxattr over AFP (but none on the other volume types I tested, including samba).

When OS X saves to a non HFS+ volume (which an AFP mounted volume apparently is), it can’t store the metadata in the actual file, so it creates a hidden file with a ._ prefix. E.g. if we save foo.txt it’ll store the metadata in ._foo.txt.

When this file is first created, it is given rw-r--r-- as file permission. However, if the metadata is updated, as it would be if we call setxattr again, the file permissions are left blank, meaning the file is no longer readable. In practice this means the metadata is no longer available to programs using getxattr.

So first workaround requires a call to chmod after setxattr (setting the file permissions correct for the hidden file).

The next problem is much more serious, it happens when updating an existing key/value pair. This is most likely a buffer overflow problem, because it seems to depend both on the call stack and the new and existing value for the key updated, but since the problem is in the kernel, the result is a kernel panic! yes, taking the entire system down and requiring a reboot!

But as mentioned, it’s only when updating a key/value pair, so if we remove the pair first (using removexattr) everything (according to my tests) works fine. Before calling removexattr we should however ensure that the file permissions are correct (since removexattr will fail if they aren’t, and e.g. copying a file from HFS+ to an AFP mounted volume may result in zeroed file permissions).

So rather than call setxattr directly, I have this NSString category:

@implementation NSString (xattr)
- (void)saveStringMetadata:(NSString*)aString forKey:(NSString*)aKey
{
    NSString* hiddenFile = [NSString stringWithFormat:@"%@/._%@", [self stringByDeletingLastPathComponent], [self lastPathComponent]];
    char const* path = [self UTF8String];
    char const* key = [aKey UTF8String];
 
    if([[NSFileManager defaultManager] fileExistsAtPath:hiddenFile])
    {
        chmod([hiddenFile UTF8String], S_IRUSR|S_IWUSR | S_IRGRP | S_IROTH);
        removexattr(path, key, 0);
    }
 
    char const* value = [aString UTF8String];
    setxattr(path, key, value, strlen(value), 0, 0);
 
    if([[NSFileManager defaultManager] fileExistsAtPath:hiddenFile])
        chmod([hiddenFile UTF8String], S_IRUSR|S_IWUSR | S_IRGRP | S_IROTH);
}
@end

And volla, it should now be safe to use metadata over AFP!

Though a problem I haven’t been able to solve is, if a files metadata is updated (regardless of volume), xcodebuild thinks the contents has also changed (and will trigger the file and its dependencies to be rebuilt).

I have checked the five dates stored in the FSCatalogInfo, but none of these are updated when calling setxattr. And in fact the problem is not just limited to setxattr, e.g. when changing color of a file in Finder, the file and all dependencies will be rebuilt next time you run xcodebuild.

If you have any idea how this can be prevented, please leave a comment or drop me a mail!