Thursday, November 19, 2009

Monotonic timers

Factor has had a calendar library for several years now. While it's great for converting timestamps to human-readable formats, calculating holidays, and finding the number of days between two dates, it's the wrong concept to use for timing code, alarms, and thread switching. In such cases where you don't need an actual date, you should use monotonic timers, which are counters that always increment from an unspecified time in the past and aren't affected by changes to the system time. Even if the user changes the clock, these monotonic timers don't go back in time -- they keep increasing. Let's look at the implementation.

Implementation of monotonic timers

Although I originally implemented monotonic timers as a Factor library, I moved the code into the C++ VM as a primitive called nano-count. To distinguish the usage of this word from the word formerly known as micros, I renamed micros to system-micros. Having the word "system" in the name of one time-returning word, and having "count" in the other, hopefully leads to less confusion on the user's part.

Windows

The code I came up with for Windows looks like this:
u64 nano_count()
{
static double scale_factor;

static u32 hi = 0;
static u32 lo = 0;

LARGE_INTEGER count;
BOOL ret = QueryPerformanceCounter(&count);
if(ret == 0)
fatal_error("QueryPerformanceCounter", 0);

if(scale_factor == 0.0)
{
LARGE_INTEGER frequency;
BOOL ret = QueryPerformanceFrequency(&frequency);
if(ret == 0)
fatal_error("QueryPerformanceFrequency", 0);
scale_factor = (1000000000.0 / frequency.QuadPart);
}

#ifdef FACTOR_64
hi = count.HighPart;
#else
/* On VirtualBox, QueryPerformanceCounter does not increment
the high part every time the low part overflows. Workaround. */
if(lo > count.LowPart)
hi++;
#endif
lo = count.LowPart;

return (u64)((((u64)hi << 32) | (u64)lo) * scale_factor);
}
It could probably be optimized by only calling QueryPerformanceFrequency once, but I don't set the processor affinity yet, so I'm not convinced it will work in every case. As you can see, it's pretty simple: the performance counter is queried and returns a number of clock cycles since some arbitrary beginning epoch, and then that time is scaled by the clock frequency to get nanoseconds.
Edit: This code contains a workaround for a VirtualBox counter bug.

Some Unix systems

u64 nano_count()
{
struct timespec t;
int ret;
ret = clock_gettime(CLOCK_MONOTONIC,&t);
if(ret != 0)
fatal_error("clock_gettime failed", 0);
return t.tv_sec * 1000000000 + t.tv_nsec;
}
Calling clock_gettime from the librt library or, on some platforms, as a system call, gives you the number of nanoseconds since an arbitrary start point in the past. The timespec struct has a seconds and a nanoseconds slots, while the timeval struct (used by system-micros) has seconds and microseconds.

Mac OSX

u64 nano_count()
{
u64 time = mach_absolute_time();

static u64 scaling_factor = 0;
if(!scaling_factor)
{
mach_timebase_info_data_t info;
kern_return_t ret = mach_timebase_info(&info);
if(ret != 0)
fatal_error("mach_timebase_info failed",ret);
scaling_factor = info.numer/info.denom;
}

return time * scaling_factor;
}
The MacOSX code is a bit different because Apple didn't implement clock_gettime. Instead, they have a couple of Mach functions that function just like the Windows code, with one returning a count and the other returning clock frequency information.

Upgraded alarms

The alarms vocabulary now uses monotonic timers instead of system time for scheduling alarms. Previously, the API for scheduling an alarm was the following, where passing f as the last input parameter would schedule a one-time alarm.
add-alarm ( quot start-timestamp interval-duration/f -- alarm )
However, this design is bad because the system time could change, resulting in a huge backlog of alarms to run. Also, most alarms were scheduled for less than a second into the future, which makes timestamps pretty useless since no date calculations are being performed. The new API takes a duration:
add-alarm ( quot start-duration interval-duration/f -- alarm)
Note that duration can be things like
  • 300 milliseconds
  • 5 seconds
  • 200 nanoseconds

Using monotonic timers

Mouse drag alarm

Here's an example of using an alarm from the mouse handling code:
: start-drag-timer ( -- )
hand-buttons get-global empty? [
[ drag-gesture ] 300 milliseconds 100 milliseconds
add-alarm drag-timer get-global >box
] when ;
The drag-gesture word gets called 300 milliseconds after a mouse button has been clicked, and again every 100 milliseconds afterwards until the alarm gets cancelled when the user releases a mouse button. The alarm is put into a global box because storing into a full box throws an error, which in this case would represent impossibility of the user dragging two things at once. Once dragging stops, the alarm gets cancelled with a call to cancel-alarm. You can look at the full source here.

Benchmark word

The benchmark word times a quotation and returns the number of nanoseconds that its execution took. Its implementation follows:
: benchmark ( quot -- runtime )
nano-count [ call nano-count ] dip - ; inline
This word simply gets the count from the monotonic timer, calls the quotation, gets a new count, and finds the elapsed time by subtraction.

Rescheduling alarms

After repeated alarms execute, they must be rescheduled to run again.
: reschedule-alarm ( alarm -- )
dup interval>> nano-count + >>start register-alarm ;
The alarm gets rescheduled interval>> nanoseconds into the future.

Remaining issues

Putting the computer to sleep on Snow Leopard in the middle of bootstrap and then resuming does not affect timing. However, is this the case with other operating systems such as Snow Vista or Linux? If not, it might not be worth worrying about. If someone wanted to test, just start a Factor bootstrap and then put the computer to sleep for awhile and see if bootstrap time increases. Otherwise, I'll get to it eventually.
Update: Someone on the Factor mailing list reported that putting the computer to sleep on bootstrap in Linux did not mess up the timing. Thank you!

Monday, August 17, 2009

Twin Cities Lisp Factor Presentation

At the next TC Lisp meeting, I'll be explaining as much as I can about Factor in a one-hour presentation. Arrive before 6pm for the happy hour beer prices. All are welcome!

Place: Common Roots Cafe
Date: August 18, 2009
Time: 6pm


View Larger Map

Friday, May 29, 2009

A new library to manage client connections, and a proof of concept chat server in Factor

Some thirty seconds into play-testing Joe Groff's terrain demo game, I realized there were no NPCs, no double-barreled shotguns or hand-cannons, and most disturbingly, no other players. Sure you can fly around, but you can't gib your friends! So I decided to do start to do something about it and write a managed server vocabulary with a generic protocol that you can extend for writing your own servers. Hey, I'm not much of an artist -- the guns will have to wait.

Managed-server infrastructure

The HTTP server already uses a library called io.servers.connections which implements a threaded-server with SSL support in 164 lines of code. A threaded-server listens for a set number of clients on an open port and handles each one individually; no client needs to know about any other. To get the code so concise, it uses libraries for concurrency, logging, sockets, and SSL that are themselves reused elsewhere.
Features of the threaded-server vocabulary:
  • the best nonblocking I/O code on every platform (completion ports, kqueue, epoll, not select())
  • connection/error logging, log rotation
  • correct error handling and resource cleanup
  • SSL support on Unix platforms
  • IPv4 and IPv6 support by default
For an HTTP or FTP server, handling each connection individually is what you want. However, for games or chat servers, you really want your users to interact. Building on top of this thread-server, I made a new tuple called a managed-server that tracks a list of connecting and disconnecting clients. You still get all of the features threaded-server implements, but now there's a new client handler that maintains a list of connected clients keyed by a username and utility words to send data to all clients.
You can also use this code to make custom binary protocols, and I'm mostly through implemented an SRP6 library to allow secure unencrypted logins after you create an account through an SSL connection. UDP support for first-person shooter and faster-paced games will also be supported when someone needs it.

The implementation of managed-server

A managed-server inherits from threaded-server class and adds a new slot called clients to store connections. Each connection's state -- the input/output streams, the local/remote socket addresses, username, a slot for passing quit messages, and a quit flag -- is wrapped inside a managed-client tuple and stored into the clients hashtable with the username as the key. In this way, it's easy to look up another client's stream and send it a message:
"wally" "hi wally!" send-client
You can also send a message to all connected clients,send-everyone, or to all but yourself:
"This one goes out to all the ladies." send-everyone-else
Here's what the tuple classes code looks like:
TUPLE: managed-server < threaded-server clients ;

TUPLE: managed-client
input-stream output-stream local-address remote-address
username object quit? ;

The managed-server protocol

A managed-server has some generics in place to guide you in creating your own servers. The first two generics are required, but the others default to no-ops unless you want to handle these events. Of course, the clients are still tracked no matter what your method does on the client-join or client-disconnect generics. The default method for handle-already-logged-in throws an error to prevent a new client from taking over the other client's session or logging in multiple times. You can override this behavior with your own perversions.
Here's the protocol:

HOOK: handle-login threaded-server ( -- username )
HOOK: handle-managed-client* managed-server ( -- )
HOOK: handle-already-logged-in managed-server ( -- )
HOOK: handle-client-join managed-server ( -- )
HOOK: handle-client-disconnect managed-server ( -- )

The implementation of a chat server using managed-server

Eventually someone will use managed-server for the networking code in a game, but until then I've implemented a simple chat server. Writing the chat server was fun and helped me to iron out a couple of bugs, which I wrote about below.

A walkthrough of the chat server protocol


The chat server code begins by inheriting from the managed-server tuple:
TUPLE: chat-server < managed-server ;
From here you go about implementing required parts of the protocol, handle-login and handle-managed-client*, so let's start there.
M: chat-server handle-login
"Username: " write flush
readln ;
The current input/output streams are bound to the client connection, so calling write will send them the login prompt. To read back the username, readln reads until a newline is sent. If you were to connect with telnet at this point, you would see the prompt and could send back a username. Then the server would kick you off because there's no implementation of handle-managed-client*.
M: chat-server handle-managed-client*
readln dup f = [ t client (>>quit?) ] when
[
"/" ?head [ handle-command ] [ handle-chat ] if
] unless-empty ;
This word handles every other message the client sends apart from the login code. Calling readln reads the client's message one line at a time and returns false when the stream closes. The quit flag is set in such a case and will be explained later. For now, suffice to say that you're quitting if readln returns false. Next, the message is checked for any content -- both false and an empty string can be safely ignored here by the unless-empty combinator. Inside the quotation, the leading slash is stripped from the input, if any, and a boolean returned by ?head decides if the message was intended for the server or the chat room.
: handle-command ( string -- )
dup " " split1 swap >lower commands get at* [
call( string -- ) drop
] [
2drop "Unknown command: " prepend print flush
] if ;
Commands sent to the server are normalized by converting to lower case and then looked up in the commands table. If you send a successful command such as /who or /nick then it gets executed; if not you get the generic "Unknown command" error.
: handle-chat ( string -- )
[
[ username ": " ] dip
] "" append-outputs-as send-everyone ;
Sending a message to the chat room is the alternative to server commands. I'm using append-outputs-as here to append together a bunch of strings, although i could easily have used 3append instead. I left this in because it's easier to change the look of the chat if you don't have to keep track of how many strings you're appending and you just let the compiler infer. Please take note: smart combinators in Factor are analogous to applying a function to a list or parameters in Lisp in that you don't need to know the number of parameters. The following two snippets will demonstrate what I mean:
(+ 1 2 10 1200)
[ 1 2 10 1200 ] sum-outputs
That's pretty much the essence of the chat server since everything else was just added for fun.

Fun with default encodings

Default encodings are terrible! Of course, you can change the encoding of a stream whenever you want, but the encoding for threaded-servers defaulted to ASCII until I changed it this evening. When I made my chat server yesterday, I forgot to set the encoding to what I wanted -- UTF8. Sending a character above 127 caused the server to throw an exception since ASCII is only 7 bits wide, and the sender would get disconnected. The FTP server I wrote started out with this bug as well, before I changed it to latin1. But now that threaded-server takes an encoding on the stack, this bug can never happen again.
So what's wrong with picking a different default encoding, maybe UTF8? Well, if I'm making a binary server, the UTF8 decoder will replace bad bit sequences with replacement characters -- another latent bug! What about binary as the default encoding, i.e. no encoding? Binary is the best option for a default, but then people who need to use UTF8 or latin1 might not know that the stream protocol supports encodings at all, and will end up doing a lot of work by hand which should be handled by the stream implementation. So not having a default encoding 1) prevents latent bugs and 2) forces the programmer to think about what they really want in each situation -- surely a good idea.

Quitting gracefully with quit flag

My first thought was just to throw an exception when I wanted to disconnect a client and cause the error handler to clean up the resources. Hopefully it's common knowledge that control flow implemented with exceptions is inefficient and bad design, in the general case. Maybe just this once? Nope, in my case the logging framework logs all exceptions, so the majority of the logs would be filled up with useless disconnect error messages. Clearly something better was needed -- the quit flag. Managed clients have a quit flag slot that is checked every time the server processes some data. Clients can quit gracefully by setting this flag to true and returning control back to the stream reader loop, and quits caused by exceptions are logged and worthy of further investigation.

Live coding on the server

After the chat server was up and running, I could add features without restarting. One of the first requested features was "/help", which required a redesign of how slash commands were handled. Instead of a case statement, now there's a word add-command that takes the implementation, the documentation, and the name of the command you want to add. Adding a command stores the code and the docs in symbols holding hashtables, indexed by the name of the command.
SYMBOL: commands
commands [ H{ } clone ] initialize

SYMBOL: chat-docs
chat-docs [ H{ } clone ] initialize

:: add-command ( quot docs key -- )
quot key commands get set-at
docs key chat-docs get set-at ;
I added a time command for fun:
[ drop gmt timestamp>rfc822 print flush ]
<" Syntax: /time
Returns the current GMT time."> "time" add-command
Someone else wanted a "/who" command -- easy enough.
[ drop clients keys [ "``" "''" surround ] map ", " join print flush ]
<" Syntax: /who
Shows the list of connected users.">
"who" add-command
There last feature I implemented was a way to change your nickname without reconnecting:
: handle-nick ( string -- )
[
"nick" usage
] [
dup clients key? [
username-taken-string print flush
] [
[ username swap warn-name-changed ]
[ username clients rename-at ]
[ client (>>username) ] tri
] if
] if-empty ;

[ handle-nick ]
<" Syntax: /nick nickname
Changes your nickname.">
"nick" add-command
Changing your nickname is straightforward but takes the most steps of all the commands I implemented. Try to understand the code -- "string" in the stack effect is the requested nickname and the clients word returns a hashtable of connections, indexed by nicknames. If the user didn't supply a nickname, remind them of the syntax for using /nick. If they did supply a nickname, check if it's in use and, if so, refuse to change their name. Otherwise, the nick change succeeded, so tell all the users of the nickname change, apply the nick change in the clients hashtable, and set the new nickname in the client.

Chat server running live

You can try out the chat server by downloading Factor and running this command:
USING: managed-server.chat io.servers.connection ; 8889 <chat-server> start-server
Or you can connect to my running chat server:
telnet trifocus.net 8889
It's just a demo and I didn't implement any limits on your nickname or what you can send, though it would be easy enough to do so. Have fun, and please let me know if you can find any bugs.

Sunday, May 24, 2009

An ID3 Parser in Factor

A new contributor, Tim Wawrzynczak, wrote an ID3 parser as his first Factor program a couple of months back. The code looked pretty good, so it was easy to refactor the way ID3v1 and ID3v2 tags are represented and to add some utility words for managing directories of MP3s. The library still needs some work, but now it can take a directory tree and recursively parse all of the ID3 headers. I also realized that Factor's mmap implementation always tried to map a file read/write, so for ID3 parsing I added a read-only mmap. The finished code is here.

ID3v1 format


ID3 tags come in two flavors -- the old ID3v1 format, and the newer ID3v2 one. ID3v1 has a fixed length of 128 bytes and, if present, is the last 128 bytes of an MP3 file. The bytes begin with "TAG" and follow with the song title (30 bytes), artist (30 bytes), album (30 bytes), year (4 bytes), comment (30 bytes), and the genre (1 byte). The problem with this approach is that you are limited in the length of all the fields and in which data you can represent.

To implement the parser, we use the Memory-mapped files vocabulary to open the file. You can treat the file as an array of bytes that obeys the sequence protocol. Checking if a file has ID3v1 headers becomes:
: id3v1? ( mmap -- ? )
{ [ length 128 >= ] [ 128 tail-slice* "TAG" head? ] } 1&& ;
The logic here is straightforward: the sequence (mmapped file) is checked to make sure it's long enough to contain a header, and then the last 128 bytes are checked against the magic bytes "TAG". Since this is a short-circuit combinator, if the length test fails then the computation will end early. In this way, you can string together long computations that use short-circuit and and or. Factor's usual and/or words take two already-evaluated objects, so the short-circuit behavior is implemented as a library of macros instead.

ID3v2 format


The newer standard, ID3v2, has more room for metadata and can store anything up to 256 MB. Its use is indicated when the MP3 begins with "ID3" or when the bytes "3DI" are present 10 bytes from the end of the file or 10 bytes before the ID3v1 header. For parsing the header, the first two bytes are the version, then a flag byte, and finally four bytes for the size of the header. The size bytes are encoded as a synchsafe number, which means that the top bit is discarded and the lower seven bits are the data. In our case, 28 bits of data are the length, which is why the maximum length is 256MB.

The actual data we want to parse is stored in frames. Each frame has a tag, which is four ascii or utf16 bytes, followed by another 4 byte synchsafe integer for the length, two bytes of flag data, and the frame data. There are many different types of frames, but some of the more common ones are tagged TALB (album title), TIT2 (title), TRCK (track), COMM (comment), TPE1 (performer). All of the frame are added to a hashtable and keyed by the tag. Looking up the title frame becomes as easy as "TIT2" find-id3-frame on an ID3 tuple.

Future work


The ID3v1 "TAG+" format is not supported yet and apparently it's hardly ever used. ID3v2 tags are only looked for at the beginning of an MP3 but may be present at the end too as of version four. Most of the ID3 tags are not parsed into meaningful data besides the raw bytes. Lastly, writing out ID3 tags is not implemented yet and would be a good first program to write in Factor. Volunteers?

Tuesday, January 13, 2009

Files and file-systems in Factor, part 2

In my previous post I wrote about the file-info and file-system-info words. Using those words, it's possible to write a portable program for directory listing (dir or ls) and one for file-systems listing (df). Uses for directory listing include file-system utility programs and FTP servers.

File listing tool

The file-listing tool is in the Factor git repository as basis/tools/files/files.factor. Which slots you see depend on how it is configured and on which platform it's running. Directory listings can be sorted by slot and the default sort is by name.

On Unix platforms, it's configured like this:
    <listing-tool>
{ permissions nlinks user group file-size file-date file-name } >>specs
{ { directory-entry>> name>> <=> } } >>sort
Listing a directory on MacOSX:
-rw-r--r-- 1  erg staff 27185 Nov 28  2008 #factor.el#
drwxr-xr-x 6 erg staff 204 Nov 17 2008 Factor.tmbundle
-rw-r--r-- 1 erg staff 30282 Nov 30 2008 factor.el
-rw-r--r-- 1 erg staff 18037 Nov 17 2008 factor.vim
-rw-r--r-- 1 erg staff 12496 Nov 17 2008 factor.vim.fgen
drwxr-xr-x 26 erg staff 884 Jan 13 11:17 fuel
drwxr-xr-x 8 erg staff 272 Nov 17 2008 icons
Windows platforms take the look of the ``dir'' command by default:
2009-01-14 00:00:53 <DIR>                Factor.tmbundle
2009-01-14 00:00:53 30282 factor.el
2009-01-14 00:00:53 18037 factor.vim
2009-01-14 00:00:53 12496 factor.vim.fgen
2009-01-14 00:00:53 <DIR> fuel
2009-01-14 00:00:53 <DIR> icons

File-systems tool

A tool for disk usage and mounted devices was easy to write once file-system tuples worked everywhere. Once again, it's configurable for whatever you want to see. The default file-system word is:
: file-systems. ( -- )
{
device-name available-space free-space used-space
total-space percent-used mount-point
} print-file-systems ;
File-systems on a Mac:
device-name   available-space free-space   used-space   total-space  percent-used mount-point
/dev/disk0s2 118599725056 118861869056 200867090432 319728959488 62 /
fdesc 0 0 1024 1024 100 /dev
fdesc 0 0 1024 1024 100 /dev
map -hosts 0 0 0 0 0 /net
map auto_home 0 0 0 0 0 /home

On Windows:
device-name      available-space free-space used-space total-space percent-used mount-point
3814506496 3814506496 6911225856 10725732352 64 C:\
VBOXADDITIONS_2. 0 0 27983872 27983872 100 D:\
0 0 0 0 0 A:\

Friday, January 09, 2009

Files and file-systems in Factor, part 1

Factor now has an easy way access to get information about files and file-systems in a high-level way across all the platforms that it supports. The API is really simple -- pass a pathname and get information back about the file or file-system as a tuple. The second part of this post will demonstrate a clone of the Unix tools ls, for listing files, and df, for listing file-systems.

File-info

There are now words to get information about files and symlinks, using file-info and link-info which map to C system calls stat and lstat. Some slots are shared across all platforms while others are only present on a particular platform. There are symbols representing all of the file types, like +regular-file+, +directory+, and +symbolic-link+. Here are some examples.
MacOSX
( scratchpad ) "resource:license.txt" file-info .
T{ bsd-file-info
{ type +regular-file+ }
{ size 1252 }
{ permissions 33188 }
{ created
T{ timestamp
{ year 2008 }
{ month 11 }
{ day 17 }
{ hour 23 }
{ minute 34 }
{ second 5 }
{ gmt-offset T{ duration { hour -6 } } }
}
}
{ modified
T{ timestamp
{ year 2008 }
{ month 11 }
{ day 17 }
{ hour 23 }
{ minute 34 }
{ second 5 }
{ gmt-offset T{ duration { hour -6 } } }
}
}
{ accessed
T{ timestamp
{ year 2008 }
{ month 12 }
{ day 9 }
{ hour 12 }
{ minute 34 }
{ second 8 }
{ gmt-offset T{ duration { hour -6 } } }
}
}
{ uid 501 }
{ gid 20 }
{ dev 234881026 }
{ ino 992362 }
{ nlink 1 }
{ rdev 0 }
{ blocks 8 }
{ blocksize 4096 }
{ birth-time
T{ timestamp
{ year 2008 }
{ month 11 }
{ day 17 }
{ hour 23 }
{ minute 34 }
{ second 5 }
{ gmt-offset T{ duration { hour -6 } } }
}
}
{ flags 0 }
{ gen 0 }
}
Windows XP
( scratchpad ) "resource:license.txt" file-info .
T{ windows-file-info
{ type +regular-file+ }
{ size 1252 }
{ permissions 32 }
{ created
T{ timestamp
{ year 2008 }
{ month 3 }
{ day 23 }
{ hour 23 }
{ minute 28 }
{ second 12 }
}
}
{ modified
T{ timestamp
{ year 2008 }
{ month 3 }
{ day 27 }
{ hour 23 }
{ minute 24 }
{ second 12 }
}
}
{ accessed
T{ timestamp
{ year 2008 }
{ month 9 }
{ day 19 }
{ hour 23 }
{ minute 8 }
{ second 41 }
}
}
{ attributes { +archive+ } }
}
FreeBSD
( scratchpad ) "resource:license.txt" file-info .
T{ bsd-file-info
{ type +regular-file+ }
{ size 1252 }
{ permissions 33188 }
{ created
T{ timestamp
{ year 2008 }
{ month 4 }
{ day 6 }
{ hour 12 }
{ minute 6 }
{ second 53 }
{ gmt-offset T{ duration { hour -6 } } }
}
}
{ modified
T{ timestamp
{ year 2008 }
{ month 4 }
{ day 6 }
{ hour 12 }
{ minute 6 }
{ second 53 }
{ gmt-offset T{ duration { hour -6 } } }
}
}
{ accessed
T{ timestamp
{ year 2008 }
{ month 4 }
{ day 6 }
{ hour 12 }
{ minute 6 }
{ second 59 }
{ gmt-offset T{ duration { hour -6 } } }
}
}
{ uid 1002 }
{ gid 1002 }
{ dev 89 }
{ ino 343452 }
{ nlink 1 }
{ rdev 1348575 }
{ blocks 4 }
{ blocksize 4096 }
{ birth-time
T{ timestamp
{ year 2008 }
{ month 4 }
{ day 6 }
{ hour 12 }
{ minute 6 }
{ second 53 }
{ gmt-offset T{ duration { hour -6 } } }
}
}
{ flags 0 }
{ gen 0 }
}

File Systems

The file-system utility word above works on file-system tuples that contain cross-platform information like the device name, the mount point, the number of free, used, and total bytes. A file-system tuple on Unix has all of the file-system information found in both statfs and statvfs while a Windows file-system object has the device-id, volume name, and byte usage slots. There is not a single win32 API call that gives as much information as on Unix systems -- instead I call a combination of GetDiskFreeSpaceEx, FindFirstVolume, GetVolumePathNamesForVolumeName, GetVolumeInformation.
On every Unix besides Linux, there is a member of the statfs or statvfs structure that gives you the file-system that contains the file. So, I had to roll my own for it to work the same way across all platforms. The algorithm is pretty simple: the follow-links word follows links up to 10 times (configurable) and once it stops, finds the parent directory and follows the links again until the directory is a member of the directories in the /etc/mtab file. If there is circularity or a broken link, it throws an error.

FreeBSD
( scratchpad ) "/" file-system-info .
T{ freebsd-file-system-info
{ device-name "/dev/da0s1a" }
{ mount-point "/" }
{ type "ufs" }
{ available-space 368117760 }
{ free-space 409702400 }
{ used-space 110110720 }
{ total-space 519813120 }
{ block-size 2048 }
{ preferred-block-size 2048 }
{ blocks 253815 }
{ blocks-free 200050 }
{ blocks-available 179745 }
{ files 65790 }
{ files-free 61501 }
{ files-available 61501 }
{ name-max 255 }
{ flags 20480 }
{ id { 0 0 } }
{ version 537068824 }
{ io-size 16384 }
{ owner 0 }
{ syncreads 0 }
{ syncwrites 0 }
{ asyncreads 0 }
{ asyncwrites 0 }
}
Windows XP 64
( scratchpad ) "k:\\" file-system-info .
T{ win32-file-system-info
{ device-name "" }
{ mount-point "k:\\" }
{ type "NTFS" }
{ available-space 174530142208 }
{ free-space 174530142208 }
{ used-space 225547444224 }
{ total-space 400077586432 }
{ max-component 255 }
{ flags 459007 }
{ device-serial 3695676537 }
}
The Factor build farm will start using file-system-info to report when a drive fills up pretty soon.

Sorting by tuple slots

Factor's sorting has been extended to support sorting tuples by multiple slots, one after the other. In a music player application, you may wish to sort your songs first by artist, then by album, then track number. If you had a tuple representing every song, the code for such a sort is now very easy:
{ { artist>> <=> } { album>> <=> } { track>> <=> } } sort-by-slots
The main word at work is sort-by-slots ( seq sort-specs -- seq' ), where a sort-spec is an accessor paired with a comparator, one of <=>, >=<, human-<=>, human->=<. Human sort first converts consecutive digits to integers and then makes the comparison, e.g. { "a1" "a10" "a03" "a2" } sorts as { "a1" "a2" "a03" "a10" } instead of { "a03" "a1" "a10" "a2" }, as it would with natural-sort.

Sorting in reverse order is possible with the new operator >=<, which inverts the result of the <=> comparator. To sort a playlist by the most played songs in reverse order (most played first):
{ { play-count>> >=< } } sort-by-slots

Source code for sort-by-slots

: slot-comparator ( accessor comparator -- quot )
'[ [ _ execute ] bi@ _ execute dup +eq+ eq? [ drop f ] when ] ;

MACRO: compare-slots ( sort-specs -- <=> )
#! sort-spec: { accessor comparator }
[ first2 slot-comparator ] map '[ _ 2|| +eq+ or ] ;

: sort-by-slots ( seq sort-specs -- seq' )
'[ _ compare-slots ] sort ;
First, a bit about how Factor's sort ( seq quot -- sortedseq ) word works. It expects a quotation that compares two objects lexicographically, which is element by element in dictionary order. Thus, the macro compare-slots expands the sort-specs into a quotation that compares tuples slot-by-slot until there is a difference. The macro-expansion for the first example looks like this:
( scratchpad ) [ { { artist>> <=> } { album>> <=> } { track>> <=> } } compare-slots ] expand-macros .
f 2 [
\ artist>> \ <=> [ [ execute ] curry bi@ ] dip execute
dup +eq+ eq? [ drop f ] when
] [ [ drop ] dip ndup ] dip call dup
[ 2 [ ndrop ] curry dip ] [
2 [
\ album>> \ <=> [ [ execute ] curry bi@ ] dip execute
dup +eq+ eq? [ drop f ] when
] [ [ drop ] dip ndup ] dip call dup
[ 2 [ ndrop ] curry dip ] [
2 [
\ track>> \ <=> [ [ execute ] curry bi@ ] dip
execute dup +eq+ eq? [ drop f ] when
] [ [ drop ] dip ndup ] dip call dup
[ 2 [ ndrop ] curry dip ]
[ 2 [ drop ] dip ndrop t [ f ] [ no-cond ] if ] if
] if
] if +eq+ or

The macro first extracts the slots and compares with the first comparator. If the comparator returns +lt+ or +gt+ then the comparison is over, but if the comparator returns +eq+ then the next comparison is invoked and the sort continues in the same way. Notice that the slot-comparator word replaces +eq+ with f to avoid short-circuiting the iteration done by 2||. The final sort-by-slots word is a trivial call to sort with the new comparator word we just defined.

Algorithmic complexity and ideas for improvement

The sorting bound is still O(n log n) for time, but of course the more comparators that you need to break ties, the longer the algorithm will take to run.
If your data has to be sorted anyway, it's possible to sort and then split the data into related slices using a sequence of accessors like the above. You could then compute the playing time of every album, or find your most-played song from every artist. Splitting with the same accessors takes O(n) time, but the odds are that you probably won't have already sorted the data. So for unsorted data, there is a more efficient way to extract features -- you can instead partition it in O(n) time and find features on each partition, avoiding sorting altogether. This will be the subject of a future blog post.