Saturday, December 30, 2006

Serialization To Disk

For fun, I wrote a TinyURL clone in Factor. You can try it out at http://wee-url.com. I needed a way to save the data to a file, so I used libs/serialize to serialize the hashtable containing the links. There was yet another bug involving vector literals -- the serialization code would reuse a vector, so the first time you serialized the output is correct, but the next time it would produce garbage.
  scratchpad SYMBOL: aa H{ { 1 2 } } aa set
scratchpad aa get .
H{ { 1 2 } }
scratchpad aa get [ [ serialize ] with-serialized ] string-out .
"hp\0\0\0\u0001Dap\0\0\0\u0001Ep\0\0\0\u0001\u0001ap\0\0..."
scratchpad aa get [ [ serialize ] with-serialized ] string-out .
"op\0\0\0\u0001:"
The fix was to change a V{ } to a V{ } clone.


Serialization to Disk Code

: save-fstore ( variable path -- )
[ [ get serialize ] with-serialized ] with-stream ;

: load-fstore ( variable path -- )
dup exists? [
[ [ deserialize ] with-serialized ] with-stream
swap
] [
drop >r H{ } clone r>
] if set-global ;

: fstore-set ( variable fstore -- )
get >r [ get ] keep r> set-hash ;

: fstore-get ( default variable fstore -- )
get dupd hash* [ swap set-global drop ] [ drop set-global ] if ;


save-fstore and load-fstore save/load a hashtable to disk. The hashtable keys are variable names and the values are the variable values. fstore-set puts your variable in the hashtable, so you should call this directly before saving to disk. fstore-get loads the data from your store. If there is no fstore or if the key does not exist, it uses your default value.

The loading code is pretty simple:
: fstore-name "wee-url.fstore" ;
SYMBOL: wee-shortcuts
SYMBOL: wee-fstore
wee-fstore fstore-name load-fstore
H{ } clone wee-shortcuts wee-fstore fstore-get
The variable wee-shortcuts stores the encoded URLs, while wee-fstore is the hashtable that the above code reads/writes to disk.

Finally, to write the store:
: save-shortcuts ( -- )
wee-shortcuts wee-fstore fstore-set
wee-fstore fstore-name save-fstore ;

Notice that we add wee-shortcuts to the hashtable before saving to disk.

I'm going to rewrite the wee-url code to use Furnace instead of callback-responder. Once I get it looking nice, I'll show you how it works.

Thursday, November 23, 2006

Compiling Factor on the Intel Mac

To compile Factor for Mac you will need:
  • Installation DVDs that came with your Mac (for X11)
  • darwin-ports
  1. Run the installation disk that came with your Mac in order to install X11. Since Apple does not offer X11 as a download for OS X 10.4 for Intel computers, you have to install from the DVD.
  2. Download the darcs binary and place it in /usr/local/bin
  3. Open terminal (Finder->Applications->Utilities->Terminal) and go to your home directory. To get the Factor source code, run:
    darcs get http://factorcode.org/repos
  4. To compile, move into the directory darcs created with cd repos/Factor and run the command make macosx-x86
  5. Download the boot image at http://factorcode.org/images/latest/boot.image.pentium4
  6. ./f boot.image.pentium4
  7. make macosx.app
  8. (Optional) Create a shell script to start Factor in your terminal window. I called mine go.
    #!/bin/sh
    DIR=~/repos/Factor/Factor.app/Contents
    $DIR/MacOS/Factor $@

    Save the file and run chmod a+x go
  9. (Optional) You can instead run open Factor.app and use Console.app to view runtime output.
To start the UI, run ./go or open Factor.app; run ./f to start Factor in the shell. If you run Factor in the terminal, I suggest using rlwrap, which adds a history buffer

Now you can code anything!

Thursday, November 16, 2006

Bootstrapping, reset-modified, and F8

Factor's module system has the ability to reload changed files from the UI, in the correct order, by calling reload-modules, which is bound to F8. When you download an image someone else created, the timestamps won't match up and when you reload the modules it will try to reload all of the Factor library. You don't want this.

There are several ways to fix the problem. First, you could generate a new image USE: image "x86" make-image and bootstrap again. Another way is to call reset-modified and then save the image. However, until you bootstrap from an image that you made yourself, will have the same problem.

If you make a module, don't forget to PROVIDE: contrib/my-module ; at the very least, or else F8 will not reload the changes you make after requireing it.

Wednesday, November 15, 2006

Using .factor-boot-rc and .factor-rc

Update: I changed the syntax to work with the new directory layout.

The two resource files Factor tries to run, if they exist, are .factor-boot-rc and .factor-rc, at bootstrap and run-time respectively. You have to create these files yourself and place them in your home directory, e.g. the directory returned by the home word.

If I used Textmate on a Mac, my .factor-boot-rc would look like this:
USING: modules namespaces ;
REQUIRES: libs/textmate ;

Here is the boot file for gvim in Windows XP:
USING: modules namespaces ;
REQUIRES: libs/vim ;
USE: vim
"c:\\program files\\vim\\vim70\\gvim.exe" vim-path set-global


Now for loading libraries. Some of the modules in contrib/ require that you load your library before running. Here are three examples of a .factor-rc that loads libraries.

Loading PostgreSQL on a Mac:
USING: alien ;
"postgresql" "/opt/local/lib/postgresql81/libpq.dylib" "cdecl" add-library

PostgreSQL and SQLite on Windows:
USING: alien ;
"postgresql" "C:\\Program Files\\PostgreSQL\\8.1\\bin\\libpq.dll"
"cdecl" add-library
"sqlite" "C:\\Program Files\\sqlite\\sqlite3.dll"
"cdecl" add-library
SQLite on Linux:
USING: alien ;
"sqlite" "libsqlite3.so" "cdecl" add-library

Tuesday, November 14, 2006

Compiling Factor on Windows

To compile Factor you will need:

  • MinGW
  • MSYS
  • darcs
  • freetype6.dll
  • zlib1.dll


  1. Download MinGW and install it.


  2. Download MSYS and install it. Make sure to tell the batch script about your MinGW install.


  3. Download the latest package version of darcs and unzip it to c:\darcsdir-w32. darcs 1.0.7 is broken on Windows, so you must also grab the 1.0.8 binary and copy it into your darcs install directory over the broken version.


  4. Set your path to include darcs. Run:
    start->Control Panel->System->Advanced->Environment Variables

    In System Variables, edit PATH, and append c:\darcsdir-w32 so that your path resembles:
    c:\Program Files\sqlite;c:\darcsdir-w32

    Start cmd.exe from the Run box and try out your darcs installation:
    C:\Documents and Settings\Administrator>darcs --version
    1.0.8 (release)


  5. (Optional) To push patches to a remote server, you will need to generate an RSA key with puttygen unless you already have a public key. Run puttygen (installed with the darcs bundle), click generate, and move the mouse around to make your key. Save both your public and private keys in c:\darcsdir-w32. Adding a passphrase is optional (I didn't do it).


  6. (Optional) On your remote Linux box running sshd, edit the file ~/.ssh/authorized_keys2 and paste your public key so that it looks something like this (but all on one line):

    ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIEAgMPa1vvlSi4ZrS7hrk/XJNwd9trM+TyvJjhkMzf3fhks
    n8TFin7dhrRfFhVdjka0ep0+RGJrUI/ja6nDyG2SvzG1ka5Ml/Nes2PtpHWcwK7fwU9gBUZNzlvXCkOk
    UKO7X7N65LnyCPjJDZa+8LPqrBeESjbVmstG4cw7uh0= doug@windows


  7. (Optional) So that darcs can find your key, start Regedit and add a key for Pageant:
    HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run
    Name: Pageant
    Data: c:\darcsdir-w32\pageant.exe c:\darcsdir-w32\private.ppk

    When you reboot or run the above command, pageant will find your key and you can begin using darcs to push patches to your Linux box.


  8. To get the Factor source, run this command wherever you want your local copy, perhaps from c:\:
    darcs get http://factorcode.org/repos


  9. Additionally, put these dlls in your Factor directory (repos/Factor/ on a default checkout):
    freetype6.dll
    zlib1.dll


  10. Start MSYS (the blue M link) and change to your Factor directory. Mine is at c:\repos\Factor, so the command is:
    cd /c/repos/Factor


  11. From here you should be able to
    make windows


  12. From cmd.exe, change to the Factor directory and bootstrap with an image (x86) from factorcode.org:
    f -i=boot.image.x86


Now you can run Factor!

f

I recommend bootstrapping and running Factor from cmd.exe because MSYS messes up stderr--you will not notice for a while if Factor has an internal error because there will be no output.

As a final note, the latest version from darcs is usually the best. If the boot image of Factor is stale and bootstrap fails, contact one of the devs in #concatenative on irc.freenode.org.

Happy Windows Factoring!

Update: Troubleshooting

Q: When starting Factor, why do I get the error 'alien-invoke-error-library "freetype", alien-invoke-error-symbol "FT_Init_FreeType"'?

A: You forgot to put freetype6.dll and zlib1.dll in the Factor directory. Download these and bootstrap again.

Q: When I bootstrap, why does Factor spew "*** Data GC (5 minor, 24 cards)" and error out with 'Word not found in current vocabulary search path delegate f no-word-name "\u000c"'?

A: On bootstrap you must pass the '-i' command line argument. The correct command line is:

./f -i=boot.image.x86

Factor 0.86 released

Factor 0.86 has been released. We took about a week to test it before releasing this time, and it promises to be the most stable release ever. Factor for Windows has a few improvements too, as a long-standing bug with the native i/o has been fixed, allowing Factor to be run in the console while still supporting native i/o with the command:
f -shell=tty
Furthermore, in 0.85, Windows Factor took several minutes to slurp through a 600 MB file; the new version takes five seconds and will speed up again when continuations compile.