Building libraries/ modules with Chicken-Scheme
« Wednesday, November 06, 2013 »

Chicken Scheme

After working with Scheme for some times, I really felt that I was missing modules from python. In python, it's quite simple a file is a module. There are also native module with a special extension and folder as module too using the __init__ syntax. Modules have there own global scope which means anything defined in a module isn't global for every modules explicitely. Generally a function/variable is isolated in its own module. In Scheme, this is a whole different thing.

Libraries and Modules

I'd say that the state of Scheme in regards of Libraries and Modules is worse and better than the situation of Javascript libraries. Recently the scheme workgroup released the new revision of Scheme which includes define-library. This is a really good news but it's not exactly all that great. A library will be the closest thing possible to a python module. It will be mostly the exact same thing as a python native module.

A native python module can be compiled using multiple source files. All the sources are then compiled into a shared object that can get imported by Python. In Scheme, a library can include multiple files and compile them into a single library just like in python. But I'll write more about it later.

The new define-library macro is a great thing but Scheme has still something missing. Unlike javascript, scheme has a rock solid definition for libraries/modules but lacks a good package manager. We could expect to see a lot more modules in the future of r7rs, but there is no real good way to publish modules to the world. Most implementations which have modules also have their own package manager. For that reason, the Scheme world is quite scattered and it might be hard to find a rock solid package that will just work. I often read this answer on how to chose a scheme implementation: “Check which implementation has the libraries you need and stick with it”. Since most libraries aren't always “cross scheme”, you might get stuck with Racket because it's the only Scheme that offers the module you need and it doesn't work on other scheme because it depends on Racket.

Designing a library in Scheme

Let's write our first module for Chicken-Scheme. We'll start with a simple hello world. The following file should be named hello.scm:

1
2
3
4
5
6
(module hello (greet)
  (import chicken)
  (import scheme)

  (define (greet)
    (print "hello world")))

If it's not clear, the first parameter is the name of the module. The second parameter is a list of exported symbols. Then we have to import some language definitions. Importing chicken and scheme should be enough to get our module working.

Now if we open a REPL in the same directory as the hello.scm and try to impor the module. We will see the following error:

Error: (import) during expansion of (import ...) - cannot import from undefined module: hello

Chicken-Scheme detects modules following the convetions that I explained above. Renaming the hello.scm to hello.import.scm should be enough. Trying again should yield something like this:

$ csi
#;1> (import hello)
; loading ./hello.import.scm ...
#;2> (greet)
hello world

Our module works and is loaded and interpreted. But we can compile our library like this:

$ csc -library test.import.scm
$ csi
#;1> (import hello)
; loading ./hello.import.so ...
#;2> (greet)
hello world

Notice that the file that is being loaded here is hello.import.so instead of
hello.import.scm. The native libraries have priority against interpreted code.
Now that you understand the basics about module definition, you might ask yourself
this following question: “Isn't .import.scm quite ugly and long to write to
tell the interpreter that the library is infact a scheme library?”. If that's the
case, you aren't alone and I might say that the people working on chibi-scheme
did find a way to make it look nice. The are using the following convention:

  • .scm is scheme code
  • .scd is a library

This is were the analogy with native python modules should start to make sense. When
writing a library in Scheme, we could separate the implementation from the module
definition. What is really important is to have a module definition that exports
symbols and files that contains code.

We could rewrite our hello library using two files.

hello.scm

1
2
(define (greet)
  (print "hello world"))

hello.scd

1
2
3
4
5
(module hello (greet)
  (import chicken)
  (import scheme)

  (include "hello.scm"))

In the code above, we can see that the code is explicitely including some sources
into the libraries code. This should be enough to join multiple scheme code into
one big library. We can include multiple sources into our library and the compiler
should be able to resolves those files at compile time.

The following commands should compile the above module correctly:

csc -J hello.scd
csc -library hello.scd
csc -library hello.import.scm

You'll end up with those libraries: “hello.so” and “hello.import.so”. Sounds good?
Well not yet. Remember? we used (import hello) to load our library in csi and it
worked. Now let's try again.

csi
#;1> (import hello)
; loading ./hello.import.so ...
#;2> (greet)

Error: unbound variable: hello#greet

    Call history:

    <syntax>      (greet)
    <eval>    (greet)   <--

We can see that the import file is getting loaded but greet is unbound. Actually,
the problem is that the when we imported the file directly using import. It was defining
the library and everything at once. By compiling hello.scd with the -J parameter,
we are only defining an import file that will corectly define the exported symbols but it
won't actually import code. The truth is that Chicken Scheme is handling module loading in
two steps.

We have to require the library and the imports. First, we could do this:

csi
(require 'hello)
(import hello)

These two following alternatives achieve the same result as above. It loads the code and import all symbols into the global space.

1
2
(use hello)
(require-extension hello)

Usually, you don't really need to worry about how to import a module. But in some cases, it might be necessary to import symbols into the global space differently. One simple example is a function that shadows a different function. In python, you could do something like this:

1
from datetime import datetime as dd

dd will be equal to datetime and datetime won't get imported into the module's locals. In Scheme, it is quite powerful. Without getting into details, while importing symbols, you can have them renamed and prefixed. You can import only one symbol from a particular module. The prefixed import are very useful.

Let say we want to import our greet function with a prefix. We would do it like this:

1
2
3
(require 'hello)
(import (prefix hello hello:))
(hello:greet)

In human words: import everything from hello prefixed with “hello:”. To call our function, we obviously will have to call it with the prefix. Calling the function without prefix is an error because the symbols without prefix are never bound to the global scope.

Putting it all together

For a project, I really needed a way to build a complete hiearchy of modules and decided to write a small utility that would just do that. The builder is looking for all files named “*.scd” in a particular directory and then compile the import module and the module itself. The builder works at some point and I'm probably going to improve it in the future. I'm not going to explain how it works but I felt I should share it.

builder.scm

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
(use regex)
(use posix)
(use make)
(use shell)

; purity is the name of the folder it will check for libraries.
; It doesn't yet support recursive search.
(define modules (glob "purity/*.scd"))

(define (basename file)
  (cadr (string-match "(.*)\\.(.*)" file)))

(define (change_ext file ext)
  (string-append (basename file) "." ext))

(define (to-import file)
  (change_ext file "import.so"))

(define (convert file)
  (list (list (change_ext file "so")
              (list file)
              (lambda ()
                (run (csc -library ,file -J))))
        (list (change_ext file "import.so")
              (list (change_ext file "so"))
              (lambda ()
                (run (csc -library ,(change_ext file "import.scm")))))))

;(pretty-print (join (map convert modules)))

(define (build)
  (make/proc (join (map convert modules))
             (map to-import modules)))

(define (clean)
  (print "Cleaning ...")
  (for-each (lambda (line)
    (run (rm ,(car line))))
    (join (map convert modules))))

(define main
  (lambda (command)
    (cond ((equal? command "build") (build))
          ((equal? command "clean") (clean))
          (else (build)))))

(if (= 2 (length (argv)))
  (main (cadr (argv)))
  (main "build"))

That's it for today!

comments powered by Disqus

Copyright © 2015 Loïc Faure-Lacroix