How To Run mk:defsystem.lisp under Corman Common Lisp

 

I recently spent some time getting the clocc distribution of defsystem.lisp working with Corman Lisp.  If you are developing a large system you need some sort of system definition facility; In Lisp you basically have two choices if you don’t want to roll your own: asdf:defsystem or mk:defsystem.  Corman Lisp has problems with some features of asdf, so that leaves mk:defsystem.  The following is a description of what needed to be done to get mk:defsystem working.

I have banged on it some, but not a lot yet.  After the next release of Corman Lisp I will petition to get the changes added to mk:defsystem so that it works out of the box. 

Applies to:
 Corman Common Lisp v3.01+, mk:defsystem v3.6i

Source:
 downloaded defsystem.lisp from
http://clocc.cvs.sourceforge.net/*checkout*/clocc/clocc/src/defsystem-3.x/defsystem.lisp
into ~/lisp/systems directory.

Required changes to source defsystem.lisp:

remove the defun near line 1082:
#+cormanlisp
(defun compile-file-pathname…)

remove the defun near line 1091:
#+cormanlisp
(defun file-namestring…)

remove the defun near line 1231:
#+cormanlisp
(defun home-subdirectory…)

remove the macro “#-cormanlisp” near line 1243 so that the region reads
(defun home-subdirectory…
instead of
#-cormanlisp
(defun home-subdirectory…

Changes to Corman Lisp:
 
Added a few definitions to [CORMAN]init.lisp for common-lisp compatibility

(defun file-namestring (path)
  (let ((name (pathname-name path))
        (type (pathname-type path)))
    (format nil “~@[~A~]~@[.~A~]” name type)))

(defun software-version () nil)

(defun software-type () nil)

(defun directory-namestring (path)
  (let ((dir (pathname-directory path)))
    (namestring (make-pathname :directory dir))))

Added location for defsystem module in [CORMAN]init.lisp:

(setq *lisp-systems-directory* (concatentate ’string (namestring (user-homedir-pathname)) “lisp\\systems\\”))
(push-module-directory *lisp-systems-directory*)

Changes for Convenience

You will probably want to change some global defaults in defsystem.lisp so defsystem doesn’t prompt you needlessly.  Here are my settings (beginning around line 1319 in defsystem.lisp). 

Variable: *oos-verbose* Setting: t
Variable: *load-source-if-no-binary* Setting: t
Variable: *bother-user-if-no-binary* Setting: nil
Variable: *load-source-instead-of-binary* Setting: t
Variable: *compile-during-load* Setting: nil

Testing:
1. Create ~/lisp/systems/test.system to contain:

(mk:defsystem “test”
  :source-pathname *lisp-systems-directory*
  :source-extension “lisp”
  :binary-pathname nil
  :binary-extension nil
  :components ((:file “test”))
  :depends-on nil)

2. Create ~/lisp/systems/test.lisp to contain:

(defpackage “TEST”
  (:documentation “A test package for defsystem”))

(in-package “TEST”)

(export ‘(test-function))

(defun test-function ()
  “Test package found and functioning.”)

3. Start Corman Lisp.
4. (require ‘defsystem)
==> ignorable compiler warnings…
==> “DEFSYSTEM”
[4a. optional: (use-package "MAKE")]
5. (mk:oos “TEST” :load :verbose t)
==> …
==> (#<FILE: test>)
6. (use-package :test)
==> T
7. (test-function)
==> “Test package found and functioning.”

Notes:

Another system definition facility, ASDF, is part of the Corman Lisp distribution, but it tended to
hang my system under XEmacs and Corman Lisp, so I have been using mk:defsystem.

A queer feature of the defpackage and export functions is that, under Corman Lisp anyway, defpackage
requires you to designate exported symbols via upcase strings (in most cases) whereas export
requires you to designate the symbols by using the symbols.  This is in accord with the CLHS, but
counter-intuitive.

The :compile option to oos seems to cause problems.  Stick with :load.
Documentation for defsystem:

Excerpted verbatim from http://rpgoldman.real-time.com/lisp/defsystem.html

The general format of a component’s definition is:

<definition> ::= (<type> <name> [:host <host>] [:device <device>]
                                [:source-pathname <pathname>]
                                [:source-extension <extension>]
                                [:binary-pathname <pathname>]
                                [:binary-extension <extension>]
                                [:package <package>]
                                [:initially-do <form>]
                                [:finally-do <form>]
                                [:components (:serial <definition>*)|(<definition>*)]
                                [:depends-on (<name>*)]
                                [:compile-form <form>]
                                [:load-form <form>])
<type> ::= :system | :module | :file
<def-or-fn> ::= <definition> | <pathname>

Using Systems with Operate-on-System

The function operate-on-system (aka: oos) is used to compile or load a
system, or do any other operation on a system. At present only compile and load operations are
defined, but other operations such as edit, hardcopy, or applying arbitrary functions (e.g.,
enscript, lpr) to every file in the system could be added easily.

The syntax of operate-on-system is as follows:

operate-on-system system-name operation
        &key force test verbose dribble load-source-instead-of-binary
        load-source-if-no-binary bother-user-if-no-binary

SYSTEM-NAME
is the name of the system and may be a symbol or string.

OPERATION
is ‘compile (or :compile) or ‘load (or :load) or any new operation defined by the user.

FORCE

determines what files are operated on: :all (or T) specifies that all files in the system should be
used :new-source compiles only those files whose sources are more recent than the binaries and loads
the source if it is newer than the binaries. This allows you to load the most up to date version of
the system. :new-sources-and-dependents uses all files used by :new-source, plus any files that
depend on the those files or their dependents (recursively). Force may also be a list of the
specific modules or files to be used (plus their dependents). The default for ‘load is :all and for
‘compile is :new-source-and-dependents.

VERSION

indicates which version of the system should be used. If nil, then the usual root directory is
used. If a symbol, such as ‘alpha, ‘beta, ‘omega, :alpha, or ‘mark, it substitutes the appropriate
(lowercase) subdirectory of the root directory for the root directory. If a string, it replaces the
entire root directory with the given directory.

VERBOSE

is T to print out what it is doing (compiling, loading of modules and files) as it does it. (default
nil)

TEST

is T to print out what it would do without actually doing it. If test is T it automatically sets
verbose to T. (default nil)

DRIBBLE

should be the pathname of a dribble file if you want to keep a record of the compilation. (default
nil)

LOAD-SOURCE-INSTEAD-OF-BINARY
is T to force the system to load source files instead of binary files. (default nil)

LOAD-SOURCE-IF-NO-BINARY
is T to have the system load source files if the binary file is missing. (default nil)

BOTHER-USER-IF-NO-BINARY

is T to have the system bother the user about missing binaries before it goes ahead and loads them
if load-source-if-no-binary is T. (default t) Times out in 60 seconds unless *use-timeouts* is set
to nil.

Changes to Require

This defsystem interacts smoothly with the require and provide facilities of Common
Lisp. Operate-on-system automatically provides the name of any system it loads, and uses the new
definition of require to load any dependencies of the toplevel system.

To facilitate this, three new optional arguments have been added to require. Thus the new syntax of
require is as follows:

require system-name &optional pathname definition-pname default-action version

If pathname is provided, the new require behaves just like the old definition. Otherwise it first
tries to find the definition of the system-name (if it is not already defined it will load the
definition file if it is in the current-directory, the central-registry directory, or the directory
specified by definition-pname) and runs operate-on-system on the system definition. If no definition
is to be found, it will evaluate the default-action if there is one. Otherwise it will try running
the old definition of require on just the system name. If all else fails, it will print out a
warning.

A Sample System Definition and Its Use
Here’s a system definition for the files in the following directory structure:

        % du -a test
        1       test/fancy/macros.lisp
        1       test/fancy/primitives.lisp
        3       test/fancy
        1       test/macros.lisp
        1       test/primitives.lisp
        1       test/graphics/macros.lisp
        1       test/graphics/primitives.lisp
        3       test/graphics
        1       test/os/macros.lisp
        1       test/os/primitives.lisp
        3       test/os
        12      test
(defsystem test
  :source-pathname “/afs/cs.cmu.edu/user/mkant/Defsystem/test/”
  :source-extension “lisp”
  :binary-pathname nil
  :binary-extension nil
  :components ((:module basic
                        :source-pathname “”
                        :components ((:file “primitives”)
                                     (:file “macros”
                                            :depends-on (“primitives”))))
               (:module graphics
                        :source-pathname “graphics”
                        :components ((:file “macros”
                                            :depends-on (“primitives”))
                                     (:file “primitives”))
                        :depends-on (basic))
               (:module fancy-stuff
                        :source-pathname “fancy”
                        :components ((:file “macros”
                                            :depends-on (“primitives”))
                                     (:file “primitives”))
                        :depends-on (graphics operating-system))
               (:module operating-system
                        :source-pathname “os”
                        :components ((:file “primitives”)
                                     (:file “macros”
                                            :depends-on (“primitives”)))
                        :depends-on (basic)))
  :depends-on nil)

<cl> (operate-on-system ‘test ‘compile :verbose t)