How To Run mk:defsystem.lisp under Corman Common Lisp
April 19, 2008
I recently spent some time getting the clocc distribution of defsystem.lisp working with Corman Lisp. If you are developing a large system you need some sort of system definition facility; In Lisp you basically have two choices if you don’t want to roll your own: asdf:defsystem or mk:defsystem. Corman Lisp has problems with some features of asdf, so that leaves mk:defsystem. The following is a description of what needed to be done to get mk:defsystem working.
I have banged on it some, but not a lot yet. After the next release of Corman Lisp I will petition to get the changes added to mk:defsystem so that it works out of the box.
Applies to:
Corman Common Lisp v3.01+, mk:defsystem v3.6i
Source:
downloaded defsystem.lisp from
http://clocc.cvs.sourceforge.net/*checkout*/clocc/clocc/src/defsystem-3.x/defsystem.lisp
into ~/lisp/systems directory.
Required changes to source defsystem.lisp:
remove the defun near line 1082:
#+cormanlisp
(defun compile-file-pathname…)
remove the defun near line 1091:
#+cormanlisp
(defun file-namestring…)
remove the defun near line 1231:
#+cormanlisp
(defun home-subdirectory…)
remove the macro “#-cormanlisp” near line 1243 so that the region reads
(defun home-subdirectory…
instead of
#-cormanlisp
(defun home-subdirectory…
Changes to Corman Lisp:
Added a few definitions to [CORMAN]init.lisp for common-lisp compatibility
(defun file-namestring (path)
(let ((name (pathname-name path))
(type (pathname-type path)))
(format nil “~@[~A~]~@[.~A~]” name type)))
(defun software-version () nil)
(defun software-type () nil)
(defun directory-namestring (path)
(let ((dir (pathname-directory path)))
(namestring (make-pathname :directory dir))))
Added location for defsystem module in [CORMAN]init.lisp:
(setq *lisp-systems-directory* (concatentate ’string (namestring (user-homedir-pathname)) “lisp\\systems\\”))
(push-module-directory *lisp-systems-directory*)
Changes for Convenience
You will probably want to change some global defaults in defsystem.lisp so defsystem doesn’t prompt you needlessly. Here are my settings (beginning around line 1319 in defsystem.lisp).
Variable: *oos-verbose* Setting: t
Variable: *load-source-if-no-binary* Setting: t
Variable: *bother-user-if-no-binary* Setting: nil
Variable: *load-source-instead-of-binary* Setting: t
Variable: *compile-during-load* Setting: nil
Testing:
1. Create ~/lisp/systems/test.system to contain:
(mk:defsystem “test”
:source-pathname *lisp-systems-directory*
:source-extension “lisp”
:binary-pathname nil
:binary-extension nil
:components ((:file “test”))
:depends-on nil)
2. Create ~/lisp/systems/test.lisp to contain:
(defpackage “TEST”
(:documentation “A test package for defsystem”))
(in-package “TEST”)
(export ‘(test-function))
(defun test-function ()
“Test package found and functioning.”)
3. Start Corman Lisp.
4. (require ‘defsystem)
==> ignorable compiler warnings…
==> “DEFSYSTEM”
[4a. optional: (use-package "MAKE")]
5. (mk:oos “TEST” :load :verbose t)
==> …
==> (#<FILE: test>)
6. (use-package :test)
==> T
7. (test-function)
==> “Test package found and functioning.”
Notes:
Another system definition facility, ASDF, is part of the Corman Lisp distribution, but it tended to
hang my system under XEmacs and Corman Lisp, so I have been using mk:defsystem.
A queer feature of the defpackage and export functions is that, under Corman Lisp anyway, defpackage
requires you to designate exported symbols via upcase strings (in most cases) whereas export
requires you to designate the symbols by using the symbols. This is in accord with the CLHS, but
counter-intuitive.
The :compile option to oos seems to cause problems. Stick with :load.
Documentation for defsystem:
Excerpted verbatim from http://rpgoldman.real-time.com/lisp/defsystem.html
The general format of a component’s definition is:
<definition> ::= (<type> <name> [:host <host>] [:device <device>]
[:source-pathname <pathname>]
[:source-extension <extension>]
[:binary-pathname <pathname>]
[:binary-extension <extension>]
[:package <package>]
[:initially-do <form>]
[:finally-do <form>]
[:components (:serial <definition>*)|(<definition>*)]
[:depends-on (<name>*)]
[:compile-form <form>]
[:load-form <form>])
<type> ::= :system | :module | :file
<def-or-fn> ::= <definition> | <pathname>
Using Systems with Operate-on-System
The function operate-on-system (aka: oos) is used to compile or load a
system, or do any other operation on a system. At present only compile and load operations are
defined, but other operations such as edit, hardcopy, or applying arbitrary functions (e.g.,
enscript, lpr) to every file in the system could be added easily.
The syntax of operate-on-system is as follows:
operate-on-system system-name operation
&key force test verbose dribble load-source-instead-of-binary
load-source-if-no-binary bother-user-if-no-binary
SYSTEM-NAME
is the name of the system and may be a symbol or string.
OPERATION
is ‘compile (or :compile) or ‘load (or :load) or any new operation defined by the user.
FORCE
determines what files are operated on: :all (or T) specifies that all files in the system should be
used :new-source compiles only those files whose sources are more recent than the binaries and loads
the source if it is newer than the binaries. This allows you to load the most up to date version of
the system. :new-sources-and-dependents uses all files used by :new-source, plus any files that
depend on the those files or their dependents (recursively). Force may also be a list of the
specific modules or files to be used (plus their dependents). The default for ‘load is :all and for
‘compile is :new-source-and-dependents.
VERSION
indicates which version of the system should be used. If nil, then the usual root directory is
used. If a symbol, such as ‘alpha, ‘beta, ‘omega, :alpha, or ‘mark, it substitutes the appropriate
(lowercase) subdirectory of the root directory for the root directory. If a string, it replaces the
entire root directory with the given directory.
VERBOSE
is T to print out what it is doing (compiling, loading of modules and files) as it does it. (default
nil)
TEST
is T to print out what it would do without actually doing it. If test is T it automatically sets
verbose to T. (default nil)
DRIBBLE
should be the pathname of a dribble file if you want to keep a record of the compilation. (default
nil)
LOAD-SOURCE-INSTEAD-OF-BINARY
is T to force the system to load source files instead of binary files. (default nil)
LOAD-SOURCE-IF-NO-BINARY
is T to have the system load source files if the binary file is missing. (default nil)
BOTHER-USER-IF-NO-BINARY
is T to have the system bother the user about missing binaries before it goes ahead and loads them
if load-source-if-no-binary is T. (default t) Times out in 60 seconds unless *use-timeouts* is set
to nil.
Changes to Require
This defsystem interacts smoothly with the require and provide facilities of Common
Lisp. Operate-on-system automatically provides the name of any system it loads, and uses the new
definition of require to load any dependencies of the toplevel system.
To facilitate this, three new optional arguments have been added to require. Thus the new syntax of
require is as follows:
require system-name &optional pathname definition-pname default-action version
If pathname is provided, the new require behaves just like the old definition. Otherwise it first
tries to find the definition of the system-name (if it is not already defined it will load the
definition file if it is in the current-directory, the central-registry directory, or the directory
specified by definition-pname) and runs operate-on-system on the system definition. If no definition
is to be found, it will evaluate the default-action if there is one. Otherwise it will try running
the old definition of require on just the system name. If all else fails, it will print out a
warning.
A Sample System Definition and Its Use
Here’s a system definition for the files in the following directory structure:
% du -a test
1 test/fancy/macros.lisp
1 test/fancy/primitives.lisp
3 test/fancy
1 test/macros.lisp
1 test/primitives.lisp
1 test/graphics/macros.lisp
1 test/graphics/primitives.lisp
3 test/graphics
1 test/os/macros.lisp
1 test/os/primitives.lisp
3 test/os
12 test
(defsystem test
:source-pathname “/afs/cs.cmu.edu/user/mkant/Defsystem/test/”
:source-extension “lisp”
:binary-pathname nil
:binary-extension nil
:components ((:module basic
:source-pathname “”
:components ((:file “primitives”)
(:file “macros”
:depends-on (“primitives”))))
(:module graphics
:source-pathname “graphics”
:components ((:file “macros”
:depends-on (“primitives”))
(:file “primitives”))
:depends-on (basic))
(:module fancy-stuff
:source-pathname “fancy”
:components ((:file “macros”
:depends-on (“primitives”))
(:file “primitives”))
:depends-on (graphics operating-system))
(:module operating-system
:source-pathname “os”
:components ((:file “primitives”)
(:file “macros”
:depends-on (“primitives”)))
:depends-on (basic)))
:depends-on nil)
<cl> (operate-on-system ‘test ‘compile :verbose t)
Slime, Swank, and Corman Lisp v3.01+
April 4, 2008
How to Run Corman Lisp with XEmacs/SLIME
I have more-or-less successfully got Corman Common Lisp running with XEmacs/SLIME. I will describe how…
In order to get things running I’ve made a number of changes to Swank and Corman Lisp. If the cookbook below doesn’t work for you I have probably forgotten to document one of the changes, so if you’ll let me know I’ll try to track it down.
Versions
This report applies to:
-
xemacs (version 21.4.21)
-
slime (version 2.0 changelog dated 2006-04-20)
-
corman common lisp (version 3.01)
-
Windows XP and Vista.
Procedure
Download and install XEmacs and Corman Common Lisp.
Modify XEmacs
Run XEmacs and make the following modifications to your init file by either editing it directly (if you know where XEmacs looks for it) or by using /Options/Edit Init File from the XEmacs menu bar:
(add-to-list ‘load-path “/Program Files/XEmacs/xemacs-packages/lisp/slime-2.0″)
(require ’slime)
(slime-setup :autodoc t)
Of course, if you have installed XEmacs in a non-default location, you must modify the load path accordingly.
Modify Corman Common Lisp
You need patched versions of the following Corman Lisp distribution files: clos.lisp, xp.lisp, and trace.lisp. These are available from Corman Lisp. When the version of Corman Lisp subsequent to v3.01 is available you will be able simply to use the normal files from the distribution and will not need to acquire patched files from Corman Lisp. In the meantime, you should add the patched files into your Corman Lisp image.
You must modify the Sys/sockets.lisp file and the Sys/winsock.lisp files from the Corman Lisp distribution as suggested here by ungil on 19 Sep 2006. With apologies, I am copying ungil’s changes verbatim here:
| $ diff -b sockets.lisp 469c469 < (winsock::listen (socket-descriptor s) SOMAXCONN)))) — > (listen (socket-descriptor s) SOMAXCONN))))$ diff -b winsock.lisp 14,15d13 < (shadow ‘(cl:listen)) < |
I have integrated these changes into my Corman Lisp image. They do not seem to have broken anything, but I cannot be sure. If you want to be on the safe side, do not add the sockets.lisp and winsock.lisp changes into your lisp image, just modify the relevant functions after initializing your lisp.
You must modify the Modules/gray-streams.lisp file so that it defines stream-terpri. Add the following code to the file
| (defmethod stream-terpri ((stream fundamental-character-output-stream)) (if (stream-start-line-p stream) nil (progn (stream-write-char stream #\newline) t))) |
If you prefer, you can download the modified file from here, and simply copy it over the old one in the Modules directory. The modified file contains a number of additional stub function definitions that should be transparent to any code currently using gray-streams.lisp.
You need to make a few additional, special changes to Corman Lisp for Slime compatibility. I have collected these in a separate file called slime-compatibility.lisp here. The file slime-compatibility makes the following modifications:
-
Fix the definition of #’user-homedir-pathname;
-
Redefine #’pprint so that it sets *print-pretty* nil;
-
Redefine #’cl:machine-instance, #’cl:machine-type, #’cl:machine-version;
-
Add a feature to *features*;
When you start Corman Lisp, you will need to reload sockets.lisp, winsock.lisp, gray-streams.lisp, and slime-compatibility.lisp so that the new definitions are visible to the system. Alternatively, you can create a new lisp image using the function #’compile-cormanlisp-image. (Consult the documentation.)
You must load all these changes into Corman Lisp before loading swank.
Modify Swank
You must make a number of modifications to the code distributed with XEmacs.
In the slime distribution file [XEmacs]/xemacs-packages/lisp/slime-2.0/swank.lisp you need to comment out a series of assertions. Open the file and search for ‘test-print-arglist’. Then comment out the progn as follows:
#-cormanlisp(progn
(assert (test-print-arglist ‘(function cons) “(function cons)“))
(assert (test-print-arglist ‘(quote cons) “(quote cons)“))
(assert (test-print-arglist ‘(&key (function #’+)) “(&key (function #’+))“))
(assert (test-print-arglist ‘(&whole x y z) “(y z)“))
(assert (test-print-arglist ‘(x &aux y z) “(x)“))
(assert (test-print-arglist ‘(x &environment env y) “(x y)“)))
;; Expected failure:
;; (assert (test-print-arglist ‘(&key ((function f))) “(&key ((function f)))”))
I have also redefined #’swank::swank-debugger-hook so that you can disable the debugger by controlling the value of *maybe-disable-debugger*. In addition to being convenient, it avoids some instabilities involved with the Corman Lisp Debugger. Here is the relevant section of swank.lisp:
#+cormanlisp
(defvar *maybe-avoid-debugger* t
“do not enter debugger if it is possible simply to abort the request”)
#+cormanlisp
(defun swank-debugger-hook (condition hook)
(declare (ignore hook))
(cond ((and *maybe-avoid-debugger* (find-restart ‘abort-request))
(send-to-emacs `(:write-string ,(format nil “~A” condition)))
(invoke-restart ‘abort-request))
(*emacs-connection*
(debug-in-emacs condition))
((default-connection)
(with-connection ((default-connection))
(debug-in-emacs condition)))))
#-cormanlisp
(defun swank-debugger-hook (condition hook)
“Debugger function for binding *DEBUGGER-HOOK*.
Sends a message to Emacs declaring that the debugger has been entered,
then waits to handle further requests from Emacs. Eventually returns
after Emacs causes a restart to be invoked.”
(declare (ignore hook))
(cond (*emacs-connection*
(debug-in-emacs condition))
((default-connection)
(with-connection ((default-connection))
(debug-in-emacs condition)))))
In the slime distribution file [XEmacs]/xemacs-packages/lisp/slime-2.0/swank-corman.lisp you need to modify the definition of #’arglist as follows:
| (defimplementation arglist (name) (handler-case (cond ((and (symbolp name) (macro-function name)) (let* ((function (symbol-function name)) (macro-list (ccl::macro-lambda-list function))) (if macro-list macro-list :not-available))) (t (when (symbolp name) (setq name (symbol-function name))) (if (eq (class-of name) cl::the-class-standard-gf) (generic-function-lambda-list name) (ccl:function-lambda-list name)))) (error () :not-available))) |
Appropriate swank-corman.lisp and swank.lisp files are available as a .zip file for download here. After the next release of Corman Common Lisp (whatever version comes after 3.01), I will see about getting the changes added to the official XEmacs distribution so the standard distribution will simply work out of the box.
Run Slime and Corman Lisp
-
Start Swank on the corman lisp side via the following steps:
-
Start [Corman Lisp]/CormanLisp.exe;
-
(load “Sys/winsock.lisp”)
-
(load “Sys/sockets.lisp”)
-
(load “Modules/gray-streams.lisp”)
-
(load “swank-compatibility.lisp”) ; from wherever you saved swank-compatibility.lisp
- Change directory to the slime distribution. For example (setf (cormanlisp:current-directory) “C:\\Program Files\\XEmacs\\xemacs-packages\\lisp\\slime-2.0\\“)
-
(load “swank-loader.lisp”) ; this will take some time
- (swank::create-server)
-
-
Assuming everything works, you should see a notice that Swank started at port: 4005.
-
Start xemacs.
-
Initiate slime via
-
Alt-x slime-connect<return>
-
Host: 127.0.0.1<return>
-
Port: 4005<return>
-
-
Assuming everything works, you should see a “Connected” notice.
If you prefer to keep the full debugging capabilities, setq swank::*maybe-avoid-debugger* t.
What Works; What Doesn’t
;;; As of
;;; 04 Apr 2008 -
;;; Functionality Description Yes/No/Kinda
;;; ===============================================
;;; C-M-x, etc. basic editing and eval Y
;;; arglist display K
;;; C-c C-c, etc compilation Y
;;; loading files Y
;;; C-c C-d d, &c describe/hyperspec Y
;;; debugger K
;;; C-c I inspector Y
;;; profiling N
;;; stepper N
;;; comm styles (:spawn etc.) N
;;; C-c C-w c, &c cross referencing N
;;; M-. edit definition Y
;;; M-TAB complete symbol Y
;;; C-c C-s complete form K
Notes:
The debugger is mostly ok. It is possible to crash it if you are ungentle. I personally find the lisp-style of debugger overkill in most instances, so it is nice to be able to disable it unless needed. If you want to disable it, at the slime repl setq swank::*maybe-avoid-debugger* t. If you want to re-enable it, setq swank::*maybe-avoid-debugger* nil.
Corman Lisp doesn’t currently feature a stepper.
The arglist display works for all user-defined functions and some built-in functions. Inexplicably, a large fraction of the built-in functions haven’t saved their arglists, so, for example, you can’t get arglist information on #’defun.
RSS 