Compilation and package system

[Kent M Pitman <KMP@SCRC-STONY-BROOK.ARPA>]

    Date: Wed, 10 Apr 1985  23:51 EST
    Message-ID: <WHOLEY.12102192890.BABYL@CMU-CS-C.ARPA>
    Sender: WHOLEY@CMU-CS-C.ARPA
    From: Skef Wholey <Wholey@CMU-CS-C.ARPA>
    To:   "David A. Moon" <Moon@SCRC-STONY-BROOK.ARPA>
    Cc:   Common-lisp@SU-AI.ARPA
    Subject: Compilation and package system

	Date: Wednesday, 10 April 1985  22:33-EST
	From: David A. Moon <Moon at SCRC-STONY-BROOK.ARPA>

	    Date: Tue, 9 Apr 1985  22:14 EST
	    From: Skef Wholey <Wholey@CMU-CS-C.ARPA>

	    First of all, it [the -*- comment convention] is not portable.

	I guess it's a matter of viewpoint which way is non-portable, depending on
	which way you're trying to port.  But let's not get into that.

    Let's get into that for just one paragraph.  Nothing in CLTL says anything
    about -*- comments.  Files with those comments will have the effect that you
    (Dave Moon) expect them to in only one Common Lisp implementation: the
    Symbolics CLCP.  The implementations done by DEC, DG, Rutgers, Kyoto
    University, and CMU will not understand them.  I realize that (by itself) this
    kind of survey is a bad way to decide language design issues, but the reason
    that those implementations do not hack -*- the Lisp Machine way is that there's
    nothing in the spec that says they should.

Don't be silly. Common Lisp programs don't have to run gracefully in all 
implementations, they just have to run. If a person has written -*- ... -*-
and not put (IN-PACKAGE ...), he has not written a common lisp program.
But that doesn't mean the Lisp Machine doesn't support IN-PACKAGE. If a 
person has put (IN-PACKAGE ...) but no -*- ... -*- that means his program
will run in the way Common Lisp says it will, but that doesn't mean it will
interface gracefully to parts of the system not specified by Common Lisp.
The Lisp Machine simply (and reasonably) suggests that in order to work
compatibly with non-Common-Lisp utilities and still be portable, you should
do -*- ... -*- and (IN-PACKAGE ...), redundantly. In fact, I suspect that
users will be content to use simply -*- ... -*- and add (IN-PACKAGE ...) only
when they get around to actually shipping files but that shouldn't matter.

	We believe that complex package setups should go in their own files.

    I have two programs that I've run on both Perqs and 3600's.  One is a Common
    Lisp version of OPS-5, and another is the Spice Lisp microcode assembler.  Each
    are contained in exactly one file, and are thus easy to move from system to
    system.  Why should I need to double the number of files in such systems?

    For us, "complex package setups" are a way of life, because we've chosen to put
    the code for communicating with each OS server in its own file.  When I sit
    down to write a graphics hack, I begin a file with
	    ;;; -*- Package: Hack -*-
	    (in-package "HACK" :use '("LISP" "ACCINT" "SAPPH" "SESAME")).
    I suppose our differences may stem from the fact that we Spice Lispers don't
    think that packages are very expensive.  Packages can be used for little hacks
    as well as big systems.  In the former case, one shouldn't need to stash this
    information in a separate file that then must be loaded by hand (or in my init
    file (or by some hook I frob in my init file)).

Of course, for this sort of thing I usually just write:there's the alternate syntax:
  ;;; -*- Package: ("HACK" :USE ("LISP" "ACCINT" "SAPPH" "SESAME")) -*-
which seems to work OK...

	    The editor knows which package to associate with a buffer created when
	    visiting this file, but the compiler and loader and everyone else just
	    read Lisp forms, the way Father Lisp wanted them to.

	This all sounds at first like it makes a lot of sense, but wait a minute.
	I fail to see how the editor needs any less information about packages than
	the compiler and the loader.  How does the editor know the names of the
	functions in the file if it doesn't know what package they are relative to?

    I guess I was being unclear.  The editor interprets -*- comments, and
    associates a package with the buffer the file was read into.  This package is
    made the default package (i.e. *package* is bound to it) whenever the editor
    parses code in the file (e.g. for incremental compilation) or the echo area
    (e.g. when asked to list the argument names of a given function).

    If the package does not exist, then the user is asked if it should be created.
    If that package uses packages other than LISP, then the user must evaluate the
    In-Package or Use-Package or Import or whatever himself before counting on the
    editor to parse code correctly.  This last step could be automated, but we
    haven't done it.

	In our system all programs that read Lisp forms from files get the package
	information from the same place and they all work consistently.

    Ah, but all those programs have to do some magic thing (not part of the Common
    Lisp language) to interpret the stuff between -*-'s in the first line of the
    file.  Should a user be able to write a portable slow-loader (that just reads
    forms from a given file) or not?  I think he should.  People should be able to
    write things like program cross-referencers in Common Lisp.  Portable Common
    Lisp (whatever that means).

Don't forget the Lisp Machine uses -*- ... -*- for other languages besides lisp.
(IN-PACKAGE ...) would not work at the top of files in other languages besides
lisp and we don't have hands in design groups for every existing language so 
cannot always get those languages to add a syntax for specifying the package.
Putting it in a comment is mostly likely to win in all languages since nearly 
every language has a comment syntax and I know of no languages for which the 
comment syntax is not compatible with -*- ... -*-.

	IN-PACKAGE was put into Common Lisp over my dead body.

    The rumors of your death have been greatly exaggerated.

	It might be worth some effort to standardize the semantics of -*-.

    How about this: Symbolics could provide a portable -*- comment hacking package.
    This would be made part of the yellow pages, and all implementations could
    "voluntarily" adopt it as a thing that gets called by Load, Compile-File, etc.
    This might then be adopted in Common Lisp Mark II (it's really no worse than
    FORMAT, I guess).  I'm still personally biased against this idea (I really do
    believe that a slow-loader should just read forms out of a file), but portable
    code would make the Lisp Machine convention a lot easier to swallow.

    On the other hand, we could decide that -*-'s don't port, and warn users about
    differences from system to system.  Rob MacLachlan has suggested a very
    sinister solution to the problem: conditionalized constructs in -*- coments...

	-*- #+LISPM{Syntax: Common-Lisp; Uppercase: Mostly;} Package: Zippy -*-

    --Skef