Paolo Amoroso's Journal

Tech projects, hobby programming, and geeky thoughts of Paolo Amoroso

I read the book Recursion via Pascal by Jeffrey S. Rohl, Cambridge University Press, 1984. I discovered this rare, little known gem by chance and, although it's available online, I also bought a cheap printed copy.

The book Recursion via Pascal.

What makes this short book so interesting? The preface opens with this sentence:

Recursion is the cinderella of programming techniques where languages such as Pascal are concerned.

Programming books usually devote little space to recursion, often to briefly introduce and dismiss it as hard to understand and inefficient.

As a Lisp enthusiast, recursion comes natural to me and is a natural fit for the language. A work entirely on recursion like this is the book I always wanted but didn't know existed.

Recursion via Pascal uses math throughout the text but the explanations are usually clear, and the elementary discrete mathematics with a few bits of calculus is not hard to follow. The snippets in Pascal are short and mostly easy to understand. This code can be ported or adapted with little effort.

What makes the book unique is the coverage of recursion techniques. For example the techniques for making recursion more efficient such as terminating recursive calls earlier, or implementing algorithms with auxiliary procedures that take fewer parameters. The book also gives guidance on when recursion is most appropriate and when it's not.

While most programming books don't say much on eliminating recursion other than remarking it's easy, Recursion via Pascal has a full chapter on this that actually explains what to do.

I'm glad to have such a handy reference work on recursion.

#books #pascal #development

Discuss... Email | Reply @amoroso@fosstodon.org

I finally fixed a longstanding duplicate definitions issue with Femtounit, my Interlisp unit test framework.

Femtounit creates a new File Manager type for unit tests, TESTS. DEFTEST, which defines a test, adds an entry of type TESTS and expands into an internal function that carries out the test when called. The problem was the File Manager noticed and tracked two objects for each DEFTEST, the TESTS entry and the function. But the function is an implementation detail that shouldn't be tracked by the File Manager or seen directly by the user.

To fix the issue, in the DEFDEFINER that creates the new type I added a call to UNMARKASCHANGED immediately after the definition of the internal function. UNMARKASCHANGED undoes the association with the File Manager the creation of the internal function establishes. I had tried DELDEF but it removes both the association and the internal function.

I also refactored the definition of the internal function to use DEFINEQ instead of assigning a LAMBDA to the function cell of the symbol naming the test, which is less clear and obscures the intent.

#femtounit #Interlisp #Lisp

Discuss... Email | Reply @amoroso@fosstodon.org

I shut down Free Python Books, the list of Python books that are free to read online or download I had been maintaining since 2019.

I archived the project's GitHub repo, which is now read-only and no longer accepts contributions. If you're interested you're welcome to fork the repo and maintain your copy.

When learning Python I came across many great free Python books, so I started keeping track of them. In early 2019 I shared my list on Reddit where it resonated with many. I later published the list on GitHub and it eventually gained over 4,500 stars, about 600 forks, and over 100 watchers.

In 2023 I rediscovered my love of Lisp and lost interest in Python, which I no longer use. Hence my decision to mothball the project.

Thanks to all who expressed appreciation or contributed, and to the Python community for producing so many great works.

#Python #books

Discuss... Email | Reply @amoroso@fosstodon.org

I got a cheap used copy of the book A Programmer's Guide to COMMON LISP by Deborah G. Tatar, Digital Press, 1987.

The book A Programmer's Guide to COMMON LISP by Deborah G.

Why did I read such an old book, published a few years after CLtL1 and well before ANSI finalized the Common Lisp standard?

I'm always looking for good Lisp books. Since Medley is my primary Lisp environment, I'm particularly interested in books published when the system was originally developed and used. These works are relevant because they cover a set of features close to the state of the Common Lisp implementation of Medley, and present a programming style typical of Lisp development in those years.

Two old reviews got me curious about A Programmer's Guide to COMMON LISP, one by Daniel Weinreb and the other by Richard Caruana.

Both reviews point out the book is different from most contemporary introductory Lisp books which focus on AI. Although Tatar's does contain some AI code, such as an interesting and complete toy expert system, the sample code spans a wider range of domains like a text formatter similar to nroff.

What sets A Programmer's Guide to COMMON LISP apart from other Lisp books is its environment independent discussion of the interactive Lisp programming process. Writing code in the editor, evaluating expressions from the editor, interacting with the REPL for testing expressions and exploring, and so on.

I've never seen the process expressed so clearly in any book, past of present. I'm familiar with it but the material is particulary helpful for complete beginners.

Although the short chapter on macros presents some interesting examples like a simplified version of defstruct, it doesn't discuss gensym and variable capture. This is unusual. But it's only one of a few issues and the book is a valuable addition to my Lisp library.

#CommonLisp #books #Lisp

Discuss... Email | Reply @amoroso@fosstodon.org

Two years ago these days I announced my blog Paolo Amoroso's Journal hosted at Write.as. Write.as is a paid instance of the lightweight, federated, open source blogging platform WriteFreely.

Deciding on a blogging platform that meets my needs wasn't easy but going with Write.as is the best blogging decision I've ever taken.

The good

Write.as is perfect for me for two main reasons. First, it provides excellent support for technical writing. Markdown proved a game changer for the ease of formatting text and source code.

The other reason is Write.as' lightweight environment reduces friction. Again, part of this is due to Markdown which doesn't get in the way of producing complex technical content.

This allowed me to publish 268 posts in two years. I never blogged so much on any other platform.

Reflecting on my first year with Write.as I wrote:

The first year on Write.as was eventful. Two of my posts went viral on Hacker News, [...]

Aside from the rewards and validation of such success metrics, an unexpected benefit of the first year of blogging at Write.as has been writing for an audience of one: me.

Since then there were a couple more Hacker News hits but the last point is even more relevant now.

The post archive is an invaluable resource, for example, for reviewing the details of past projects I no longer remember and need to resume working on, or for other projects. The blog turned out as a sort of personal lab notebook, and sometimes others read what I write.

The less good

Much as I love Write.as and am grateful for what it helped me accomplish, it still has a few limitations and minor annonyances that introduce unnecessary friction.

Write.as has no post preview. A workaround is to publish an anonymous unlinked post that doesn't show up on the blog, and continue editing until it looks good. But anonymous posts don't render the blog's theme, Markdown tables, MathJax, and media embeds. Which is a problem for technical writing.

When a post is ready for public distribution I move the relevant anonymous post to the blog. However, the action doesn't trigger emailing the post to the readers who subscribe to the blog as a newsletter. A fix is to set the datestamp of the anonymous post to the current date and time just prior to moving the post to the blog.

Finally, Write.as doesn't have a global language setting for blogs and the option has to be set manually for every post. Which I did for all 268 of them.

The future

Most of the time these issues are minor. The only major limitation is the lack of a real, full post preview. I hope the issues will eventually be fixed but I still highly enjoy Write.as. The platform encourages writing and makes blogging enjoyable.

I'm happy with Write.as and the experience of the past two years strenghtens my choice. No changes ahead, I'll just continue using Write.as.

#blogging

Discuss... Email | Reply @amoroso@fosstodon.org

Medley is a residential environment for Interlisp and Common Lisp development.

With some effort it's possible to use Medley as a traditional file based Common Lisp environment. But in specific cases a better approach is to bring in Medley's residential environment Common Lisp sources created in file based environments.

In this post I explain the latter, i.e. how to use TextModules to import Common Lisp files into the residential environment. I go over the steps for converting an example program, the database of CD music tracks in Chapter 3 Practical: A Simple Database of Peter Seibel's book Practical Common Lisp.

Motivation

Using the tools and facilities of the residential environment, such as the File Manager, is the normal way of developing new Lisp programs. To run existing Common Lisp code you don't plan to change often, you can also use Medley as a traditional file based environment.

For existing code written in file based environments you want to use and further develop in Medley, a better option is to import the code into the residential environment and continue working from there. This is what TextModules helps to do.

What is TextModules

TextModules is a Medley tool for bringing Common Lisp sources into the residential environment and place them under the control of the File Manager. It can also do the reverse, i.e. export the File Manager descriptions and metadata to Common Lisp sources accessible from file based environments.

Using TextModules is a one time process. You run the tool once to import the code, then use and modify it with the tools and facilities of the residential environment.

The documentation of textModules starts from page 305 (page 335 of the PDF) of the Lisp Library Modules manual.

Preparing the Common Lisp files

This example involves two of the source files of Seibel's book, packages.lisp and simple-database.lisp in directory practicals-1.0.3/Chapter03 of the code archive.

Unlike CL:LOAD on Medley, TextModules doesn't require any special formatting of Common Lisp source files. For example, they don't need to begin with a semicolon character.

However, the Common Lisp implementation of Medley is incomplete and not ANSI compliant, so be sure to remove or adapt any unsupported forms. This is the case of the database example: packages.lisp makes current the package CL-USER which is missing from Medley. The fix is to substitute xcl-user for cl-user in the file, as XCL-USER is the Medley equivalent of CL-USER.

Another source of incompatibility is the LOOP macro. In Medley it's only a stub that runs an infinite loop no matter what clauses a call specifies.

To import with TextModules code that contains LOOP it would normally be necessary to replace any calls with equivalent expressions. Since this post focuses on TextModules I just use the LOOP calls intended to run an infinite loop, and ignore the others.

Running TextModules

As noted, importing with TextModules is the one time process of running the tool for every source file. Once in the residential environment, you save and manipulate the code as any other code under the File Manager.

First off, load TextModules by evaluating (FILESLOAD TEXTMODULES) at an Interlisp Exec. All its exported symbols are in package TM. Next, call the function TM:LOAD-TEXTMODULE for every Common Lisp file, which is similar to CL:LOAD with some additional processing.

Most Common Lisp programs comprise a file packages.lisp with package definitions, and a number of additional .lisp files that contain the bulk of the code. This dependency requires passing the files to TM:LOAD-TEXTMODULE in the proper order.

The file packages.lisp of the database defines the package for simple-database.lisp, so start with the former. At a Xerox Common Lisp (XCL) Exec with prompt > evaluate:

> (tm:load-textmodule "packages.lisp" :module "SIMPLEDB" :package (find-package "XCL-USER") :install t)
IL:SIMPLEDB

The only required argument is the input file packages.lisp. However, by default TM:LOAD-TEXTMODULE uses the same input file name as the name of the program for the File Manager. It wouldn't make much sense to call a database PACKAGES.LISP. A better choice is to pass the :module parameter with the more descriptive name SIMPLEDB.

The code in packages.lisp begins with an in-package form. To make sure the in-package symbol is accessible without qualifier, it should be read in a package such as XCL-USER that imports the standard Common Lisp symbols. Hence the argument :package (find-package "XCL-USER") in the call.

The argument :install t installs the definitions in the running system. Although not strictly necessary, it's useful for diagnostic purposes and because you likely want to continue working on the imported code.

Next, process simple-database.lisp by evaluating at a XCL Exec:

> (tm:load-textmodule "simple-database.lisp" :module "SIMPLEDB" :package (find-package "XCL-USER") :install t)
IL:SIMPLEDB

Again, the file begins with in-package and the reason for passing the :package argument is the same as for packages.lisp.

Saving the imported code

At this point the imported code is in the running Lisp image and the File Manager is ready to manipulate it. You can check the File Manager noticed the imported definitions by calling FILES? at an Interlisp Exec with prompt :

← (FILES?)
To be dumped:
SIMPLEDB ...changes to VARS: SIMPLEDBCOMS
                       VARIABLES: COM.GIGAMONKEYS.SIMPLE-DB::*DB*
                       FUNCTIONS: COM.GIGAMONKEYS.SIMPLE-DB::MAKE-CD, 
         COM.GIGAMONKEYS.SIMPLE-DB::ADD-RECORD, 
         COM.GIGAMONKEYS.SIMPLE-DB::DUMP-DB, 
         COM.GIGAMONKEYS.SIMPLE-DB::PROMPT-READ, 
         COM.GIGAMONKEYS.SIMPLE-DB::PROMPT-FOR-CD, 
         COM.GIGAMONKEYS.SIMPLE-DB::ADD-CDS, 
         COM.GIGAMONKEYS.SIMPLE-DB::SAVE-DB, 
         COM.GIGAMONKEYS.SIMPLE-DB::LOAD-DB, 
         COM.GIGAMONKEYS.SIMPLE-DB::CLEAR-DB, 
         COM.GIGAMONKEYS.SIMPLE-DB::SELECT, 
         COM.GIGAMONKEYS.SIMPLE-DB::WHERE, 
         COM.GIGAMONKEYS.SIMPLE-DB::MAKE-COMPARISONS-LIST, 
         COM.GIGAMONKEYS.SIMPLE-DB::MAKE-COMPARISON-EXPR, 
         COM.GIGAMONKEYS.SIMPLE-DB::UPDATE, 
         COM.GIGAMONKEYS.SIMPLE-DB::DELETE-ROWS

To save the definitions to the symbolic file SIMPLEDB call MAKEFILE from an Interlisp Exec:

← (MAKEFILE 'SIMPLEDB)
{DSK}<home>medley>il>SIMPLEDB.;1

Calling TM:LOAD-TEXTMODULE for every source file, and saving the result to a symbolic file with MAKEFILE, completes the import process.

You may terminate the session and resume later. When you're ready to proceed you can load, run, and modify the imported program as any other code under File Manager control.

Loading and running the imported code

In a new Medley session evaluate (FILESLOAD TEXTMODULES) at an Interlisp Exec. The tool must be in memory whenever you work with imported code, as TextModules sets up a special file environment and readtable the code needs to be read in.

Next, load the symbolic file of the database program by evaluating at a XCL Exec:

> (load "SIMPLEDB")

; Loading {DSK}<home>medley>il>SIMPLEDB.;1
; File created 14-Feb-2024 02:44:46
; IL:SIMPLEDBCOMS
IL:|{DSK}<home>medley>il>SIMPLEDB.;1|

One of the main entry points of the program is the function add-cds to add new records to the database, one record for each music track of a CD. A typical run from a XCL Exec looks like this:

> (setf *package* (find-package "COM.GIGAMONKEYS.SIMPLE-DB"))
#<Package COM.GIGAMONKEYS.SIMPLE-DB>
> (add-cds)
Title: Punch My Cards
Artist: The Fortrans
Rating: 6
Ripped [y/n]: n
Another? [y/n]: y
Title: Lisp n Roll
Artist: The Garbage Collectors
Rating: 8
Ripped [y/n]: y
Another? [y/n]: y
Title: Cdr Care Less
Artist: The Garbage Collectors
Rating: 7
Ripped [y/n]: y
Another? [y/n]: n
NIL

Since all the symbols of the program are in the package COM.GIGAMONKEYS.SIMPLE-DB, and none are exported, for convenience make the package current by setfing *package* as above.

I intentionally didn't rename the package or create nicknames. This is to show that Common Lisp code may be imported and used with minimal or no changes.

The program provides select and where to query the database. But where uses CL:LOOP features Medley doesn't support. To stay close to the original code, instead of modifying where you can use another function in Seibel's book. Define and call the specialized query function select-by-artist to search the database by artist:

> (defun select-by-artist (artist)
    (remove-if-not
     #'(lambda (cd) (equal (getf cd :artist) artist))
     *db*))
SELECT-BY-ARTIST
> (select-by-artist "The Garbage Collectors")
((:TITLE "Cdr Care Less" :ARTIST "The Garbage Collectors" :RATING 7 :RIPPED T) (:TITLE "Lisp n Roll" :ARTIST "The Garbage Collectors" :RATING 8 :RIPPED T))

Continuing the development

The imported code works. Now you can load, compile, run, edit, and save it as any other program developed in the residential environment under the File Manager.

You no longer need the original files packages.lisp and simple-database.lisp because you work only with SIMPLEDB. But remember to load TextModules with (FILESLOAD TEXTMODULES) in every session in which you use SIMPLEDB. It's a minor inconvenience but, for automating the task, you may add a loading command to the INIT initialization file or to a script that loads the program.

#CommonLisp #Interlisp #Lisp

Discuss... Email | Reply @amoroso@fosstodon.org

ChromeOS Stable 121 rolled out to my ASUS Chromebox 3 and brought with it a one-click option to upgrade Crostini.

Crostini, the Debian based Linux container of chromeOS, was running Bullseye prior to that. ChromeOS 121 popped up a notification with a button offering to upgrade to Debian Bookworm 12.4. After backing up the container I clicked the button in the notification, skipped the backup option as I had already done it, and clicked another button to start the upgrade.

The process was uneventful. ChromeOS displayed a dialog with status messages informing on the progress, then a final message confirming the successful completion.

I checked out the main Linux programs I use and they all seem to be working fine on Bookworm, as well as everighing else. Some programs actually look better as they're built on GUI frameworks that come with an updated and refreshed design.

Upgrading Crostini from Buster to Bullseye a couple of yeas earlier was less smooth. Back then chromeOS didn't provide any user interface for activating the upgrade, so I had to manually run a script. Although the process completed with a few errors, Bullseye has always worked fine on Crostini since then.

#chromeOS #Linux

Discuss... Email | Reply @amoroso@fosstodon.org

Managing Lisp code in the residential environment of Medley differs from similar tasks in traditional file based Common Lisp systems.

In a previous post I explained how the residential environment of Medley works, discussed some of its facilities and tools, and introduced a workflow for managing Common Lisp code under the residential environment. This leverages the way Medley is designed to work and minimizes friction.

In this post I explain how to use Medley as a file based environment, albeit with some friction and reduced functionality.

I show how to edit, organize, and load pure Common Lisp files, i.e. source files not under the control of the File Manager. Pure files are ordinary text files that contain only Common Lisp code, with no metadata or font control commands like the code databases the File Manager maintains.

Motivation

Letting the residential environment track and control code is the most convenient and productive way of using Medley for Lisp development.

But you can still manually manage pure Common Lisp files. For example, you may want to use from Medley some code you mainly intend to load into the environment and don't change often. This is the case of external libraries and programs developed elsewhere, which you call from new code or run in Medley.

Using Common Lisp code not developed for the pre ANSI implementation of Medley may need adaptation. So let's see how to edit source files.

Editing source files

To create or modify pure Common Lisp files use the TEdit word processor of Medley, not the SEdit structure editor. SEdit doesn't directly manipulate the arbitrary, unstructured text of pure files.

TEdit can read and write ordinary text files, but not by default. To save a file as ASCII execute the command Put > Plain-Text. Execute Get > Unformatted Get to load such a file.

The main downside of using TEdit for Common Lisp is the program is not Lisp aware. It doesn't support automatic indentation, prettyprinting, perenthesis matching, and code evaluation. Another major problem is cursor keys don't work in Medley, which severely limits text editing with TEdit. The Medley team is well aware of the issue.

An alternative is to use your favorite Lisp editor to edit the Common Lisp sources outside of Medley and then load the files from Medley.

Format of Pure Common Lisp files

Although Medley can load pure Common Lisp sources, these files do have some minimal formatting requirements. A pure Common Lisp file must begin with a semicolon ; character, otherwise the system assumes it's an Interlisp file.

Other than that a pure file can contain any Common Lisp feature Medley supports, in any order the language allows.

Loading source files

Once a pure Lisp file is available, load it in Medley with the usual file loading forms such as CL:LOAD. The way symbols are interned depends on whether the file defines or makes current any Common Lisp packages.

Let's go over the main cases: the code has no packages; a single file contains both package and function definitions; the code is split between a file for the package and another for the function definitions.

No package definition

The simplest type of pure file contains no package forms, just function definitions and other Common Lisp expressions.

In this case Medley interns symbols in the USER package without exporting them. The Medley implementation of Common Lisp is closer to CLtLx than ANSI, so it provides the standardized packages LISP (nicknames: COMMON-LISP and CL) and USER instead of COMMON-LISP and COMMON-LISP-USER.

For example, suppose this file SQUARE.LISP defines a function calc-square to compute the square of its argument:

;;; SQUARE.LISP

(defun calc-square (x)
  "Return the square of X."
  (* x x))

After evaluating (load "SQUARE.LISP"), at the > prompt of a Common Lisp Exec you call the function using the proper package qualifier:

> (user::square 3)
9

If the USER package isn't adequate for your code, on Medley CL:LOAD also accepts the keyword parameter :package to supply a package to which *package* is bound when reading the file. For example, at the > prompt of a Xerox Common Lisp (XCL) Exec you can pass the package XCL-USER with a form like:

> (load "SQUARE.LISP" :package (find-package "XCL-USER"))

You must pass an actual package object to :package, not a package designator like :xcl-user.

Since XCL-USER is the default package of XCL Execs no qualifier is required for calling the function:

> (square 3)
9

Package and definitions in one file

Not using packages may be adequate for very small files. However, typical Common Lisp programs do define their own packages.

This example file PKGDEMO.LISP defines the package PKGDEMO with nickname PD that exports the function FUN, which just prints a message:

;;; PKGDEMO.LISP


(in-package "XCL-USER")

(defpackage "PKGDEMO"
  (:use "LISP" "XCL")
  (:nicknames "PD")
  (:export "FUN"))


(in-package "PKGDEMO")

(defun fun ()
  (format t "Hello from PKGDEMO:FUN."))

The file holds both the package definition and the rest of the code. It's important that the first form in the file be in-package to make current a known package like XCL-USER. XCL-USER is the Medley equivalent of COMMON-LISP-USER and uses the LISP package with the standard Common Lisp symbols.

After loading PKGDEMO.LISP from an XCL Exec with (load "PKGDEMO.LISP") you can call the exported function like this:

> (pkgdemo:fun)
Hello from PKGDEMO:FUN.
NIL

or via the package nickname:

> (pd:fun)
Hello from PKGDEMO:FUN.
NIL

Separate package and definition files

Unlike PKGDEMO.LISP in the previous example, most Common Lisp programs split the code between at least two files, one holding the package definition and the other the function definitions. Let's rewrite PKGDEMO.LISP by storing the package in the file PKGSPLIT-PKG.LISP:

;;; PKGSPLIT-PKG.LISP


(in-package "XCL-USER")

(defpackage "PKGDEMO"
  (:use "LISP" "XCL")
  (:nicknames "PD")
  (:export "FUN"))

The function definition goes into the file PKGSPLIT-FUN.LISP that begins with an appropriate in-package followed by the rest of the code:

;;; PKGSPLIT-FUN.LISP


(in-package "PKGDEMO")

(defun fun ()
  (format t "Hello from PKGDEMO:FUN."))

Finally, make sure to load the package first with (load "PKGSPLIT-PKG.LISP"), then the function with (load "PKGSPLIT-FUN.LISP"). Loading the files creates the same objects as the single file example PKGDEMO.LISP and you can call the function the same way:

> (pkgdemo:fun)
Hello from PKGDEMO:FUN.
NIL
> (pd:fun)
Hello from PKGDEMO:FUN.
NIL

#CommonLisp #Interlisp #Lisp

Discuss... Email | Reply @amoroso@fosstodon.org

One of the cool features of Lisp is examining and modifying a running program.

This allows, for example, to correct a bug by inspecting, editing, fixing, and resuming a program that breaks and lands in the debugger because of an error. To gain familiarity with the process, I recorded a screencast of a Medley session in which I use the debugger to fix a bug in a running Interlisp function and conclude the computation.

Medley provides advanced debugging facilities and tools in the Break Package, including the Break Window which is the main user interface of the debugger. “Package” as in module or subsystem, not Common Lisp package.

In the recorded session I fix this broken Interlisp function to compute the square of the argument:

(DEFINEQ (CALC.SQUARE (X)
  (* Return the square of argument X.)
  (TIMES X Y)))

The bug is trivial, a typo. The call to the TIMES multiplication operator passes Y as the second argument instead of the function parameter X.

I begin the recorded session by defining the function with the SEdit Lisp editor. Next, from the Exec (a Lisp REPL) I call the function (CALC.SQUARE 3) and get the error Y is an unbound variable. Then I execute the RETRY Exec command. RETRY evaluates the latest expression and forces entering the debugger if it yields an error.

The topomost few backtrace entries in the Break Window are internal functions called by CALC.SQUARE. From the Break Window's middle-click menu I invoke the REVERT command to move the point of execution back to the CALC.SQUARE call before the error.

This selects the CALC.SQUARE frame where the bug is most likely to be. Inspecting the bindings of the frame provides a clue.

X has the expected value 3 and I confirm it by evaluating X at the debugger REPL. But Y, the variable the error references, doesn't show up in the stack frame. Evaluating Y yields the same error. It's a hint Y shouldn't probably be there and is likely a typo. Therefore, I execute the debugger's EDIT command to open the code of the current function in SEdit. From SEdit I fix the typo and evaluate the modified definition.

The new definition is now in the Lisp image and the current stack frame is still the CALC.SQUARE call in the backtrace. The OK debugger command continues execution from the point of the break, letting the program run the corrected code. The Exec from which I originally called the buggy function finally returns the expected value of the square of 3: 9.

The REVERT, EDIT, and OK commands may be typed at the debugger REPL but I invoked them from the menu to make the menu itself and its options explicit. Similarly, the Done & Close SEdit menu command has the associated keychord C-M-x.

#Interlisp #Lisp

Discuss... Email | Reply @amoroso@fosstodon.org

My exploration of Medley as a Common Lisp development environment proceeds with setting up a workflow for writing and saving code.

The workflow consists of a series of steps in a specific order using appropriate Lisp REPLs and tools. It supports writing the simplest type of Common Lisp software, i.e. programs or libraries in a single package that exports some symbols. I'll eventually extend the workflow to more complex cases such as programs with more than one package.

Since the steps are not intuitive, especially for Medley novices, in this post I'll describe the workflow in detail. But first there are a few concepts to introduce.

Why do you need a workflow in the first place? Because the differences between Medley and modern environments constrain how and in what order Common Lisp code may be written and managed. Before examining the constraints let's describe the differences.

The residential environment of Medley

Writing single package programs is straightforward in modern file based Common Lisp environments. In a new file you just define the package with DEFPACKAGE, then in the rest of the file or at the top of a new one you place a matching IN-PACKAGE followed by the code.

Although it's technically possible to do the same in Medley this doesn't take advantage of its facilities, and you may actually need to fight the system to accomplish what you want. Indeed, Medley is not an ordinary environment. Not only it predates current Common Lisp implementations, it supports a different development process.

In file based Common Lisps you directly edit source files and evaluate or load the code into the running Lisp image.

Medley instead is a “residential environment” in which you edit and evaluate Lisp objects that reside in the image — hence “residential”. Then you save the code to files that are more like code databases than traditional source files.

You don't edit the code databases, which Medley calls “symbolic files”. Rather, you use the SEdit structure editor to modify the code in memory and the “File Manager” to save the code to disk. The beginning of a symbolic file defines metadata, the “file environment”, which describes the Common Lisp package and readtable associated with the code.

The File Manager, also known as “File Package” (not to be confused with Common Lisp packages), is a facility that coordinates the development tools and code management tasks. It notices the changes to Lisp objects edited with SEdit or manipulated in memory, tracks what changed functions and objects need to be saved to symbolic files, and carries out the actions for building programs such as compiling or listing them. The File Manager has some of the functionality of Unix Make.

Figuring what changed is easy in modern Common Lisp environments, as you know which files you edited and need action like saving or compiling. System definition tools like ASDF can track this for you. In Medley it's the File Manager which tracks the changes that take place in the running image and need to be synchronized to disk.

Motivation

How do the peculiarities of Medley constrain writing Common Lisp code and require a tailored workflow?

The functions and objects of Interlisp programs usually live in the same namespace of Interlisp and its tools. As a consequence, functions and Lisp objects may be mostly defined in any order and accessed without package qualifiers. No special handling is necessary with the File Manager either.

With Common Lisp code, however, a subtle complication arises due to packages and exported symbols.

A good explanation of why things are different, and how the File Manager and file environment interact, is in the documentation of TextModules, a tool for importing Common Lisp code created outside of Medley. Although in the context of TextModules, these remarks give an overview of the same issues the File Manager faces with other Common Lisp code. The TextModules chapter of the Lisp Library Modules manual says on page 311 (page 341 of the PDF):

It is important to separate the environment of the file from its contents because the File Manager (not TextModules) first reads all the forms in the file, and then evaluates them. Text based source files sometimes change the package as needed. This cannot work for the File Manager since the file's forms are all read and then executed, i.e. the package changes would not occur until after the entire file had been read, and forms after any IN-PACKAGE form would have been read incorrectly.

In other words, unless you define packages and access symbols in the proper order, you'll get subtle errors. The solution is a workflow that avoids such errors.

More information on dealing with packages in Medley is in the sources referenced in section “Documentation” of my post on using Common Lisp on Medley.

The workflow

How does the workflow order the development tasks to achieve its goal?

At any one time the workflow accesses only defined symbols and ensures the running Lisp image stays synchronized with the symbolic file. It's not the only or the best possible workflow, just one that works. I put it together after extensively reading the documentation and experimenting.

As I said in the overview of Medley as a Common Lisp environment, when coding in Common Lisp I use two Executives (Lisp REPLs), a Xerox Common Lisp (XCL) Exec and an Interlisp one.

The former is for testing, running, and evaluating Common Lisp code. I use the Interlisp Exec for running system tools and interacting with the File Manager. Since the tools and File Manager facilities are in the Interlisp package, referencing them from a Common Lisp Exec would require qualifying all symbols with the IL: package.

What follows assume you're familiar with basic Interlisp and File Manager features such as SEdit, file coms, FILES?, and MAKEFILE. If not I recommend reading the Medley primer, particulary Chapter 7 “Editing and Saving”. Also, unless otherwise noted, you should carry out the steps in sequence in the same Medley session.

Let's start.

Defining the file environment and a minimal package

Suppose you want to write a Common Lisp program or library stored in the file SINGLEPKG. The package SINGLEPKG, nicknamed SP, will export the two functions FUN1 and FUN2.

From now on, denotes the prompt of an Interlisp Exec and > that of a Xerox Common Lisp (XCL) Exec. Sometimes I'll tell you in which Exec to evaluate expressions.

The first step is to define the file environment. At an Interlisp Exec evaluate:

← (XCL:DEFINE-FILE-ENVIRONMENT SINGLEPKG :PACKAGE (DEFPACKAGE "SINGLEPKG" (:USE "LISP" "XCL")) :READTABLE "XCL")

For now don't use other packages or export any symbols, just enter the form as is.

Although the file environment references package SINGLEPKG, the package doesn't exist yet in the running image. To synchronize the image with the file environment evaluate the definition of a minimal package from an Interlisp Exec:

← (DEFPACKAGE "SINGLEPKG" (:USE "LISP" "XCL"))

Next, from an Interlisp Exec evaluate (FILES?) and, when asked where the SINGLEPKG file info should go, respond yes, enter SINGLEPKG as the file name, and confirm the creation of the file.

Defining the first function

Everything is ready to define the first function FUN1. At an Interlisp Exec call SEdit with (ED 'SINGLEPKG::FUN1 '(FUNCTIONS :DONTWAIT)) and select DEFUN from the menu. Enter the code of FUN1:

(DEFUN FUN1 ()
  (FORMAT T "Hello from FUN1."))

Save and exit with Ctrl-Alt-X and test the function at an XCL Exec (an Interlisp Exec will do too):

> (SINGLEPKG::FUN1)
Hello from FUN1.
NIL

It works, so at an Interlisp Exec evaluate (FILES?) to associate FUN1 with the file SINGLEPKG.

Completing the package definition

The Lisp image contains the new symbol FUN1 in package SINGLEPKG but there's no symbolic file yet, let alone an exported symbol in the file. Therefore, to keep things in sync you need to update the package definition by exporting the function name. This is also an opportunity for adding the SP nickname to the package.

At an Interlisp Exec evaluate (DC SINGLEPKG) to open the file coms in SEdit. Just after the XCL:FILE-ENVIRONMENTS form enter:

(P (DEFPACKAGE "SINGLEPKG"
     (:USE "LISP" "XCL")
     (:NICKNAMES "SP")
     (:EXPORT SINGLEPKG::FUN1)))

Both colon characters : are required in the function name. The P File Manager command tells the system to execute the following Lisp expressions at load time, so loading SINGLEPKG will define the package and export the symbol.

Save and exit with Ctrl-Alt-X and, at an Interlisp Exec, save the file with (MAKEFILE 'SINGLEPKG) to reflect the updated definitions. This creates the file SINGLEPKG on disk.

Before doing anything else it's better to make sure the package is properly defined and the function exported.

The most reliable way is to exit the Medley session with (IL:LOGOUT), start a new session and, from an Interlisp Exec, evaluate (LOAD 'SINGLEPKG). If there are errors you will have to go back and check whether you went through all the steps correctly. You may need to delete the latest or all versions of the file SINGLEPKG.

Assuming there are no errors the package is properly defined and the function exported. To double check, from an XCL Exec call the exported function like this:

> (SINGLEPKG:FUN1)
Hello from FUN1.
NIL

Since it works you may proceed development by editing the existing function or defining new ones. In the former case you employ SEdit, FILES?, and MAKEFILE as usual. Just make sure to pass to ED the package-qualified function name like (ED 'SINGLEPKG:FUN1 :DONTWAIT).

Things change if you want to define new functions.

Defining more functions

As planned you proceed to define a second function FUN2 exported from package SINGLEPKG. If you're still in the Medley session in which you've just loaded the file SINGLEPKG, continue from there. Otherwise start a new session and load the file with (LOAD 'SINGLEPKG) from any Exec.

At an Interlisp Exec define the new function with (ED 'SINGLEPKG::FUN2 '(FUNCTIONS :DONTWAIT)) and select DEFUN from the menu:

(DEFUN FUN2 ()
  (FORMAT T "Hello from FUN2."))

Save and exit with Ctrl-Alt-X and test the function at an XCL Exec:

> (SINGLEPKG::FUN2)
Hello from FUN2.
NIL

Call FILES? to associated FUN2 with file SINGLEPKG.

As already done for FUN1 you need to modify the package definition to export the new function. At an Interlisp Exec edit the file coms with (DC SINGLEPKG) and add the symbol FUN2 to the export clause, which will now look like this:

(P (DEFPACKAGE "SINGLEPKG"
     (:USE "LISP" "XCL")
     (:NICKNAMES "SP")
     (:EXPORT SINGLEPKG:FUN1 SINGLEPKG::FUN2)))

SEdit replaced the double colon of SINGLEPKG::FUN1 with a single colon as in SINGLEPKG:FUN1. But you still have to type :: for FUN2 because FUN2 hasn't been exported yet.

Save and exit with Ctrl-Alt-X and, at an Interlisp Exec, save the file with (MAKEFILE 'SINGLEPKG).

To synchronize the Lisp image with the symbolic file, which will allow to call the exported function as (SINGLEPKG:FUN2), either load the file SINGLEPKG from a fresh session or, just after editing the file coms, evaluate a revised package definition at an Interlisp Exec:

← (DEFPACKAGE "SINGLEPKG"
    (:USE "LISP" "XCL")
    (:NICKNAMES "SP")
    (:EXPORT SINGLEPKG::FUN1 SINGLEPKG::FUN2))

Either way, to check that everything works evaluate at an XCL Exec:

> (SINGLEPKG:FUN2)
Hello from FUN2.
NIL

Success!

For every new function or Lisp object you want to export from the package, go through the steps of this section again, making sure the Lisp image and the symbolic file stay synchronized. The steps are, in order:

  1. edit the new function
  2. call FILES? to tell the File Manager about the function
  3. edit the file coms to update the package
  4. save the file with MAKEFILE
  5. evaluate the revised package definition

That's all, you're finally ready to develop single package programs. This workflow may seem convoluted at first but things will come more natural as you gain experience with Medley.

#CommonLisp #Interlisp #Lisp

Discuss... Email | Reply @amoroso@fosstodon.org

Enter your email to subscribe to updates.