Modern, functional Common Lisp: myths and best practices

Generally, here's how to connect to a remote running lisp image, and send changes from one's comfortable editor: https://lispcookbook.github.io/cl-cookbook/debugging.html#re...

With k8s, you just create a service of type NodePort and it assigns a free external port for that service, which is mapped back to a specific port in the Lisp container.

kubectl port-forward my-lisp-pod 4005:4005

then M-x slime-connect localhost 4005

Outside of k8s, which I'm not familiar with, ssh can do port forwarding.

ivan4th had the right answer. Use the "kubectl port-forward" command.

BTW, here's a k8s-hosted app I wrote using this approach: https://github.com/atgreen/red-light-green-light

You are correct, of course. Java, Python, C++, etc. also have huge ecosystems.

CAPI is great, very well designed and implemented.

The advantage of Swift and SwiftUI is that my application can run on both a MacBook and iPad, sync data with iCloud.

The advantage of LispWorks is that I could fairly easily support both Mac and Windows (and Linux).

I spent two evenings last week playing with Clozure CL and its Cocoa support. It has low level APIs and I had a peculiar issue with sometimes not being enter text into input components.

EDIT: to be clear, in general I love Clozure CL. In this instance I was probably shooting myself in the foot, somehow, or maybe the issue was that I blew the install in some way getting it running on Catalina which took over an hour. The Catalina problems are hopefully just short term issues.

I would far rather have CL's explicit representation of the different types of equality, than many languages approach of either a) "you can't do that" or b) just getting it wrong sometimes. So I would hardly call that behind the times, although the interface could be nicer I suppose.

I recommend Fset, which other commetners have mentioned.

I wrote a wrapper for it and SERIES, called folio, followed by a revised version called folio2. If your requirements resemble mine when I wrote it, you might findit useful.

You can find it here:

  https://github.com/mikelevins/folio2/

What I do: Write pure functions, but don't shy away from mutating lexically scoped state in order to improve performance. For global stuff, wrap it with classes since objects are a commonly understood way to reason about state.

It's a matter of not modifying things once constructed. CL has hooks to help do that, although a user can always get around them.

A package I've been involved with lately is fset, which is available through quicklisp, or at

https://github.com/slburson/fset

It has some interesting features, including "functional setf expansion". This would turn something like

(setf (fcar x) y)

into something equivalent to

(setf x (cons y (fcdr x)))

(where "fcar" and "fcdr" are the same as "car" and "cdr", except when they are in a setf-able place form.)

(fcar and fcdr are not in fset; I used those names just for exposition here.)

It could work with nested accessors, but only if it bottoms out in a variable.

https://github.com/smithzvk/modf like setf, but doesn't mutate the data (didn't try).

https://common-lisp.net/project/fset/

By the way, I found a lot of your best practices questionable to say the least. Since it's obvious that you are a newcomer to CL, I would refrain from producing "best practices" type blog posts until I had a few years of experience under my belt.

For that reason, I also found your post confusing and I'm inclined to categorize it as "mostly rehashing stuff that is already there" rather than strong signal. There is ample, good, introductory material for CL on the net, we should strive to think before we dilute it with derivative posts.

They’re not? Take a look at at a Final Cut Pro.

I don’t like this argument and paints a picture that Lispers are overly concerned by some academic aspect of programming. Many people, myself included, use Lisp because it lets us get something done more quickly, more efficiently, more easily, or all three.

It’s true. Importing flask in Python and starting a server takes about 1/2 as much code as in Lisp. Some extraordinarily routine stuff Python has down to a one-liner. A lot of people have also written more Python libraries, so chances are you’ll be able to cobble any random doodad together with a huge dependency tree. But after a certain very short prototyping period, I end up fighting Python’s terrible deployment, terrible efficiency, and slapdash language implementation when trying to build something robust.

I can only speak from the perspective of using Clojure, I toyed around with Racket and have no experience with CL.

That said, I disagree with some of the points you make:

> availability of useful libraries

Clojure is embedded in both JS runtimes and the JVM. Both feature massive ecosystems.

In terms of getting things done:

Languages like Python, JS and so on are fantastic at plumbing. But when it comes to modelling data-structures, manipulation and algorithms then they are far less expressive and productive than Clojure from my experience.

I personally think the main issue really is the learning curve. There are quite a number of things that 'suck' about Clojure initially:

1. In a Lisp you are essentially manipulating an AST rather than writing statement-based text. Initially this is cumbersome and taxing both while reading and writing programs.

2. In a functional Lisp like Clojure (and other FP languages) there is a wide variety of commonly used functions which are used to manipulate and compose data-structures and other functions. Especially when reading code this can be daunting.

3. Setting up a environment correctly for REPL driven development as a beginner and tuning it as an intermediate user is quite an undertaking in comparison to many other languages.

4. Clojure specifically being a hosted language forces you to understand the hosted ecosystem as well plus the wiring between the host and Clojure.

The payoff for all those four points is worth it:

1. Search for paredit visualizations like this one: http://danmidwood.com/content/2014/11/21/animated-paredit.ht.... Manipulating code this way requires mechanical prowess and exercise but scales up really nicely, especially alongside better understanding of Lisp code in general.

2. The incredible variety of functional building blocks scales really well with your experience and understanding. You are programming with a series of descriptive expressions, rather than lower level statements. Code becomes more declarative, dense and expressive. Abstraction is much more fluent.

3. A nicely set-up environment enables very fast development cycles and allows for understanding pieces of code of any scope in isolation, because you can evaluate, change and test any expression instantly.

4. This is in my experience a necessary evil to get higher adoption and a the massive library ecosystems of JS and Java.

I assume there are a lot of capable developers who simply cannot get over 1 possibly while being pushed back by 2-4. And I have to admit: If I wasn't already infected with Lisp at a younger age, I probably wouldn't have bothered with Clojure or any Lisp. But now it is my favorite for manipulating and modelling data AKA doing 'business-logic' and transformations between APIs.

The above post didn't say unpopular, they said 'underpopular'. While that's a bit of a neologism, it just implies that it's less popular than it should be.

Also: "Lisp is the most fun you can have programming, yeah from the start," while subjective, is by most accounts, wrong. Especially when referring to Common Lisp; there could hardly be a more convoluted Lisp than Common Lisp.

Chronically popular? Unless you're using some weird definitions, I would expect to see Lisp used more if you were correct.

If I were doing the above a lot, I'd write compose as a macro (or better yet a compiler macro). In that case, the only thing above that needs to be changed is the setq, which would have to be written as

  (setf (fdefinition g) (compose f f))

Programming with a lot of higher order functions won't feel as clean in Common Lisp due to the #' and funcall.

Basic Lisp programming with lists is a lot cleaner in a Lisp in which the empty list is false, and in which accessing nonexistent parts of a list (including the empty list) is a safe no-op that yields nil.

Ashwin Ram's (cdr (assq key a-list)) almost works in Common Lisp in the form of (cdr (assoc key a-list)). See: https://ashwinram.org/1986/01/28/a-short-ballad-dedicated-to...

Imperative programming is better supported in Common Lisp because the evaluation of the arguments of most forms, including function calls, is ordered, mainly left to right, so side effect embedded in expressions will show stable, portable behavior. Scheme function calls have unspecified evaluation order, much like C. (In Common Lisp, the only unspecified aspect is whether the function cell is sampled before the arguments are evaluated, or just before the function is called. It's extremely rare for the arguments of a function call to be redefining the function cell, needless to say.)

Common Lisp forms return a predictable value. If it doesn't make sense for a form to return a value, its value is not "undefined", but either just nil or else "no values". When a value returns no values, and an attempt is made to use its value anyway, then nil is produced as the value. You will not see some annoying #<undefined> in a Lisp REPL coming from a procedural construct.

I like it. But the really nice thing about lisp is if there's something I dont like, I can just hack the evaluator to fix it.

CL compiles down to the metal in most cases (ABCL being a notable exception). I've written device drivers in CL, realtime code in CL, and many CL implementations even allow you to write in assembler. I've done that too, in order to take advantage of SIMD hardware. The above would be -- problematic -- on top of the JVM.

There also might be an issue of speed, since CL is compiled to the metal. Modern JIT JVMs probably have mostly eliminated CL's speed advantage, but I'd still be surprised if some things in SBCL (arguably the best-optimized CL) didn't run faster than in Clojure.

Yeah, that's why the company I work for lets me use Clojure occasionally; we have a metric ton of libraries written in Java and being able to interop with them more-or-less seamlessly is a huge selling point for Clojure.

Most adequate for what? Peer acceptance?

Thanks for the links! I'll check them out. I like the "higher level" of Python libraries better than C or even Java, because I tend to write what most people would call prototype code. Small user base, small applications, and the really harsh constraint is development time from concept through validation.

No-doubt-unnecessary boilerplate: I understand that people have language wars, and it's possible you may have identified me as belonging to one side or another, but I'll pass. That whole state of affairs is just unfortunate.

In addition, check out py4cl: https://github.com/bendudson/py4cl

(And a Numpy clone: https://numcl.github.io/numcl/)

Thanks! Nothing would be more awesome than Lisp (or Scheme, I don't care) with easy transparent access to modern libraries.

Yeah, I read it. Can you explain how "red herrings" differ from "myths"? Is the distinction really that important?

My point was that it is not just myths (or red herrings either) that are holding Lisp back. There are some issues based in reality as well.

Offhand, recently I've used NumPy, telnetlib, openpyxl, TkInter. I haven't used Lisp for a very long time, because of my (perhaps faulty) perception that some of these things (and the next one I will need) might not be there, or might be less usable than their Python counterparts.

Yeah, I agree save-lisp-and-die is cool. On the other hand you can just write a python script, copy it over and have it run on literally millions on computers out of the box (even accounting for the python3 debacle) – that's pretty handy, too.

Isn't Common Lisp's save-lisp-and-die roughly analogous Pyinstaller? I mean they both work but nobody seems to be using them. In my experience Common Lisp code is most commonly deployed with an interpreter.

CLI utils: you're missing the point. Of course you can write some simple terminal apps with common lisp, but can you name a single one that anyone who is not a lisp fanatic uses? If not, ever wonder why?

GUI: would you develop a commercial app (to pay your rent) in any of these apart from CAPI?

ML: You are joking, right?

addendum: for iOS, Android and mac: possible MOCL (proprietary) https://github.com/CodyReichert/awesome-cl#implementations

Thanks for the list! :)

If you think spamming random links to old toy/dead/in-house projects you found on Cliki and that contained words like "machine learning", "Games" or "GUI" (mcclim – really?[]) constructs an effective rebuttal to "no ecosystem to speak of", I'm not sure I know what to say. But I'll go through your list in more detail if you want me to.

[] For the record CLIM had interesting ideas, but last I looked, calling McCLIM's implementation of them a toy would be charitable.

Ideology aside, most proprietary compilers (mostly for Forth, Pascal and Lisp) I have used are of _excellent_ quality.

A lot also provide you the source code once you pay. AllegroCL is one of them I think.

The FOSS experience is a poor imitation of Lisp Machines.

It's a very important implementation detail. Explicit loops tend to imply mutation, which is contrary to idiomatic functional programming. Recursive calls don't require mutation but do require TCO to achieve equivalent space complexity. Constant-factor optimizations are one thing but failing to perform TCO turns constant-space algorithms into linear-space algorithms (or linear ones into quadratic, etc.). It's less a matter of "optimizing" the calls and more a matter of not wasting limited stack space on data which is clearly not required to execute the remainder of the program. One might as well label the practice of freeing stack frames when a function returns "Function Return Optimization" (FRO) and consider it a mere "implementation detail". After all, wouldn't it be much simpler to grab new memory every time the program needs some storage space and never bother with cleaning it up? It would certainly make debugging easier with all those old variables retained for the life of the program and not constantly overwritten by new data. However, programs written for a language without guaranteed "FRO" would look very different from normal programs, much as programs designed to compensate for the lack of guaranteed TCO look very different from idiomatic functional programs.

Haskell uses a different (data-centric, non-strict) evaluation model where recursive definitions don't result in recursive calls, so traditional TCO isn't as relevant. Recursion is used very heavily in Haskell—which has no first-class looping constructs—but the resulting programs generally do not require large stacks. It's not unusual to be able to run even large Haskell programs with a 1 KiB maximum stack size (+RTS -K1k). Space leaks are possible, of course, but they take place in the heap.

Haskell implements call-by-need reduction, in which all calls are tail calls.