Why we need Lisp machines

> We don't "need" Lisp machines. We "need" Lisp software. What made a Lisp machines extraordinary wasn't the hardware, it was the software. Nothing today is impeding one from writing such software, except time, energy, willpower, and/or money.

Discussed here https://news.ycombinator.com/item?id=30800520 The main issue is that Lisp, for all its inherent "power", has very limited tools for enforcing modularity boundaries in code and "programming in the large". So everything ends up being a bespoke solo-programmer project, there is no real shared development. You can see the modern GC-based/"managed" languages, perhaps most notably with Java, as Lisps that avoided this significant pitfall. This might explain much of their ongoing success.

The greatness of Lisp -at least when it comes to end-user empowerment- and (I think) the only differentiating factor that most other languages have still not caught up to, is the cybernetic philosophy with its roots in Licklider (man-computer symbiosis) and Engelbart.

Building an environment that strongly and uncompromisingly expresses this philosophy at its core is a serious undertaking in terms of time investment. Emacs has been in continuous development for 37 years and while it is still not as good as Genera, it's certainly "good enough" for lots of people and definitely captures the spark of this philosophy.

In the Common Lisp world, we've had plenty of tries (MCL, Hemlock, McCLIM) but they've all failed to get people to coalesce and generate progresss.

Maybe the fact that Emacs exists is a curse in that people realize the barrier they'll have to clear to make something substantially better and decide to devote their energies into more easily realizable efforts.

Anyone interested in the computing holes that can be filled by lisp machines should check out Urbit. There is an vibrant and growing community of people building a network of personal servers running a lisp-like functional OS. It uses an identity system secured by the Ethereum blockchain and it has created a bottom up economic incentive for developers to participate. They are starting to solve unique problems that couldn't be addressed on currently prevalent platforms. Urbit is an affirmation; we can ditch the nihilism.

> Yet, it appears the median and mean Lisp programmer is producing Yet Another (TM) test framework, anaphoric macro library, utility library, syntactic quirk, or half-baked binding library to scratch an itch. Our Lisp programming environments are less than what they were in the 80s because everybody feels the current situation with SLIME and Emacs is good enough.

I don't think this is true. Not anywhere close. Most such examples are small, and probably only took a small number of hours to produce. While "superlative" stuff takes very many man hours to create. So just by seeing that there are many throwaway testing frameworks or whatever, you cannot tell where most of the work hours actually go. A half baked binding library takes 20 minutes to make, while a proper high quality rendering engine takes hundreds if not thousands of hours.

The Lisp population is thin on people making cool shit because the Lisp population in general is thin.

I'd say it's both. It seems most lisping is done high on the stack. Some are doing assembler level lisp (or scheme) but less bare metal / OS / system oriented lisp.

I wonder what the lisp os guy are thinking about OS / UI these days.

Personally, I think the problem is that CommonLisp is just another programming language, whereas Lisp really shines when it provides a full-fledged programming environment. Nowadays, it would seem best to create such an environment on top of commodity hardware as a "virtual machine" that abstracts away from the hardware in a general, portable way. However, a good environment (1) needs a purpose, and (2) somebody needs to write it. Lisp currently fails on both points. The purpose used to be symbolic AI and NLP among other things. Nowadays it could be the same, or a web framework with integrated distributed computing and database, or scientific computing, or a server for certain business services, etc. There are many application domains for which a virtual "Lisp machine" would be beneficial but it needs to be developed for one of those real world applications, not just as a toy like existing attempts of building Lisp machines. And in my opinion the problem really is (2), developer power / size of the community. If you exclude super-expensive legacy Lisps, the current Lisp community doesn't even have a fully supported Lisp-native editor (portable Hemlock is not good enough) and also doesn't have good enough object persistence / native Lisp-based databases. Both are the foundations of any integrated Lisp machine.

People sometimes claim CL+Emacs+Slime provides the full interactive experience. I just can't agree with that at all. I have tried, and the experience was not substantially different from Go development and development in any other fast-compiling language with Emacs. In some respects, it's even worse than with various modern languages, even though most of those languages are strictly inferior to CL from a pure language perspective. If editing files in Emacs is all there is to the allegedly great Lisp experience, and developers at the same time have to deal with all those idiosyncrasies of CL such as CL's filesystem path handling, ASDF, and tons of poorly documented libraries, then I can't really see the advantages of CL. The language is powerful, don't get me wrong, but a truly interactive experience is totally different. Smalltalk managed to keep this experience but for some reason the Lisp community seems too have lost this vision. I guess the developer community is just not large enough.

Anyway, before someone tries to build another "close to metal" Lisp machine or tries to revive any old Lisp machine, I'd rather wish the community would focus on creating truly integrated development environments that abstract away from the host system and are fully hackable and editable from the ground up while maintaining security and multi-user access. A "virtual Lisp" machine with virtual OS, so to say. If that's developed for a certain purpose like building and deploying extremely portable web applications, I believe it can have a great future.

Sorry for the long rant. This is just my impression after having programmed in various Lisp dialects for the past three decades.

> "We don't "need" Lisp machines. We "need" Lisp software."

Nobody goes into Java because their self identity is "a Java programmer" to gather a team of people to create a Java machine running Java software to unleash the power of Java for the masses by enabling them to do everything in Java for the sake of doing everything in Java, By Java, With Java, For Java. And if they do talk like that they would be a Sun Microsystems marketing pamphlet from 1999, or a joking reference to Zombo.com, or suspected of having zero interesting ideas and defaulting to Ouroboros-naval-gazing.

Adobe Photoshop Lightroom is C++ and Lua. Blender is C++ and Python. Excel is C++ and Visual Basic for Applications. LibreOffice Calc is C++ and Python. These are large, popular, programmable systems which exist today and are good enough; good enough for people to spend lots of money on them, good enough for people to spend years skilling up in them, good enough that once they existed people wanted them to keep existing and they haven't faded into the past.

The added allure of an imaginary rebuild of them like "if you took the lid off Excel you'd see VBA inside so you could rework the way it handles multiple sheets using only a skill you have and software design skills and Excel-internals knowledge you don't have" would get a hearty side-eye and slowly backing away from most Excel users. "Everything inside looks the same" is as attractive as "if you open your car trunk you'll see leather seats and plastic dashboard components making it move" or "if you sit in this car you're sitting on engine parts because the top priority is that a welder in a scrapyard can build the entire car without leaving their comfort zone". There are certainly people who want that, but the way the world hasn't developed that way suggests it isn't particularly desirable. Even when such things have been built, people can today use a Smalltalk, an APL, save their running work in a memory-dump and reload it and rewrite parts of it in itself, people flocked to Jupyter notebooks instead.

> "[1] Kandria is a neat platformer developed entirely in Common Lisp"

https://cdn.akamai.steamstatic.com/steam/apps/1261430/ss_a3f...

Without mocking a team who has built, developed, polished and planned to release a project, because that is respectable, it also looks like computer gaming of the early to mid 1990s Intel 386/486 era; remeniscent of Prince of Persia, Gods, Lemmings, Monkey Island. But it needs an Intel i5, 4GB RAM and 1GB storage. It's not even released yet and has no reviews, but you describe it as 'superlative' ("Of the highest order, quality, or degree; surpassing or superior to all others") - are you rating it so highly based on it being written in Lisp or what?

You are aware that Lisp machines understood several different flavors of Lisp? The Symbolics ones understood Zetalisp and Common Lisp at least. Were they on the market today they could be convinced to run Clojure and Scheme as well. There are a few old-timers developing Common Lisp applications to run on modern hardware, using Symbolics hardware.

In fact, Symbolics shipped compilers for other non-Lisp programming languages, including C and Ada. These interoperated smoothly with Lisp code, much more so than they do under Unix. In this demo, Kalman Reti compiles a JPEG decoder written in C, and replaces the Lisp JPEG decoder that came with the system with it, yielding a performance boost:

https://www.youtube.com/watch?v=o4-YnLpLgtk

> In fact I conjecture that this is the reason Unix is more popular than Lisp -- because Lisps don't interoperate well. They haven't built up a big ecosystem of reusable code.

And why are there so many? IMO the language is too flexible for its own good. It promotes this curious intellectual solo-competition where you try to prove you are worth of the elite who get all the fancy FP stuff and make all this bespoke functionality.

It's almost impossible for Lisp to be popular. To be popular means a lot of people use it, but that means it can't be complicated much above the median. But because it lets individual programmers push intellectual boundaries it self-selects itself out of this pool. Any big collaborative project will attract this type of developer and soon the lesser developers (who don't dare object for fear of appearing dumb) are not able to contribute.

Just my opinion, if a little dramatic.

> Honest question: how do you communicate between two Lisp processes on two different machines? I know Clojure has EDN (which is sort of like JSON : JavaScript), but I haven't heard of the solutions for other Lisps.

Probably TCP or UDP based protocols like essentially every cross-network communication in every language today.

EDIT: Also, it should be noted that JSON does not, itself, allow you to communicate across a network. It's just a serialization format. You still have to select some protocol for actually doing the communication. If your answer to the question "How do you communicate between two JavaScript processes on two different machines?" is "JSON", you've failed at actually answering the question.

The canonical Lisps still widely used today are Common Lisp, Scheme and Emacs Lisp. They all belong in the same family, and syntax / semantics are close. Porting code from Scheme to Common Lisp can be a lot easier than going from Python 2 to Python 3.

Clojure is something else entirely which is why a lot of people don't consider it a Lisp.

> Honest question: how do you communicate between two Lisp processes on two different machines?

If you want to use built-in object serialization, there is print and read.

An unappreciated means of code reuse under *nix is the static and dynamic library. This seems to be the go-to whenever you need something more involved than simply reusing a full binary via pipes.

> Honest question: how do you communicate between two Lisp processes on two different machines?

Depends what level of integration you want: custom tcp/udp protocols, HTTP and websockets are all pretty easy. But, you can also use something like this over a trusted network/vpn: https://github.com/brown/swank-client

> how do you communicate between two Lisp processes on two different machines?

Same as every other language: you pick a protocol and use it on both sides. Many of us already have enough JSON in play that it makes sense to start there.

> Honest question: how do you communicate between two Lisp processes on two different machines?

using continuations with sexps is insanely powerful

send (+ 1 3 (lambda (x) `(handle-response, x)))

What one could do was running hardware Lisp Machines from Symbolics on VME boards inside a SUN: the UX400 and UX1200.

Later Open Genera was sold as a Virtual Lisp Machine running on a DEC Alpha / UNIX system.

>> UNIX isn’t good enough anymore and it’s getting worse

>Why exactly?

Personally? We're in a bit of a transition point, and a lot of the technologies aren't working together like they used to.

An example, on my laptop I want to run android apps. The way to do this that actually works well (waydroid) only supports wayland. Unfortunately I use x2x to control another display remotely, and x2x doesn't work properly under wayland, and never will due to wayland's security choices.

So like, what am I supposed to do here? Not run android apps? Not use tools like barrier/synergy/x2x?

This is one of many many frustrations I've had from this new generation of wayland/systemd/etc. Hopefully it gets better eventually but it does feel a lot like the rug is constantly being pulled out from under me for no good reason...

Now I don't think a lisp machine is going to fix that mind you, but it is a concern.

Xerox PARC workstations could run Interlisp-D, Smalltalk, Mesa/XDE, Mesa/Cedar, thanks to this little thing RISC failed to kill, microcoded CPUs.

> > UNIX isn’t good enough anymore and it’s getting worse

> Why exactly?

Two reasons:

1 - systemd (which is moving linux towards becoming a systemd OS)

2 - developers and companies moving ever more towards web apps (which will eventually make the underlying OS irrelevant, as the browser becomes the OS) (incidentally, web assembly seems to herald the end of the open/transparent web too, as we're eventually all be running opaque binary blobs on our web browsers)

There's a old book all about just that, which included and popularized Richard P. Gabriel's paper, "The Rise of Worse Is Better":

https://en.wikipedia.org/wiki/The_UNIX-HATERS_Handbook

https://web.mit.edu/~simsong/www/ugh.pdf

>The year was 1987, and Michael Travers, a graduate student at the MIT Media Laboratory, was taking his first steps into the future. For years Travers had written large and beautiful programs at the console of his Symbolics Lisp Machine (affectionately known as a LispM), one of two stateof-the-art AI workstations at the Lab. But it was all coming to an end. In the interest of cost and efficiency, the Media Lab had decided to purge its LispMs. If Travers wanted to continue doing research at MIT, he discovered, he would have to use the Lab’s VAX mainframe.

>The VAX ran Unix.

>MIT has a long tradition of mailing lists devoted to particular operating systems. These are lists for systems hackers, such as ITS-LOVERS, which was organized for programmers and users of the MIT Artificial Intelligence Laboratory’s Incompatible Timesharing System. These lists are for experts, for people who can—and have—written their own operating systems. Michael Travers decided to create a new list. He called it UNIXHATERS:

    Date: Thu, 1 Oct 87 13:13:41 EDT
    From: Michael Travers <mt>
    To: UNIX-HATERS
    Subject: Welcome to UNIX-HATERS

    In the tradition of TWENEX-HATERS, a mailing list for surly folk
    who have difficulty accepting the latest in operating system technology.
    If you are not in fact a Unix hater, let me know and I’ll remove you.
    Please add other people you think need emotional outlets for their
    frustration.

https://www.goodreads.com/en/book/show/174904.The_UNIX_Hater...

https://wiki.c2.com/?TheUnixHatersHandbook

>I'm a UnixLover, but I love this book because I thought it was hysterically funny. Many of the war stories are similar to experiences I've had myself, even if they're often flawed as a critique of Unix itself for one reason or another. But other UnixLovers I've loaned the book to found it annoying rather than funny, so YMMV.

>BTW the core group of contributors to this book were more Symbolics Lisp Machine fans than ITS or Windows fans. ITS had certain technical features superior to Unix, such as PCLSRing as mentioned in WorseIsBetter, but having used it a bit myself, I can't see that ITS was superior to Unix across the board. The Lisp Machine on the other hand, although I never used it, was by all accounts a very sophisticated environment for programmers. -- DougMerritt

https://news.ycombinator.com/item?id=13781815

https://news.ycombinator.com/item?id=19416485

>mtraven on March 18, 2019 | next [–]

>I founded the mailing list the book was based on. These days I say, Unix went from being the worst operating system available, to being the best operating system available, without getting appreciably better. (which may not be entirely accurate, but don't flame me).

>And still miss my Lisp Machine. It's not that Unix is really that bad, it's that it has a certain model of computer use which has crowded out the more ambitious visions which were still alive in the 70s and 80s.

>Much as the web (the Unix of hypertext) crowded out the more ambitious visions of what computational media for intellectual work could be (see the work of Doug Engelbart and Ted Nelson). That's a bigger tragedy IMO. Unix, eh, it's good enough, but the shittiness of the web makes humanity stupider than we should be, at a time when we can ill afford it.

https://medium.com/@donhopkins/the-x-windows-disaster-128d39...

>> UNIX isn’t good enough anymore and it’s getting worse

> Why exactly?

Beside the defects well stated in the Unix Hater's Handbook, unix violate it's own principles since many years. Original unix idea was: desktops like Xerox SmallTalk workstations are too expensive and complex for most needs, so instead of a real revolution of an extraordinary outcome we decide to limit ourselves to most common needs in exchange of far less costs. No GUIs, no touchscreen, no videoconferencing and screen sharing [1] just a good enough CLI with a "user language" (shell scripts) for small potatoes automation and a bit of IPCs for more... Well... For more there is a "system language" (C) that's easy enough for most really complex task.

That was a success because no one really like revolutions and long terms goals especially if they demand big money while many like quick & done improvements at little price.

However in few years unix start to feel the need of something more than a CLI and some GUIs start to appear, unfortunately differently than original Xerox&co desktops those UIs were not "part of the system, fully integrated in it" but just hackish additions with so interoperability, just single apps who have at maximum cut&paste ability.

> Sure, but it also requires rewriting a lot of these things, introducing and fixing new bugs... It feels like the good ol' "let's rewrite this program" that quite frequently doesn't live up to the expectations

We need desktops again, witch means not just "endpoints" or "modern dumb terminals of modern mainframes named cloud", but desktop computing, since desktop development is essentially abandoned since many years and even back then was in a bad shape we need to restart from the classic desktops. LispM was ancient, hackish, but are still the best desktop we have had in human history so a good starting point. We have some kind of LispM OS/OE here: Emacs, still alive and kicking so there is something to work with, that's is. Emacs is already a WM (EXWM) have countless features and it's already "plugged" in modern bootloader OSes to have hw, driver and services. It just need to evolve.

[1] yes, you are reading correctly and no, I'm not wrong, I'm talking about the famous NLS "Mother of all the Demos" from 1968 https://youtu.be/yJDv-zdhzMY

I agree, especially with the statement that Unix isn’t good enough and getting worse.

I feel like that was one of the core assumptions and point of the article, but it didn’t have any explanation beyond “multiple programming languages.” Feels a bit flat to me.

Before Lisp Machines were killed in the market it was clear that new architectures were needed and a few were under development, even RISC like CPUs. They weren't released.

But Lisp at that time was already fast enough on standard RISC chips (MIPS, SPARC, ALPHA, POWER, ...). Later the 64bit RISC chips also provided enough memory space. SPARC also had some tricks for Lisp implementors.

Currently the assembler coded Ivory emulator is 80 times faster on Apple's M1 than the last Ivory hardware (the Ivory Microprocessor from Symbolics was released end 80s).

Speed is relevant for some use cases, sure, but not at all for a ton of others. Memory, disk and CPU are almost free in this new world, so why are we computing like it's 1990 still? It's time for some different abstractions than file -> process -> file.

The vast productivity gains of Smalltalk and Lisp were because they discarded those abstractions and programmers were free for others.

Presumably OP posted this after noticing Phantom came up a few days ago. https://news.ycombinator.com/item?id=30807668

The RISC-V folks are working on additions for special support of "complex language runtimes". Pipelined, SIMD and superscalar are all well and good, but what kills pure software-side support is always heavy branching and dispatching. These operations are genuinely much faster and more power-efficient when implemented in hardware.

How is the ARM not a "JavaScript Machine"?

https://stackoverflow.com/questions/50966676/why-do-arm-chip...

>Why do ARM chips have an instruction with Javascript in the name (FJCVTZS)?

https://community.arm.com/arm-community-blogs/b/architecture...

> Lisp is not type safe.

It is type safe. While Lisp is not statically typed, its typing discipline is strong: operations performed on incompatible types signal recoverable errors.

> Lisp is not type safe.

Typed Racket is, and that is why I love it

> Testing new changes requires more time than rebooting.

no it doesnt

> Lisp is not type safe.

yes it is

> And with a single address space you have win9x security.

Address space != protection boundaries. These are nearly orthogonal concerns. Where single address spaces might become less useful today is in dealing with Spectre vulnerabilities, though formalizing more explicit requirements about information domains (as in multilevel security, which is a well-established field of OS research) might help address those.

The 1108 wasn't really special purpose Lisp hardware. One could run other operating systems on it. What made it special purpose was the loaded microcode for the CPU.

> Life is really good on modern hardware.

Agreed: On modern CPUs.

More support for the additional hardware features like GPUs, media processing engines and the neural network engines (see the M1 Pro/Max/Ultra) would be welcome.

I feel like a lot of posts like this are pining for the complete lisp machine -user environment- and overestimating how necessary/important the hardware architecture would be to getting back to that today.

I can manage to context switch between different lisps fine but I do sometimes wonder in e.g. a slime+SBCL setup how much that context switching is costing me.

A number of Lisp fans seems to care. That’s why we regularly see articles on HN trying to convince other developers to use Lisp. It has been going on for years. Article after article written by frustrated Lisp fans, not understanding why the language they love is not mainstream. Often making bizarre claims about non-Lisp developers not being smart enough to “get” Lisp or whatever. Not having a clue that most developers care about a lot more than just the programming language. The best way to show the “power” of Lisp is to develop commercially successful software using it. That will do way more to convince smart developers to try out Lisp than writing yet another Lisp article trying to “sell” Lisp.