Brian Lovin
/
Hacker News

Show HN: Common Lisp running natively over WebAssembly for the first time

soi-disant.srht.site

A month or so ago, I ported a Common Lisp implementation (npt) to WebAssembly to make a silly blog post, because I was bored and have a lot of time on my hands to waste with things like this (I don't have a job, and because I have next to no experience, these meaningless, silly projects tend to fill what time I do have).

This is significant as it's the first time Common Lisp in particular has ever been hosted on it; wasm has a few poor decisions in its design that make it less-than-conducive to being a target for Common Lisp, and a lot of the more interesting implementations require an implementation to already be on the platform for bootstrapping purposes.

My previous attempts using other implementations haven't gone so well, despite throwing a lot of time at it (as an example, I have a fork of Eclipse Common Lisp, a defunct implementation from the 1990s, sitting on my disk with a few hundred lines of changes that I finally got to successfully compile and run a handful of very basic expressions, but it blows up when you try and define anything). In comparison, I was pleasantly surprised by how little I had to do, even though I did end up scrapping loads of lines of my own changes to npt in the process as I got a handle on how to make it work acceptably.

The Emscripten toolchain and I don't get along, partially because I don't like inlining ECMAScript into my C and vice-versa, so it's little more than a neat little demo right now.

You can load slightly more complex programs into it by hijacking the "imp" ECMAScript function every few hundred milliseconds with strings containing complete forms (this is essentially a batch processor, so there's no interactivity that allows it to wait while you decide what the rest of a form should be). Only one at a time, though. It's not that fancy.

If you mess up at all, even just a little error, it will crash. This is by design; I disabled the debugger. It's a giant hack, and the hack I eventually decided on left it impossible to have a debugging experience, with the benefit of getting to use a closer-to-unmodified npt.

This could be more useful, if I spent more time on it, but it's more fun if it's just a demo. I hope you enjoy the toy I made for you.

https://en.wikipedia.org/wiki/Batch_processing

If you don't know what forms are in the context of Common Lisp:

http://www.lispworks.com/documentation/HyperSpec/Body/03_aba...

http://www.lispworks.com/documentation/HyperSpec/Body/26_glo...

Daily Digest email

Get the top HN stories in your inbox every day.

aidenn0

This probably doesn't count as "natively" but I've run ABCL[1] under Doppio[2]. Startup times are under a minute in Chromium based browsers and under an hour in Firefox. I've run into zero stability issues, but its no speed demon.

[edit]

Just tried again today and Firefox gets to a REPL in about 3.5 minutes, while chromium is still right about 1 minute.

1: https://abcl.org/

2: https://plasma-umass.org/doppio-demo/

metagame

Thank you for making it!

Unfortunately, a JVM on wasm would be quite difficult for the same reason that Lisp over wasm is quite difficult (I had actually looked for wasm JVMs before trying anything). I had no idea there was a JS JVM implementation! That's very cool.

kaba0

There is also TeaVM which is very fast and has both a js and a wasm target.

aidenn0

TeaVM lacks reflection and a class loader, both of which are necessary for ABCL.

gumby

Great work! You write,

> …wasm has a few poor decisions in its design that make it less-than-conducive to being a target for Common Lisp…

Could you say a bit more about those design decisions?

miloignis

Not OP, but as a fellow Lisp-on-wasm developer one problem is that the current release doesn't support tail call elimination. There's a proposal for it (a tail_call instruction), and Chrome has implemented it, but the Firefox/Spidermonkey team hasn't prioritized it, so it's sat for a couple years. At least two implementations (maybe two browser implementations? I don't recall) are needed for standardization, so things remain.

You know, I wonder how seriously I could be taken if I duck-taped a wasm-capable browser together out of Servo and Wasmtime to make a second implementation and push it forwards...

gumby

There is no branch instruction at all?

miloignis

There is, but all control flow is structured and the argument/return address stack is out of your control. You have blocks, loops, and if/else statements. The behavior of a branch instruction differs based on context - in a block, it jumps out of the block, in a loop it jumps to the beginning of the loop.

If you really wanted full tail-call behavior, you would either have to compile every function into the same mega-function in a loop and have some sort of if/else tree, or use trampolines (which would additionally require either storing parameters in memory somewhere or using the same type signature for all functions, since function calls are typed).

Overall, it's not a super great situation for true tail-call elimination. For now, I've implemented limited tail-call elimination for single-function recursive calls (transforming them into an in-function loop), and that's patched things up enough for me to continue working for now until I either need to come up with an optimized trampoline or the tail_call instruction finally gets standardized.

moonchild

I believe there are challenges related to nonlocal transfer of control, as well as multiple return values. More damningly, if I recall correctly, lisp implementors were consulted and their feedback ignored (just like apl implementors with .net, back in the latter's infancy).

mbrock

I think a major limitation is that a WebAssembly module can’t run dynamically generated code, which is a huge part of typical Common Lisp implementations.

jhgb

I was thinking about this, and one solution that came to my mind was serializing the heap into a temporary image and restarting the environment from an updated Wasm module with new code. This would also collect all garbage at the point of transitioning to the new code. You might need to be very careful to do this reasonably quickly, but in principle I don't see why this wouldn't work. The idea came to me when I was thinking about how an interactive-but-native-speed Oberon environment could be implemented in Wasm.

nickmain

You could certainly generate a new Wasm module on the fly and then execute it from Javascript. Linking and sharing of memory should be possible.

Pyodide can dynamically load libraries that are separate Wasm modules so it is worth checking out how that works.

aidenn0

IIRC inter-module calls are more expensive than intra-module calls though.

WASM also doesn't allow for functions with variadic return counts.

pmoriarty

"a WebAssembly module can’t run dynamically generated code"

Why does it have that limitation, and is there any hope that it will someday be overcome?

aidenn0

On your computer, heap, stack, code, and global data all share the same address space. On the WASM virtual machine, only heap and global data share the same address space. Code and stack are abstracted away by the machine.

It was designed to support C-like languages, so you can do the equivalent of loading a DLL (or dynamic shared object). However in Lisp it is not unusual to compile just one function at a time. It's definitely possible to create a DLL for each function and load each DLL, but it requires a completely different architecture than most Lisp implementations use (in which, when you compile a function, it just writes the machine code to the heap).

There are similar impedance mismatches with so-called W^X systems (where you are disallowed from having a page be both (W)ritable and e(X)ecutable.

metagame

Thanks for the only compliment in the thread! I appreciate it, especially given I've been such a fan of your writing and life for such a long time.

I think that the people who responded to you covered much of it, but you can find more by doing a web search for it. I'd find you links myself, but it's early in the morning and I'm a little tired.

catchclose8919

...god, my eyes ...why is everything CL plagued by such horrible design choices (hyperspec, Lisp-IDEs... all!) - why such ugly colors, ugly typography, bad contrasts, ugly logos, ugly diagrams, ugly supporting graphics?!

I know that even the language itself is kind of the opposite of "beautiful", but the way all docs, blogs, websites etc. look ...seriously, is this intended to scare away any aesthetically sensitive people? Programming languages are about aesthetics too, and Lisp at its core (not CL ofc) is absolutely beautiful!

dang

"Please don't complain about tangential annoyances—things like article or website formats, name collisions, or back-button breakage. They're too common to be interesting."

https://news.ycombinator.com/newsguidelines.html

Your comment particularly broke the Show HN guidelines:

https://news.ycombinator.com/showhn.html

christophilus

Oh, wow. I thought you were just another HN hater, but nope. I think my eyes are bleeding.

metagame

I took a picture of a box of pens, snatched the hex values from it, and then threw them directly onto the page. I'm glad you like it!

User23

I like the cut of your jib sir.

emacsperson

Its actually quite readable in eww, the emacs web browser :^)

coryrc

The hyperspec is copyrighted and licensed in a way where we can't change it at all. It's like the primary source to learn so impossible for anyone knowledgeable to clean-room reverse engineer a new one.

jhgb

At least in my country you'd be perfectly within your rights to distribute a program that would modify it automatically for every reader who'd want to read a modified version, since as per our copyright act, you're entitled to do pretty much any modifications to the copyrighted works that you possess, as long as you don't redistribute them.

aidenn0

The final draft of the ANSI specification is freely available to use. There's at least one transcription of it available in the public domain[1]

1: https://github.com/phoe/clus-data/tree/master/live/cl

catchclose8919

...you're not joking, right? jeeezus

fami-com

http://clhs.lisp.se/Front/Help.htm#Legal >Permission to copy, distribute, display, and transmit the Common Lisp HyperSpec is granted provided that copies are not made or distributed or displayed or transmitted for direct commercial advantage, that notice is given that copying, distribution, display, and/or transmission is by permission of LispWorks Ltd., and that any copy made is COMPLETE and UNMODIFIED. IN PARTICULAR, the material that MUST appear in the copy includes:

>...

>Permissions related to performance and to creation of derivative works are expressly NOT granted.

>Permission to make modified copies is expressly NOT granted.

>Permission to add or replace any links or any graphical images to any of these pages is expressly NOT granted.

It's one of the most restrictive licences.

jhgb

> why such ugly colors, ugly typography, bad contrasts, ugly logos, ugly diagrams, ugly supporting graphics

What? It looks perfectly fine on my monochromatic Genera monitor!

Jokes aside, Firefox's reader view makes it only marginally more readable. I wonder if it took the author extra effort to make it look like that, and whether there's a better page sanifier than the reader view.

metagame

All of the CSS is handwritten for exactly the intended effect that it gave.

sj4nz

PureScript always sets the bar high for this in my mind, here's an example documentation page: https://pursuit.purescript.org/packages/purescript-effect/4....

dtagames

So true! Also, there are no decent UI frameworks for Lisp, so it's impossible to build a full stack app that looks good in a modern browser without adding a TS or JS web component layer. And... There's no modern IDE for Lisp.

I think this aspect of ignoring UIs and aesthetics has seriously held back CL.

metagame

That's not true at all, and I'm tempted to write something to prove you wrong.

My aesthetic choices were intentional, and almost nothing in your comment is right.

smcn

I'd love read that rebuttal. And I'd ensure that I read it in its gloriously yellow form.

lobstrosity420

> Also, there are no decent UI frameworks for Lisp

How about CLOG and cl-cffi-gtk?

dtagames

Ok, thank you for those references I had not seen. CLOG looks like it creates the UI separately and essentially "streams" it to you over a websocket. While novel, that introduces it own problems. Perhaps this why there aren't any working examples that I could find?

The CL library you linked is for connection to GTK, a desktop UI scheme. So, while interesting, that doesn't really help someone who is trying to develop a browser-native app, which is where the world has gone at the moment.

I think Lisp got left behind on the journey, and I think this UI problem is one of the top reasons, the other one being terrible-non-IDE-like-substances.

undefined

[deleted]

brokenkebab2

Talking about the page behind the link - it's not like Sistine chapel, of course, but it's completely ok. In digital realm your perception is very strongly affected by medium/device/environment. Maybe that's the case. Not commenting about hyperspec though

jedimastert

As someone that has (admittedly minor) struggles with reading and visual processing, I would call the accessibility at least absolutely atrocious.

I don't know if I feel more or less betrayed by the fact that the HTML seems to be more or less bespoke and a screen reader would have no problem with the page, but all of these choices feel all the more intentional.

metagame

The page was written to render well if you turn reader view on, and the color schemes are all in one line and easy enough to change. The only thing missing from the reader view is the aside, which is silly and in proper contrast on the original page.

However, the contrast is WCAG AA-conformant (except for the links that aren't in a black box, which aren't important links, as I went out of my way to confirm as I wrote the post). The page is actually pretty accessible.

Accessibility is important to me, as many people I've known in my life have been disabled, but so are silly aesthetic choices.

It was intentional, but it also was well-intentioned. That's why it's readable in more or less every web browser, regardless of whether it supports the one line of CSS I'm using, and accessible to screen readers (although I didn't throw in any elements specifically for them, the page is written simply enough that it should work, intentionally).

alpaca128

The contrast between light blue for links and vibrant yellow for the background is absolutely not okay, I find it almost impossible to read.

Yellow as background can work well, but this is not the way.

kotborealis

Great work! A small nitpick — the page is a bit hard to read, links have low contrast, you can check it using accessibility tools.

rsstack

I initially rolled my eyes that it's yet another pointless text contrast complaint - and then I opened the site. My eye sight is great and that contrast (and border) is not great :)

undefined

[deleted]

undefined

[deleted]

FraaJad

So much complaint about color schemes. Hitting the Reader View formatted the page beautifully.

I'm thankful it is a simple HTML page that could be easily formatted using browser-built-in tools.

ngcc_hk

Is there something like a box that you can key in something and it works. Reader view give you text. Also that box?

I lost … perhaps I expect a REPL or … anyway not good to sell CL-webassembly I think.

metagame

As it says in the article, stdout is wired to the browser console. So you open your browser console to see it.

It's not meant to be sold, it's just a little toy.

undefined

[deleted]

ngcc_hk

Like reading Rfc in 1990s … a bit odd choice to use this format and font to sell anything these days.

dang

"Please don't complain about tangential annoyances—things like article or website formats, name collisions, or back-button breakage. They're too common to be interesting."

https://news.ycombinator.com/newsguidelines.html

metagame

Nothing's being sold; it's just a little toy.

undefined

[deleted]

Existenceblinks

I hope someone would create a tutorial which is using a toy programming language to compile to webassembly from scratch. Using existing language is too opaque to understand anything.

metagame

That sounds like a good idea. Have you considered writing it?

Existenceblinks

I was trying to compile my internal data and encounter "what's the right way to represent string in Webassembly" so that's why I came up with the demand for tutorial from a toy language because it has less features to worry about.

I don't understand why I got downvoted, it seems HN users with power to downvote just see everything is a nail they wanted to exercise their power.

Anyway, I'd probably write one when I actually implement it. I know linear memory is available but not sure whether or not having GC or mechanism that make memory safe.

undefined

[deleted]
Daily Digest email

Get the top HN stories in your inbox every day.