λ-2D: An exploration of drawing as programming language
media.mit.edu> To be able to draw programs that look visually appealing themselves, such that someone would want to put a frame around them and hang them on the wall.
I've done that with the predecessor function in lambda diagram form:
┬──────────
┼─────┬────
┼─────┼─┬──
│ ──┬─┼ ┼ ┬
│ ┬─┼─┼ │ │
│ │ ├─┘ │ │
│ ├─┘ │ │
└─┤ │ │
└─────┤ │
└─┘You could easily repurpose that code for drawing Quantum Circuits and thus representing quantum computations. I'm tempted to make a pull request this instant and do it myself.
That would be very nice.
This reminds me of Frege's Begriffschrifft
https://commons.wikimedia.org/wiki/File:One_page_from_Frege%...
This is very cool. I've always been very disappointed that programming is so utterly dominated by the linear text format. It feels like so much potential for expression and feedback is getting left on the table, making use of our capacity to understand and work with spatialisation, patterns, sounds, colours, dynamics, etc would make a lot of sense.
Bret Victor's The Future of Programming and other talks come to mind - https://youtu.be/8pTEmbeENF4
As someone who has done thousands of hours of programming in a visual language (IAR Visual State), I hate it. The promise of an easy visual overview quickly drowns in a sea of edge cases.
A picture says a thousand words, sure, but precicely what those words are turns out to be important. We use text for programming because it allows us to express ourselves unambigously.
I sure would like to see more visual representations of text programs though, this seems like the best of both worlds to me.
I've also had clients that used Simulink/StateFlow modeling for embedded state machines and to me it works up to a certain degree of complexity.
It works really well for a quick prototype, but for anything more envolved I find the cognitive fatigue of a complex visual model is much greater than keeping a mental model and looking at a narrow scope piece of text.
I wish we had a smarter feedback loop between trying an idea (aka typing stuff) and reacting to results. Right now tdd or type inference doesn't really produce backward infered hints. Types do help exhaustive matching but you may be able to reverse trials output back to function types and conditionals.
Your editor would become a convergence game.
A REPL?
Like SmallTalk?
Smalltalk is live object but, afaik[0], the base interaction is still work for edit, work for test, work for resolution. I thought about making them one concept.. something like gradually pruning input spaces. test and code being somehow isomorphic (each failing test is implicitely mirrored in the working code).
[0] I only did a pharo MOOC and read some bits about smalltalk but considering the crowd, it's possible someone has an image with similar if not better ideas.
As a thought experiment, the analogous argument could be made about the linearity of the HN comment threads (despite their tree structure, similar to programming languages). However somehow I doubt that adding spatialization, patterns, sounds, colors, dynamics would be turn out to be an overall improvement.
This guy is such a genius. I'm continuously blown away by his stuff. His work just keeps getting better and better. It's mind-boggling. He's also the author of all this stuff on GitHub which is incredible:
His work is timeless. to me it has this sort of timeless quality to it: people will look at it in like a hundred years or 400 years and be like yeah this person was a genius, they were doing like 2022-Renaissance level stuff at this time. That's what I think... And obviously there's incredible groups like what openai just did with the drawing technology. But as an individual contributor this person's incredible.
Also I love the relentless exploration of code as an artistic medium.
I saw this post come through on Reddit yesterday and spent some time going through his projects. I have the same response as you. His stuff is very inspiring and wants me to lean more heavily into my whims and ideas. What I really, really enjoy about his projects is the culture embodied in them.
> Coding in λ-2D is somewhat like playing Minecraft or Factorio, but it's even better because I can call it research.
I love the honesty in this sentence so much.
I've messed around with this same graphical language, though only on paper. As I understand it, the lambda/apply nodes are the two morphisms for reflexive objects[1], and these are something like string diagrams for them -- though I haven't fully understood how to interpret oriented wires from this point of view...
There a paper referenced by [2] (a project shared on HN in the last couple months) that goes into a graphical language for lambda calculus with the same apply/lambda nodes, but with explicit copying and dropping. The paper is Lafont, "Interaction Combinators" from 1997. (You have to squint to see why it's got anything to do with the lambda calculus.) A similar graphical language is Chemlambda by Buliga, which uses graph rewrite rules with very similar node types.
Programming by drawing things in 2D on a fixed grid is an interesting idea I've wanted to experiment with. (I'd imagined a more Factorio-like Puredata or Max/MSP. It's something that came to mind from having read about [3].)
[1] https://ncatlab.org/nlab/show/reflexive+object [2] https://github.com/Kindelia/HVM/blob/master/HOW.md [3] https://100r.co/site/orca.html
> I've messed around with this same graphical language, though only on paper. As I understand it, the lambda/apply nodes are the two morphisms for reflexive objects[ https://ncatlab.org/nlab/show/reflexive+object ]
That's a useful pointer. It's interesting that the nLab folks seem to specifically point out higher-order abstract syntax as the underlying intuition behind these 'lam' and 'app' morphisms. So if actual string diagrams exist for these, the same sort of graphical representation may well be applicable beyond the untyped lambda calculus itself, to any syntax that has some equivalent to "variables" that can be used for substitution as in beta-reduction. (Note that the graphical representation of a typed lambda calculus is fairly obvious and intuitive, but only because functions then get different types based on their arity, etc. and this is directly reflected in the visual syntax. Once you go untyped, it doesn't seem that this can be true.)
The untyped lambda calculus seems pretty uninteresting compared to 'typed' counterparts that tend to come with more interesting graphical visualizations out of the box, viz. string diagrams (where the topological features of the diagram have their counterpart in useful program transformations).
Also, I thought it would be fairly well established by now that the De Bruijn notation for the untyped lambda calculus, with prefix function abstraction but postfix application, better reflects its underlying properties since it allows for an intuitive visual understanding of reduction, where applications can be directly matched up to abstractions.
This is really cool!
I worked on this very same idea (a programming language without text) during my grad school years, and it's online here, if anyone is curious:
Here's an example of one creator's projects (definitely not me!):
Really cool project! It reminds me of a program I used to make little games called Corel Click 'n Create.
Very nice!
However written languages convey not just what a program does, but also, through the names of things, what it means. Unfortunately there is no validation of (or other computation with) the meaning - so what it seems to mean may or may not have any bearing on what it actually means, either now or previously.
Thus a program is not just how humans talk to machines, but also (one of many ways) humans talk to humans.
They do show some diagrams with comments.
It's true that there's no validation of comments. But there's also no validation that the name of a variable is what that variable actually means, or that the name of a function is what that function actually does.
Indeed - but with text programs you are forced to provide a name (hopefully a meaningful one!)
Honestly dithering over what to name something is the biggest timesuck, I wish I was unprincipled enough to just name them a,b,c... maybe use a drawing of a flow chart as the readme.md xD
There are 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors.
I haven't understood the details of this language yet, but on first look isn't it a... circuit board designer?
There are already games that allows you to place electric components, each as a pixel on a screen, where these pixels are powerful enough to allow you to code arbitrary programs. I particularly like The Powder Toy, with enough components to make a fast calculator: https://powdertoy.co.uk/Browse/View.html?ID=1214884
The author arK created some of the most powerful electronic-based saves in The Powder Toy. He also made a SNAKE game which you should definitely check out. They are impressive.
Redstone in Minecraft should also fit into this category, but the world is too big, components are less powerful and game tick is slow, so writing programs in it is too clumsy.
The author [1] was somewhat recently featured on Hacker News [2] for their Fishdraw [3] project, if you're wondering why their name might look familiar. Always a treat to see what they come up with next!
This reminds me of nothing so much as the computers people implement in The Powder Toy [0] - beautiful, impressive, inscrutable.
Those animated runs look like they would be super useful for debugging. Offloads a ton of working memory that comes with tracking stack traces in your head when you can just see a visual representation of the stack and watch data flow in real time.
There is a lot of interesting work related to computation using shapes in the Shape Grammar[0] field. With shape grammars, rather than using computer vision to parse the pixels, computation can be done using the intrinsic properties of shapes. There is some great work being done at Georgia Tech in Dr. Economou's Shape Computation Lab[1]. For beginners, they have created an unsigned binary adder using purely shape representations as numbers[2].
[0] https://en.wikipedia.org/wiki/Shape_grammar
[1] https://shape.design.gatech.edu/index.html
[2] https://shape.design.gatech.edu/Machine/Projects/12_Circuits...
I love this. It'd be great to see further exploration of this theme. It could impact not just programming, but our ability to create and share ideas.
Obviously text is the most powerful and intuitive means of programming, but I wonder if it's the best? Perhaps more generally, is language as we know it really the best way to convey meaning?
I've no idea on such topics. My curiosity has just been roused by this ambitious project.
There's a bug where if you have an animated run, but switch programs, the first animated run will keep executing over phantom pathways. You can start another animated run at the same time. The resulting visuals and audio are really interesting.
I also love the option to obfuscate the code in two separate ways. This project rules.
This is pretty cool, the programs end up looking pretty pleasing to eye. Looks like schematics.
Absolutely fascinating. I actually put a lot of time a while back into looking for a way to "draw" either a psudeo-code specification language, or just Python, on an iPad for "programming doodling" that could be scanned and parsed for later execution (spoiler: handwriting code outside of a drawing program is terrible). I love the idea of being able to bring computation closer to a hand-to-paper medium. Computer vision processing of code -> computation in an editor would be a dream. Please keep at this.
Cool, but in practice, for software development you need version control. Has anyone ever seen a workable solution for diffing -- let alone merging -- _any_ graphical language?
No, but that is probably because such a thing hasn't ever really been needed. To be a reasonably good product, it would probably need to parse the language to show both an actual pixel-for-pixel difference and a what-is-practically-different diff. Merging seems next to impossible with more complex programs, unless there could be an art->program->art program. The problem with that is that it would (probably) destroy any comment art or organization that was in the program before. The more I think about it, the more I think that a merge for this sort of file would be a really cool semester project for a comp sci class.
> because such a thing hasn't ever really been needed.
I think this is backwards. I think the lack of collaboration is exactly why graphical programming can't be used by most. It's so badly needed, that it's not even usable without it, for the majority of people.
I think it would be extremely cool to be able to transpile other languages into this language's diagrams and then watch the execution.
Very interesting! Reminds me a bit of ladder logic, a 2D visual programming paradigm based on wiring together relays - but at a higher level of abstraction, since ladder logic is mostly based on simple boolean operations and doesn't really have any notion of defining functions.
The fractal tree and UTM examples are absolutely beautiful and mind-bogglingly simple.
Who immediately thought: Factorio!
So cool. Does anyone build serious projects in max MSP? There is something satisfying about building visually that is repeated here. Those patches quickly get unwieldy though.
Thisreminded me of another programming thing based on lambda calculus: https://justine.lol/lambda/
Looks like LabVIEW https://en.wikipedia.org/wiki/LabVIEW
It looks very cool, like circuits. I bet these kinds of languages will obsolete text based languages in the future for being more powerful and easier to use.
Reminds me of SuperHack - https://www.hacker.org/sh/
Gr8 piece. It would be awesome to see that in 3D (VR) with autogenerated layouts. (i.e. force directed graphs).
This made me wonder, of course if electrical circuits can be expressed as pretty much 1:1 WYSIWYG diagrams, why not computer programs?
Crazy. Alien tech.
What was that language or system where it would read what you wrote down on a piece of paper and then execute the program ?
Perhaps you're thinking of Dynamic Land:
I made a proof of concept language once of something along that idea.
Since the purpose of all programming languages is to be readily readable by biological OCR systems, I would think all. ;)
perhaps Dynamicland?
How would version control be done?
The animated run would probably be very interesting with a program emulating the Enigma machine.
It's very cool and the presentation is clear. The animation of the parsing is fun as well =)
quite a few plc's (programmable logic controllers) are programmed graphically as well.