Literate: A Modern Literate Programming System
literate.zbyedidia.webfactional.comIt looks like this software breaks compiler error messages.
One of the great features of Knuth and Levy's CWEB is that errors are reported in terms of locations in the literate source. This feature, which works by way of line pragmas, is essential for proper debugger support and, in my opinion, sanity while coding larger projects.
If it is missing, it is often better to skip the "tangle" step entirely and have the literate source map directly to the program source line for line. Trying to "pattern-match" the compiler errors against tags in the source file is a painful waste of time. The 1-1 mapping is quite practical for more functional languages if one writes code in a style more like SICP than the TeX sources.
Thanks for the comment. Because this "Literate" software outputs nicely formatted code (including automatically included comments identifying code blocks) in the target language, it's not too big a deal, you can just see what line the compiler errors are referring to in the target source code. CWEB's approach of treating the tangled code as only intended for machine reading has the major drawback that you can't share the target source code with others and expect them to be able to read it; they need to buy into CWEB too. So there's a trade-off here.
> CWEB's approach of treating the tangled code as only intended for machine reading has the major drawback that you can't share the target source code with others and expect them to be able to read it; they need to buy into CWEB too.
There's not much of a trade off there. Either you use the tools that generated the code and edit their inputs, or the first time you update the code and rebuild your changes get clobbered.
And if people really find that literate programming makes the code that much harder to follow that they'd rather not use it, maybe it's not a great idea to use it.
It seems like this could be solved by making it an option? During development, generate line number pragmas to improve the error messages. Then when you have a working program, format the code nicely.
That's an excellent idea. I don't think it would work for all languages, but it's still a good solution, and will be added to the TODO list.
Interesting that you tried to imitate CWEB to some extent. I wrote my own tool (literate-programming on npm) ignoring very much the syntax of these other tools. My thought is that one reason Knuth's literate-programming did not catch on is that the syntax did not look very nice.
Looking at your examples, it doesn't strike me as very markdown-ish. There are @ symbols, and dashes to delineate code blocks. For me, markdown uses # Headings and uses tabs or ``` for code fences. I prefer the tab approach, but fences allow for language syntax highlighting for those who like that kind of thing.
Looking at the HTML output, I'm not sure this consitutes proper literal programming due to the amounts of 'contextual noise' that the processor generates. I think the goal is to have the program read more or less like a book, and this...
{Print "hello world" 2} ≡
printf("Hello world"); This code is used in section 1.
...is not exactly it. It feels more like program-in-program Matrixy thing than a document to be read.
The output is basically designed to look like the output of Knuth and Levy's CWEB tool. Maybe a better example to look at on the "try it" page is the wc.lit example, which makes a word count program. It's based on Knuth and Levy's wc.w tutorial example, and gives a similar output.
Of course, there are advantages of Literate's output compared with CWEB's; it's more flexible in that you can embed arbitrary HTML in the output, so you can easily put images into your explanations, and you can put as many code blocks and text blocks as you like in each paragraph.
I think a problem with literate programming is that programs are never written from top to bottom, at least not in conventional programming languages. The same holds for books, of course, but not in such a dramatic way. Therefore, I think the best way to read a program is not from top to bottom, but rather by using some kind of search tool.
By the way, does anybody know of an automated way to "grade" the quality of code documentation? I think this is what we need on repositories such as github, as it could stimulate the production of documentation of better quality (in most cases the documentation of OSS projects is rather poor).
Yeah, it just seems to me that wonderfully written literate documentation would be poorly organized software, and wonderfully organized software would be poorly organized documentation.
I guess what I mean by that is that as software scales out, your code will be more along the lines of "do this logic whenever this function/method happens to be invoked". So it would be great for documenting the workings of a function/method, but it would be poor for actually documenting the lifecycle of a logic path, like the lifecycle of a request. I certainly wouldn't have one file where I handle a request, do some business logic, and interact with a database, even though that is how you might document a lifecycle the request.
I fully agree with this and the grandparent. I've written a handful of literate programs, and I found that it worked nicely for a parser, which read from top to bottom in a single file, and it worked quite well for some unit tests, where the extra documentation and the tests themselves - which are quite self-documenting anyway - sat together rather nicely (though it was less well-received by my colleagues, who decried it as neurotic and pointless). My other attempts - particularly a small web app with a bunch of GUI components - were much less successful.
Did you write it as a single file that breaks up into multiple files or was each separate file corresponding to a separate literate document?
My tool takes the approach of more of a literate-project approach where one file may generate multiple files (and multiple files could generate one file). The idea is that the organization is completely up to the author with no boundaries between how they want to organize it.
And referring to earlier comment, I would write the same flow of logic for a single thought in the same place. There is the flow of a single process and there are commonalities across processes. I think a good literate programming allows the uniqueness of a single flow to be put into one place and the commonalities to be extracted and applies to such a flow.
I believe Knuth published the entire source code for TeX as a book consisting of the output from his CWEB tool. I would love to browse through it...
I mean, regardless of whether literate programming is viable for "enterprise software" or whatever, that's a wonderfully interesting cultural artifact... Inspiring!
Some implementations of Unix were also published as books, as well as the original source code for PGP (as a free speech trump card to defeat cryptography export restrictions).
These books, I presume, are archived by the Library of Congress, and will be around for a long time. Maybe one day we'll need them. Like if GitHub suddenly explodes and all the world's software is lost!
How to read and write programs is still something of an open question, as far as I'm concerned. The effort involved in caring for a "literate" code base, and indeed the literacy required, is probably too much for most projects. I don't think that makes it a "bad" model. It seems good to have tools available, and for people to try it out.
The Haskell world does a bit of literate programming, and tools like lhs2TeX are quite nice for preparing papers and such. There's something very beautiful about carefully written code that makes enough sense to be published as a "literate" paper.
I wonder, might a better metaphor be that of a journal, instead of a book?
A program has history, which, by the way, is stored in VCS, planning tool and communication, not the source code it self,
So how about a literate VCS?
Maybe you start with a tree structure to outline release planning and done branches, and break it down to individual updates against the current state of the source.
Might even keep contextual history like annotated repl invocations and throw away test code around.
I wonder how much is lost or gained by writing a different document (.lit) than one reads (.html). E.g., how much would a LyX-style (http://lyx.org) editor contribute? I'm sure it's been discussed before, but where?
Quick editing fix, under the section "features of this tool". I think this; "The code that is generated is indented property" should read as 'indented properly'.
Thanks, it's fixed now.
How is it different from org/babel?
One difference is that Literate isn't tied to a particular text editor like org/babel is tied to Emacs. You can edit .lit files in any text editor. That said, while there already exists a Vim plugin to handle .lit files, there isn't yet any Emacs mode, or plugins for other text editors. Such contributions would be welcomed.