The new Jupyter Book
blog.jupyter.orgThis is a really good move. A lot of people have been offering Jupyter Notebooks along with their Books. This should bring down the effort in getting the book out. An important note though, R world has for something called 'bookdown' for quite sometime and many R developers have used it to write books. https://bookdown.org/
The only worry is that now a lot of data science enthusiasts are going to try to publish their Jupyter Notebooks collection as Kindle books!
I'm personally a huge fan of `mdbook`, which seems to be the Rust equivalent of R's `bookdown`. I've switched out internal documentation over to it from `gitbook`.
+1 for mdBook, easy to install (as compared to say honkit [0], a gitbook fork, that requires npm). I've recently started using it to generate web version of my ebooks.
mdBook is nice indeed but it still lacks a proper way to create PDFs. `bookdown` has a very smooth way to do that. The only thing I don't like is that is very R centric in terms of syntax for adding tables, graphs and whatnot. It's not pure markdown.
For things that will be online only mdBook is great.
I'm currently testing honkit which is a gitbook-cli fork and it seems OK.
> It's not pure markdown.
Be fair, it's not trying to be and not claiming to either.
That's nice!
For first time i've seen huge resource about R.
I love the Jupyter ecosystem but it's frustratingly messy sometimes. There's already "nbconvert" built in to Jupyter (lab), built on Pandoc. It's extremely under-documented but it's possible to extend it with templates to generate arbitrary outputs. I've done this to directly generate branded PDF reports via latex. Jupyter Book has a lot of overlap with this feature, but rather than building on nbconvert/pandoc, it starts from scratch with Sphinx.
Oh well! Keeps things interesting, I suppose!
The JupyterLab extension system is the biggest offender imo.
"Oh, I need to install the python package and the jupyter extension separately? And there are no breadcrumbs if I install one but not the other? And I have to figure out which versions are mutually compatible with each other and jupyterlab because 'everything latest' sure isn't? And I have to install extensions to get features that used to be built-in? And then I have to explain the install procedure to the person I'm trying to share with, twice, because he didn't believe me the first time? UGGGGGGGHHHHHHHH"
Imagine you manage all this but they change something fundamental and you have to (get everyone else to) reinstall all of it.
We maintain a custom internal Docker image that everyone uses for their JupyterLab needs to make this manageable; that way, we only have to get everything to work once, and everyone else can just docker pull :latest and be good. It's become a rather large image, since it needs to have everything anyone uses, but for a small-ish team, this has worked well for about two years or so now.
Another upside: There is a GCP gcloud one-liner to start a VM for a given Docker image, and the image is designed to work both locally and in a GCP VM, with notebooks in a git repository that gets checked out automatically on container creation, so switching from laptop to 16-core VM to churn through a large dataset from a bucket is pretty seamless.
nbconvert/pandoc is very limited in how it can be extended because it only relies on jinja templating rather than a proper AST. You couldn't achieve a lot of the things that sphinx/jupyter book can now do with that. Trust me I know because, before starting work on https://github.com/executablebooks/MyST-Parser I used the nbconvert/pandoc approach to create https://github.com/chrisjsewell/ipypublish ;-)
This is interesting, so do you think the JupyterBook solution could/should replace nbconvert?
I was always a bit frustrated by the way nbconvert pulled in pandoc as a dependency, but then didn't make use of any of its power (pandoc surely has a powerful AST). Instead they re-implemented filters and templates themselves. It struck me that it would have been much cleaner to write a proper 'ipynb > AST' Pandoc reader, and then expose pandoc AST filters and templates.
I was a bit suspicious of moving away from pandoc to sphinx, because sphinx seems markedly less powerful and general than pandoc. However, you seem to think it's the opposite! Does Sphinx have an AST? I can't see any reference?
pandocs AST is actually a little limited, in terms of extensability, because it has to resove to JSON. The scorreia.com/software/panflute/code.html is a good python API for it.
By contrast sphinx uses docutils AST: https://github.com/live-clones/docutils/blob/master/docutils..., the documentation is not great (working on that) but it is pretty powerful.
In fact, pandoc now does have an ipynb reader and writer. You can convert between ipynb and any of the formats pandoc supports, and you can manipulate the AST using filters.
Now use that to replicate the example pages here: https://myst-nb.readthedocs.io/en/latest/examples/basic.html... and here https://jupyterbook.org/interactive/launchbuttons.html, and let me know how you get on ;-)
Oh, this is huge for me!
My main concern with jupyter is that I prefer orgmode (and by a lot), but I know that it's niche and most people don't even heard about it ipnbs are preferred form of notebook exchange.
I will check how good translation to/from orgmode is then.
Oh! Thanks! I'll definitely take a look at that!
This looks great, as extended commonmark support knocks down the last major pain point to switching to Sphinx for docs in the projects I've worked on.
The only remaining tooling gap is a markdown linter implemented in python.
Why a python linter? Text editors are usually language agnostic and separate from other tooling.
If the rest of the documentation workflow is in python, it would remove the need to install another entirely separate scripting language runtime to do those checks on every developer laptop, CI system, etc.
>I've done this to directly generate branded PDF reports via latex.
Can you elaborate on this a little? Do you mean via Jupyter notebooks?
Here's an example where nbconverts a notebook and a template into a formatted tex file, and then a journal-ready PDF:
https://codeocean.com/capsule/9155944/tree/v1
The nbconvert syntax is
and then `pdflatex` converts the tex file to a PDFjupyter nbconvert --exec --to latex --template ieee.tplx --output ../results/editorial.tex editorial.ipynbYes, via Jupyter notebooks -- using the same machinery that gives the 'File > Export Notebook As...' functionality.
My method is roughly this:
1. Create a Python package that subclasses nbconvert.exporters.LatexExporter -- in the setup.py you register the class to 'nbconvert.exporters' in the 'entry_points' dict.
2. You can do general filtering/compiling by registering 'Exporter.filters'. The compile process can be modified by implementing a custom 'from_notebook_node()'
3. You can include a folder containing jinja template(s) in the package to implement whatever styling you want
4. I have some standard metadata that I pull from the notebook/cell metadata (using the standard Jupyter/lab frontend metadata sidebar) to define things like document titles, authors, etc.
5. Do the conversion like: jupyter nbconvert --to=myfancyexportformat notebook.ipynb
Wow someone use the features I wrote years ago ! <3 You might want to submit a blog post to Jupyter.org to show that this is possible and make it more known.
That work you did those years ago made me a hero at work :) Thanks!
Never had the opportunity to really document it, so thinks for finding it; I would have hoped for a much wider variety of exporter to be published. I'm still happy about the "pip install package" --to=qualname. I thin you can hardly make it simpler.
It's honestly amazing how far this thing that started as ipython has become. From expanding into notebooks, to multiple languages, to huge collaborative web-hosted coding applications. I've been a huge fan of iPython from the very start and I'm always amazed by seeing this project grow.
In my job I often have to figure out how to solve a problem involving talking to an API or manipulating data in some way and then sharing that information with others. Jupyter notebooks make that so easy, it’s the first tool that I go to. Being able to mix documentation and code, as well as visualizing output, aligns perfectly with the way I think.
I’m not entirely sure I have a use for this book feature, at least not yet, but this is a cool addition.
I keep my own notes on how to use obscure API features that I've learned, in Jupyter.
A colleague of mine, when first introduced to Python and Jupyter, remarked that Jupyter could be a great platform for writing tests.
If you have your own notes feel free to suggest docs updates !
I wish that the notebooks supported Typescript or C# or something I'm more familiar with, out of the box.
For C# I have LinqPad, which I'm a huge fan of but never discovered something with similar functionality for typescript.
I've seen some open source kernels[0] but I'm not sure about the quality, did anyone try them? Deno core in Jupyter would have been amazing.
[0]: https://github.com/jupyter/jupyter/wiki/Jupyter-kernels
For JS and Typescript there’s Quokka, which I haven’t tried and doesn’t really seem to be in the style of a notebook, but might be helpful? https://quokkajs.com/
There is actually a JS adaptor/extension for Jupyter but I found it pretty clunky to use, and since I’m equally comfortable with Python as JS, I just stuck with Python.
For Typescript Stackblitz might be interesting, but it’s not like a notebook, and it’s not a run-locally tool.
observablehq is the equivalent of jupyter for nodejs. Before that we have runkit.
For C#, have you seen .NET Interactive[0]?
F# is supported so at least there is some .net support.
I don't think there's a better stack in all of scientific computing than Julia + Jupyter Books. Lovin' it.
Julia + Pluto will probably give it a run for the money.
They had a fun talk at JuliaCon 2020: https://www.youtube.com/watch?v=IAF8DjrQSSk :)
Super neat! It’s like a mash up of Excel and Jupyter, using something that looks similar to functional reactive programming
I'm considering choosing something for a writing project with some diagrams that I might like to be executable / interactive, so that brought me here. It might be nice if the Jupyter Book and the Executable Book Project web pages acknowledged a bit more that there are alternatives out there and discussed the differences. I'm particularly thinking of Observable -- it looks like Observable has a good article comparing Jupyter to Observable: https://observablehq.com/@observablehq/observable-for-jupyte...
I was just discussing with my friend (who's currently writing his master's thesis) about how frustrating integrating Python results with LaTeX papers can be. In my experience (who wrote his master's thesis last year), it was not necessarily the worst, but it required me to complete my methodology and presentation entirely before putting things in LaTeX, because otherwise I would have to spend ridiculous amounts of time updating values that impact multiple tables and graphs.
I hope this is it. LaTeX won't go away anytime soon, but being able to turn a notebook into a near-finished thesis/book (in TeX) would be a-mazing.
Call me old-fashioned, but I dislike Jupyter notebooks, or really anything that I can’t easily use with my editor of choice, diffing tools, etc.
Jupyter should just be an editor and execution engine that operates on plain text code, using specially formatted comments to delineate cells. If I want to edit it with vim and execute it using plain old Python I should be able to.
Well the nice thing is you can, Notebook is just a file format; but many tools allow you to select a bunch of text or use comments as delimiter and still execute using Jupyter. JupyterLab text editor does that by default. ANd I've seen vim plugins for that.
Notebook document are _meant_ to keep output as they are design for research and where the records of your results are important, but you also don't have to use ipynbformat with the notebook UI either.
small nitpick : yikes! Jupyter's blog is on Medium :(
Yeah, but unfortunately we haven't found anything that allow easy collaboration with folks who don't know how to use GitHub, and migrating away from medium is a paiiin. Though if you have a solution, I would be happy to push for it on the Jupyter side.
Unsure if you mean solution for migrating or solution for a good alternative platform, but Ghost has received a decent amount of praise. https://ghost.org/vs/medium/
I've seen ghost, the problem is actually migrating the content, and while there are services they are too expensives for an OSS project, but thanks.
Maybe this jupyter book can be used as a blog?
I would just teach people how to use github. lol.
That is my POV, but I'm not the only one deciding, and not all people contributing to the blog are coders, we had request for execs/manager for guest blog posts to review before publishing so a lot of poeple were against github.
curious: how about Wordpress ?
I’m out of the loop. What’s wrong with Medium?
From the top of my head:
- captures creativity into a centralized proprietary platform on which creators have no control
- modal dialogs nagging you to create an account when you just want to read an article
- uniform, soulless, dull visual design
- heavy pages loaded with JavaScript that don't add any value for the reader
Make all the articles by default behind paywall, and even as an admin on the blog.jupyter.org I can' deactivate that, I need to ask the author that submitted it.
When do I use Jupyter Notebook and when do I use Jupyter Book? Do I need both? Which is more LaTeX friendly?
As I understand it, Jupyter Book takes one or more Jupyter Notebooks and converts them into a static "book" output (looks like it primarily targets HTML output but with LaTex/PDF possible)
Yes that's about right :) The book can contain a mixture if Notebooks and normal markdown files, and also text representations of notebook files (see https://jupyterbook.org/content-types/myst-notebooks.html) The initial focus has been HTML, but now there is work ongoing to create a nice LaTeX/PDF "theme"
I mean even for a single notebook the display looks pretty nice. Might consider using for single notebook books just to avoid dealing with the hassle of themes and jupyter extensions.
Couldn't find the answer in the docs so I'll ask here; does this support all the languages that Jupyter does, or does it require kernel specific hooks to function? I'm specifically interested in producing some C# docs for for our internal application framework.
Is there a Jupyter alternative that's just for JavaScript? Like Observable but not Observable.
I built Starboard (https://starboard.gg), its interface is similar to Jupyter, but it runs entirely in the browser (which has pros and cons)
There is Storybook that might scratch some of the itch.
"MyST Markdown is a superset of Jupyter Markdown (AKA, CommonMark), "
Hey Jupyter guys, can you please support asciidoc instead of inventing another MD format.
Sorry only 24h per day. Most of the work is volunteer. If you want ascii doc you'll need to put some elbow grease.
asciidoc is independently maintained https://asciidoctor.org/docs/asciidoctor.js/
you will save the effort needed to maintain a new format.
Sure, great ! Do you also have a Python binding for nbconvert ? There are about a dozen places where we render and sanitize markdown can you go add hooks for asciidoc as well ? There also need to be an option for user to select the formatting they want, and likely store that in the notebook format ; I guess it will take you only a couple of hours to patch it and translate. The docs will need fixing as well.
Once you've done it for notebook, can you do it for Lab, VS Code, Nteract, Cocal, Colaboratory ? Thanks.
Anyone have a quick tldr on the differences between Jupyter Lab, Jupyter Book, and Jupyter Notebooks?
Jupyter Notebook = view notebook as a single page – or depending on context, the document format `.ipynb`. Lab allow you to have panels like an IDE; Jupyter Book is to publish a collection of .ipynb file as a website.