Settings

Theme

Closer to production quality Python notebooks with `marimo check`

marimo.io

31 points by dmadisetti 2 months ago · 23 comments

Reader

wodenokoto 2 months ago

I’m very interested in moving beyond Jupyter notebooks so I have my eye on marimo.

My understanding is it’s just python code with a bit of notebook hints and an idea that gives the notebook experience.

So I didn’t quite catch from the article why ruff and pylons aren’t enough.

  • akshayka 2 months ago

    Hi! Thanks for your interest. marimo is much more than that — unlike traditional notebooks, marimo is "reactive", meaning it models notebooks as dataflow graphs and keeps code an outputs in sync. Moreover, marimo notebooks are not "just notebooks". They can be seamlessly run as interactive web apps or as Python scripts.

    Here is our original ShowHN post that explains what marimo is all about: https://news.ycombinator.com/item?id=38971966

    Blog that goes deeper: https://marimo.io/blog/lessons-learned

    • daxfohl 2 months ago

      I was just thinking about something like this a couple hours ago.

      Jeremy Howard (fast.ai, etc) posted a few weeks ago about his new AI class [1] and I was pumped. I went through the fast.ai course a couple years ago and had been wondering if another would ever come out, and the overview of this sounded really interesting. But the first thing the announcement went into was Jupyter Notebooks, and it kept on and on about them and I rolled my eyes. I remember they were my least favorite part of the fast.ai class. They've always felt awkward to me, being used to a code editor and all the functionality it provides, and I'm sure I'm not alone.

      I'm sure Jupyter is great for fast iteration if that's what you do all day, but if you spend most of your life in a standard code editor, you really want something that starts from there and adds a bit of interactivity on top, rather than jumping to a completely different concept. I haven't tried marimo yet, but I might give it a shot for my next project.

      [1] https://news.ycombinator.com/item?id=45455810

      • spwa4 2 months ago

        Just use VSCode, which will give you full language server editing in ipynb files?

        • daxfohl 2 months ago

          Maybe. At least that way it doesn't feel like some separate half-baked proprietary code editor from the 90s, which is the vibe I get from Jupyter. But a regular .py file with a couple attributes that can generate notebooks still sounds far nicer, assuming it works fairly seamlessly.

  • ravila4 2 months ago

    I haven’t used it, but from looking at Marimo’s examples and docs, I’m not convinced by some of its design choices. The idea that you can run notebook cells out of order is supposed to be a strength, but I actually see it as a weakness. The order of cells is what makes a notebook readable and self-documenting. The discipline of keeping cells in order may be painful, but it’s what makes the flow of analysis understandable to others.

    Also, I find the way Marimo uses decorators and functions for defining cells pretty awkward (Although it’s nicely abstracted away in the UI). It looks like normal Python, but the functions don’t behave like real functions, and decorators are a fairly advanced feature that most beginners don’t use.

    For me, Quarto notebooks strike a better balance when it comes to generating sharable documents, prototypes, and reports. They’re git-friendly, use simple markup for defining cells, and still keep the clear, linear structure.

    However, Marimo might be the best tool for replacing Streamlit apps and “production notebooks” (Although I’d also argue that notebooks should not be in production).

    • dmadisettiOP 2 months ago

      marimo has a quarto extension and a markdown fileformat [1] (marimo check works on this too!). The python fileformat was chosen such that "notebooks" are still valid python, but yes- the format itself is almost an implementation detail to most "notebook" users. Cells _are_ actually callable and importable functions though (you can give them a name), but the return signature is a bit different from what's serialized.

      > The discipline of keeping cells in order may be painful, but it’s what makes the flow of analysis understandable to others.

      We might have to agree to disagree here, you can still chose to have your notebook in order and something you can be disciplined about. The difference is that a marimo notebook can't become unreproducible the same way a jupyter notebook can, _because_ the order doesn't matter.

      But thanks for the feedback!

      [1]: https://github.com/marimo-team/quarto-marimo

dmadisettiOP 2 months ago

marimo notebooks double as python Python programs, which means they can be lint-ed like any other code. `marimo check` catches notebook-specific issues (variable redefinition across cells, circular dependencies) with Rust-inspired (and uv; uv is amazing) error messages. It outputs JSON for AI agents to self-correct, integrates into CI pipelines, and includes `--fix` flags for automatic repairs. The team is already using it in their own CI and seeing Claude Code iterate on notebooks without human intervention.

  • sixo 2 months ago

    Unfortunately the code in Marimo notebooks implements cross-variable references as untyped function arguments, meaning that nothing passed between cells can be type-safe. This makes it very hard to use tooling directly on the Python representation of notebooks.

    • dmadisettiOP 2 months ago

      Arguments are typed if explicitly specified in definition, e.g.

          ```python
          @app.cell
          def my_cell():
              x : int = 1
              return (x,)
      
          @app.cell
          def consumer(x : int):
              y = x
              return
          ```
      
      We've talked about building out implicit typing into our serialization- but that may be a larger can of worms.
      • sixo 2 months ago

        That would be wonderful. A lot of the arguments are just imports, and these would have probably have the largest upside. It would be great if there was a construct like

            import numpy
        
            @app.cell
            def my_cell(np: types.ModuleType[numpy]):
                ... editor treats np like `import numpy as np` ...
        
        
        I do not use Python enough to know if something like this can be hacked together. But if not, I imagine could be PIPed into the standard library if a convincing case were made for it.
        • dmadisettiOP 2 months ago

          We've addressed this too by allowing a setup cell:

              ...
          
              with app.setup:
                  import numpy as np
              
              @app.cell
              def cell_uses_np():
                  # no np needed in signature
                  result = np....
          
          The best part about this pattern is that it enables "pure" (only dependent on setup cell) functions to be addressable:

              with app.setup:
                  from typing import Optional, Any
          
              @app.function
              def my_fn(arg: Optional[Any]) -> Any:
                  ...
          
          but nice convergent design :) glad to see to see we're addressing pain points
          • sixo 2 months ago

            ah, I recall seeing that feature but had not taken a look at it for imports.

            Imports ARE the case where untyped cell arguments are most annoying, but of course it would be nice to get it for free in all cases.

  • aitchnyu 2 months ago

    IME Agents work fine on human readable error messages.

ZeroCool2u 2 months ago

Any updates on the revamped VS Code extension? That's kind of a deal breaker for me.

loveparade 2 months ago

I wish this didn't have AI in it. I've been looking for a Jupyter alternative that is pure python and can be modified from a regular text editor. Jupytext works okay, but I miss the advanced Jupyter features. But I really don't want to deal with yet another AI assistant, especially not a custom one when I'm already using Claude/etc from the CLI and I want those agents to help me edit the notebooks.

Take out all the AI stuff and I'd give it a try. I use AI coding agents as my daily driver, but I really don't need this AI enshittification in every tool/library I'm using.

  • daxfohl 2 months ago

    Reading the article, I don't think it has AI. They've just made the tools in a way that AI assistants can also use them, and so fix linting errors without anyone needing to fine-tune the LLM on the syntax.

    That's actually pretty slick. I've been wondering how we could avoid blocking innovation in programming languages because of the death cycle of "no training data on language -> LLM can't learn language -> Assistant can't code language -> nobody uses language -> no training data on language".

    • loveparade 2 months ago

      Yeah, reading the docs it seems you are right. The landing page mentions AI-native at the very top and all over the place, so I got the wrong impression that it's somehow tightly coupled to an AI integration. But looks like it's optional.

  • dmadisettiOP 2 months ago

    > But I really don't want to deal with yet another AI assistant, especially not a custom one when I'm already using Claude/etc from the CLI and I want those agents to help me edit the notebooks.

    So funny story- you can use exactly the same CLI tools in your notebook. Zed built out the ACP spec [1] which lets Claude Code go anywhere that implements it (as of Oct 2nd; emacs, vim, zed and marimo [2])

    [1]: https://github.com/agentclientprotocol/agent-client-protocol

    [2]: https://zed.dev/blog/acp-progress-report

  • 3eb7988a1663 2 months ago

    I hate how much I lean into VSCode, but the Python interactive mode gets you a really good live coding environment. Instead of Jupyter cells, you have a regular .py file with chunks of code prefixed with a `# %%`. VSCode gives you a similar experience to a notebook, with the same controls (Run Above Cells, Restart and Run All, etc). So something like

      # %%
      import polars as pl
    
      # %%
      df = pl.DataFrame()
      df.shape
    
      # %%
      def foobar():
        return 1
    
    Since it is a regular .py file all of your existing tooling will work with it. The one thing you lose vs a Jupyter notebook is saved output. I mostly use these .py files, but have a few .ipynb notebook files for when I want to commit the output from some important task.
  • akshayka 2 months ago

    You can not use the AI features, nothing is enabled by default (you have to bring your own keys)

Keyboard Shortcuts

j
Next item
k
Previous item
o / Enter
Open selected item
?
Show this help
Esc
Close modal / clear selection