Show HN: Mr. Graph – A graph definition and execution library for Python
mr-graph.readthedocs.ioWhat:
Mr. Graph is a python library designed to make composing graphs of sync and async functions easy! Use google style docstrings to automagically create dataclasses and chain together function calls into graphs.
Why:
I like the design of Dagster, but not the latency. For apps and systems engineering, sometimes I want to compose a graph out of regular python functions. I don;t need all the heavy machinery that comes with a full workflow engine.
Current features:
- Use with either async or sync functions
- Uses google style doc strings to name return values.
- Creates dataclasses for each function's output.
- Can infer pipelines from input and output signatures
- All directed acyclic graph layouts supported. linear, fan-in, fan- out.
Future Features:
- Better examples for use with async calls (like LLMs)
- Splitting dataclasses, better error handling, logging improvements.
This is under active development. Any feedback, interest, or contributions are appreciated. Thanks!
github link: https://github.com/mcminis1/mr-graph Slight tangent: is anyone aware of a library (in Python or JS… or C) for specifying DAG pipielines that supports caching results (based on input)? Make is closest I’ve found, but there’s probably something better out there. Edit: I should look at other build tools, but ideally I want a library. Edit again: I will be running this locally. Forgot to mention that. Maybe Dagster with Assets? https://docs.dagster.io/concepts/assets/software-defined-ass... fn_graph: https://github.com/BusinessOptics/fn_graph (My favorite one ^) pyungo: https://github.com/cedricleroy/pyungo Loman: https://github.com/janushendersonassetallocation/loman Tributary: https://github.com/timkpaine/tributary alkymi (https://github.com/MathiasStokholm/alkymi) is such a library (disclaimer: I'm the author). I built alkymi to help me cache complex data pipelines, while still having all the niceties of Python. I haven’t dug too deep yet, but this sounds exactly like what I was looking for, thanks! There is a mistake in your first example, mixing up m and n. Does this automagical system mean that I need to write docstrings in your preferred style to get the names lined up?