swanky-python

Install slime and slime-star and clone this git repository. Then in your emacs config add:

(add-to-list 'load-path "~/path/to/swanky_python/slimy-python")
(push 'slime-py slime-contribs)

This is the configuration that I use, with vim style keybindings and using doom emacs, consult, and company. It'd be much appreciated if people could contribute sample configurations based on vanilla emacs with standard keybindings, and integrating with other completion frameworks besides company.

With pip:

pip install -e ~/research/swanky_python/

With uv:

uv pip install -e ~/path/to/swanky_python/

You probably want uv pip rather than uv add as the rest of your team might not be happy with the weird new dependency. Though if you're working alone or somehow all collaborators are emacs users ok with trying experimental unstable software, you could use uv add --dev

The -e flag is to install in editable mode, as this is in very early-stage development, and you will inevitably run into bugs in the course of normal usage that you will need to edit and fix.

Python objects are displayed as presentations rather than as plaintext. This includes results in the repl, function arguments and local variables in backtrace buffers, values within the object inspector, and function arguments and return values in trace buffers. This means it shows a text representation of the object, but keeps a reference to the actual python object on the backend, so you can open it in the inspector, copy it to the repl, assign it to a variable, display documentation, and more. You can also add custom actions to the right click menu for presentations of different kinds of objects.

Any presentation can be opened in the inspector, which for most objects will show their attributes, although you can write custom inspector displays for your classes. slime-py-inspector-filter will bring up a transient menu where you can toggle the display of different attribute types like dunder and private ones. A useful shortcut is slime-py-inspect-last-result which will open the inspector on the last evaluation result. slime-py-whos will open an inspector view of all global variables similar to IPython's %whos command, and will refresh it after every evaluation. slime-inspector-describe will open the slime-help page for the currently inspected object.

When editing python code you can evaluate the whole file (slime-compile-and-load-file), the current class (slime-py-eval-class), current function (slime-eval-defun), current block (slime-py-eval-block-at-point), current selection (slime-eval-region), current statement (slime-py-eval-statement-at-point), or current identifier (slime-py-eval-sexp-at-point). This will evaluate within the context of the module associated with the active file. The module that the REPL evaluates within can be changed with slime-repl-set-package.

On any uncaught exception, slime opens an interactive backtrace buffer. Here you can see all stack frames with their arguments and local variables as presentations. With the cursor on a stack frame slime-py-set-repl-to-frame will spawn a repl in the context of that frame, and sldb-show-source will show the source code with the expression that triggered the exception highlighted, after which you can select a region and run slime-py-eval-region-in-frame.

The various slime-help functions show documentation. This is mostly done by introspection on doc strings, type signatures, and attributes in the running process. So for example you can see in the video that python tells us the pytest package distribution is installed and a short description, but we need to import it before we can see documentation for its modules. In the help page for classes, a link is provided to the right of attributes and methods with the class where they are inherited from. imenu is supported for quick navigation within help pages.

slime-list-threads presents a table with information on all running threads. From there you can press d or slime-thread-debug to get a backtrace buffer for any thread.

If an async event loop is running, slime-list-threads will default to presenting a table with information on all asyncio tasks, and slime-py-toggle-threads-tasks switches between viewing threads and tasks. Here you can press x or slime-thread-kill to cancel a task, subject to the limitations of task cancellation in asyncio. In the video you can see hung requests finally return with "Internal Server Error" when we cancel the associated Task. As for threads you can open a backtrace buffer, but it's considerably less useful as async in python is stackless. It uses Task.get_stack() which for suspended coroutines just returns the stack frame where that task was created, and for running coroutines returns the call stack up to the frame where the task was created, but not the currently executing frame inside the Task. To get that we can toggle to the thread view and get the backtrace for the event loop thread.

To use this, the swank backend must be started after the event loop has started, from the same thread.

You can trace functions and methods with slime-trace-dialog-toggle-trace, and untrace all traced functions with slime-py-trace-dialog-untrace-all. All arguments and return values for traced functions will be shown as presentations in the *slime-trace* buffer. However, the presentations work by simply holding on to a reference of the object, not by making a deep copy of it. So while the text in the *slime-trace* buffer is the repr of the object at the time it was passed, by the time you inspect the presentation it may have mutated. For example in the video the file object is closed by the time we inspect it.

At present there's no full debugger. The options for debugging are the repl and backtrace buffer, the trace, thread, and async views, improved print debugging with pp (print presentation), and the integrated AI. You can load tracebacks saved with pydumpling into the interactive backtrace viewer with slime-py-load-pydumpling

In the future I plan to integrate with dape for traditional step debugging, and add more advanced tracing debugging.

To use pp import it with from swanky_python import pp. It's like basic print debugging except that it prints a presentation of the objects rather than text, so you can go back and inspect it, copy it to the repl, and more. Use it by:

pp(object)

or:

pp("text description", obj1, obj2, ...)

When passed a single object it returns the same object, so that any expression can be wrapped with pp(expression).

When the cursor is within any function call, slime-autodoc-mode shows the function signature in the echo buffer, with the current argument highlighted. For completion we use jedi, which does surprisingly good type inference. For example in the video despite function foo being untyped, it sees that it's called with a str and provides str completion options.

We use code adapted from the autoreload extension for IPython to update old references to functions and classes with the new version when code is changed. The goal is to enable interactive development where you never need to restart your program on changes. If you run into situations where you do need to restart python for changes to take effect, please open an issue.

slime-py-import-symbol-at-point speeds up handling imports. By default it uses name the under the cursor, but it will use the region if selected, to allow importing modules with a dot in the name. If the name is a module, it will use import name. Otherwise it will search for the name in all modules and use from module import name, prompting if found in multiple modules. By default it tries to filter to avoid showing options in private modules or where a name has been reimported outside of its package. To show all possible modules, call with a prefix argument. It will add the import statement to the current file, sort the imports, and evaluate the import statement. When run in the repl, it will just evaluate the import statement as there's no file to add it to.

Also if you use a name without importing it and a NameError is raised, and autoimport detects that the name is available for import, then in the backtrace buffer one of the provided restarts will be to automatically import it and retry the evalution request.

It comes with a local AI model which is quite effective for debugging.

Set ~/.slime-secret, add slime-tramp to slime-contribs, and add to your config:

(add-to-list 'slime-filename-translations
             (slime-create-filename-translator
              ;; machine-instance is `uname -n` on the remote
              :machine-instance "remote"
              :remote-host "remote.example.com"
              :username "user"))

Often when we develop a python script that processes a bunch of files or makes a bunch of web requests, it will die with an unhandled exception part way through. This can turn what should be a fast development cycle into an extremely slow one, as we wait minutes for the script to rerun and reach the same point as before, only to crash with a different exception, which we then fix and rerun again. Lisp and Smalltalk addressed this by not unwinding the stack on exceptions, dropping you into a debugger and allowing you to fix the error and resume execution without having to restart your program from the beginning. Eventually I'd like to patch CPython to support this, but for now I have a much more basic solution that somewhat addresses the problem in practice. Normally when writing a script it's clear what function represents a unit of work that could fail, ie the function to process a single file or make a web request. You can add a @swanky_python.restart decorator to that function to be presented with an interactive backtrace buffer on unhandled exception. At that point you can edit and fix the function and retry execution of just that function call, ignore the exception and have the function return None or any other value, reraise the exception, or abort execution. This makes writing quick and dirty scripts quicker. You can just code the happy path rather than worry about handling every possible error. Then when you get some data that causes an exception, you can fix your code to deal with it and resume execution without having to wait for it to rerun from the beginning.

This can be combined with other python retry decorators like tenacity. Our swanky_python.restart decorator will simply be ignored if you aren't currently connected to swank from emacs. So for example you could put the @swanky_python.restart decorator inside the tenacity @retry decorator to drop into the interactive backtrace buffer on exception if we're connected, and if not then try to retry with tenacity. Or we could use it the other way around to first have tenacity retry a few times before bothering us with manually looking into the exception.

Basic refactoring support is provided through integration with jedi:

1. Inline variable at point with slime-py-refactor-inline
2. Rename variable at point with slime-py-refactor-rename
3. Extract expression around point, or the selected region, to a variable with slime-py-extract-variable. Extract it to a function with slime-py-extract-function.

In the future I plan to integrate rope for more advanced refactoring support.

When the swanky python backend listens for a connection on TCP, the first message it receives needs to contain the contents of ~/.slime-secret

When you are running python and emacs as the same user, you don't need to worry about this, as emacs will generate a random ~/.slime-secret file if it doesn't exist, readable only by the current user.

For remote development, you'll need to manually create a ~/.slime-secret file where python is running, with the same contents as ~/.slime-secret for your emacs user. This is to ensure some other user can't connect to your python process and run code in it.

If you're connected via swank to a python process on some server or in some untrusted container, you don't want it having access to your emacs process. The python process can send elisp for emacs to evaluate if slime-enable-evaluate-in-emacs is enabled, although it's off by default. Also in order to let jedi infer types and provide better completions, we allow python to request the contents of the active emacs buffer. Emacs will only send it if it's in python-mode or slime-repl-mode, but be aware that if you're connected to swank running in an untrusted container or environment you shouldn't open sensitive python code from another project.

If you find any other way that the swank backend can run code in or retrieve information from emacs, I consider it a security issue so please let me know.