Clojure Debugging ’13: Emacs, nREPL, and Ritz

[NOTE: The release of cider deprecates much of the content here.  I will post an update on Clojure Debugging ’14 early in the near year]

I’m ramping up for a new set of development projects in 2013 and 2014.  My 2010 era setup with slime and swank-clojure is unlikely to remain a viable approach throughout the project.  I’ve decided it is time to join the nREPL community as well as take advantage of some of architecture innovations there which may make it easier to debug the distributed systems I’m going to be working on.

Features I’m accustomed to from common lisp slime/swank:

• Code navigation via Meta-. and Meta-,
• Fuzzy completion in editor windows and the repl
• Documentation help in mini-buffer
• Object inspector.  Ability to walk any value in the system
• Walkable backtraces with one-key navigation to offending source
• Evaluate an expression in a specific frame, inspect result
• Easy tracing of functions to the repl or a trace buffer (in emacs)
• Trigger a continuable backtrace via watchpoint or breakpoint

Only the first three of these features is available in the stock nrepl.  The rest of this post will discuss how to setup a reasonable approximation to this feature set in Emacs using nREPL middleware providers as of May 2013.

nREPL is a tooling framework for allowing editors (clients) to connect to a running Clojure instances (servers) to utilize information in the environment to navigate code, complete symbols, and dynamically evaluate code.  As compared to slime/swank, nREPL is focused on being transport agnostic, and communicates by sending asynchronous command/reply maps between the two components.  It also defines a middleware framework to add functionality on top of the basic definitions of transport and minimal methods to support a REPL.

A Basic Setup

Let’s get the basic nREPL working with the Emacs nrepl.el client and configured with some reasonable defaults.

First, install a fresh Emacs 24.

Configure ELPA using MELPA by editing your ~/.emacs.d/init.el

Restart emacs and do M-x package-list-packages and install: ac-nrepl, clojure-mode, melpa, nrepl, nrepl-ritz, and rainbow-delimiters.

Let’s configure these packages:

Make sure you have Leiningen 2.0.0 or later installed and type lein repl to start a network repl and connect to it from the command line. It will print ‘nREPL server started on port xxxxx’. You can now type M-x nrepl in emacs, enter 127.0.0.1 and type the port number xxxxx to connect.

This basic setup allows you to do a number of useful things:

• REPL-based interaction with your Clojure instance
• Navigate via M-. and M-,
• Get simple backtraces on errors
• Load files C-c C-k
• Describe current symbol via C-c C-d
• Nice auto-completion just about everywhere
• Auto-doc help in mini-buffer for head of current expression
• Visit the current nREPL via C-c C-z

Simple Object Inspector

But that’s not adequately satisfying. Let’s get a little basic inspection going via ‘C-c C-i’. This is a little involved while we’re waiting for the library to be properly packaged, but this is a good example of deploying a new middleware library onto both the clojure server and the emacs client.

git clone https://github.com/vitalreactor/nrepl-inspect.git
cd nrepl-inspect
lein install


Update your init.el after the (require ‘nrepl) statement:

Configure your ~/.lein/profiles.clj:

Restart your nrepl and emacs (or load/eval the file) and reconnect to your repl.

=> (def testing {:foo :bar})
=> testing


Typing ‘C-c C-i’ will prompt you to accept the value in the minibuffer.  You can also type an arbitrary expression which will be evaluated in the currently active nrepl or editor namespace.  You can navigate into sub-structures with Tab, Shift-Tab, and Enter.  Type ‘l’ to pop back to the prior value and ‘g’ to refresh (if you are inspecting something like an atom or agent).  It currently does not truncate long sequences, so beware of infinite seqs or very large maps!

[NOTE: After the initial post, I rewrote Technomancy’s javert library to allow value browsing and extensions and updated the above section to refer to this new library]

Working with Ritz

Hugo Duncan has done a fantastic job of creating rich functional middleware for nrepl.el. There is legacy support for the swank protocol, because this code base was derived from swank-clojure, but that is best ignored these days.

We’ll do the easy stuff first:
M-x package-list-packages
Install nrepl-ritz


Update our init.el:

Update profile.clj to enable the server middleware:

This enables two new commands in nrepl.el: nrepl-javadoc and nrepl-apropos. You can bind those to appropriate keys in Emacs as shown above.

Some of the middleware like “doc” has been ported directly to tools.nrepl so at least some of the ritz middleware is now redundant to the default distribution. Ritz has several other packages that don’t appear to be in nrepl.el or tools.nrepl, but I haven’t evaluated them yet. These include project middleware (update and switch projects without a restart), connection to a codeq database, and source form tracking.

Ritz Debugger

The big missing features described at the beginning are interactive backtraces, breakpoints, watchpoints and tracing. The Ritz debugger supports all but the last, but is a bit more complex to use.

We’ve already loaded the requisite functionality in our profile.clj and init.el. All you need to do to enable the debugging features are to navigate to your leiningen project.clj file and call M-x nrepl-ritz-jack-in. This launches the two processes and starts a debugging session.

You can enable interactive backtraces on errors by doing M-x nrepl-ritz-break-on-exception after jacking in. When exiting backtraces, hit ‘1’ instead of ‘q’. My system locks up if I hit ‘q’. You can navigate stack frames by M-p and M-n, see local variables with ‘t’, go to the source of a frame with ‘v’, evaluate a value in a particular frame with ‘e’, etc. (Use ‘C-g m’ to see all the mode options).

You can also set breakpoints via C-c C-x C-b which pops you into the interactive stack frame. The nrepl-ritz-breakpoints buffer appears broken in 0.7.0.

Finally, you can see a thread list using M-x nrepl-ritz-threads.

The lein-ritz package supports a new lein command: lein ritz which starts a two JVM processes. The first is your standard system, the second is a debugger process that uses JPDA/JDI to debug the first process. (See Hugo’s presentation on the topic)

Attaching to this via M-x nrepl is not working for me at present.

Remote Debugging

How can we use these features when inspecting or debugging remote systems? For now, I recommend sticking with the base nrepl functionality, selected middleware, and avoid ritz until it matures further as the setup is complicated and I don’t fully understand it yet.  If you understand it, comment below and I’ll fix this!

To use the middleware, you’ll need to update your project.clj to include tools.nrepl and any middleware packages in your project dependencies. You can then start a server with middleware as follows:

[NOTE: After the initial post, I realized you need to explicitly require and pass middleware to the start-server invocation.  This was missing earlier.]

By default, it only listens to localhost so you’ll want to setup an SSH tunnel between your Emacs system and your application server with port number.  If you are a PaaS user, then each platform has it’s own quirks.  The Clojure community are heavy Heroku users, so there is some good work specific to that platform.

The big benefit of nrepl is the ability to use different transports to communicate between your client and a remote nREPL server. Both Drawbridge for HTTP/HTTPS and ritz-hornetq are good examples of this. I will write more about remote debugging in a few months after my new project has settled on what approach we’ll use for remote debugging.  I know the Immutant team is trying to add Ritz for remote debugging purposes.

Summary

Slime/Swank for clojure is rapidly passing into obscurity. Switching over today is not absolutely required, but because Clojure developer attention is mostly focused on nREPL now, packages are unlikely to be maintained for older setups. The switch is now inevitable. Fortunately, nREPL is maturing rapidly and projects like Ritz are filling in the gaps.

Read this post in Japanese!