A few weeks ago I started working on s-rack, a modular synthesizer written in Rust. An earlier post describes my first week of development.
The big news since then is that this project now runs inside a web browser through WASM as a compilation target.
I followed the example at eframe_template.
I found that Trunk is a builder for
WASM apps. During development it will watch your code, recompile it when it
changes, inject a link to the WASM binary into an HTML template you
provide, and cause the page to reload. It will also generate a production
deployable build in a dist/ folder when you are ready.
I refactored out the audio setup code into its own struct and associated
functions so that the code in main -- which is different on native than
in WASM -- doesn't have to be concerned with it. After making this change
and making it so the audio engine only starts after a user interaction on web,
the WASM version of s-rack was building without any issue.
Input and output handles
When I wrote about this project a week ago, I anticipated that I would need to create widgets with drag and drop behaviors to act as the input and output "plugs" next to modules.
I found that with egui it was very easy to create widget like objects that
could take an egui Ui, call its .allocate_space(), and test for interactions
with the pointer including with drag and drop.
In fact, it was extremely easy to implement dragging an output into an input or
vice versa, as the Response.dnd_* functions allow payloads of any type. For
example, I used Response.dnd_set_drag_payload on each port widget to make
the information about the port (a pointer to the module and which port it is)
draggable to other ports that can receive it with Response.dnd_release_payload.
Oscillator aliasing
As I predicted, naively generating waveforms with fast transitions like a square wave or sawtooth wave results in undesired artifacts -- extra tones which seem unrelated to the tone being generated.
Luckily, adopting a PolyBLEP function to generate corrections for each hard transition turned out to be really easy and worked like magic.
Warning! Annoying noises!
Execution planning
Though after the user has built a patch a module may have more than one (or no) connections to other modules, when the audio driver asks for audio, we want to run each module's audio processing code once and only once. Additionally, we want to run the modules in an optimal order.
Consider the graph above, made up of modules. (Direction of arrows signify a dependency, not the direction of signal flow, which is reverse.) Modules take input buffers of a certain size, process them together with their internal state, and produce one or more output buffers. A produces an output buffer consumed by B.
Periodically the sound library asks for a new buffer of audio to send to the sound driver, which triggers the execution of all modules in the order determined by an execution planner which I created. My initial execution planner was pretty naive and only tried to walk the graph from the output module (E), making sure each module was only executed once. If the modules are processed in the wrong order, as was sometimes the case, delays are introduced, as the output buffers of dependencies may not have been generated yet. They would generate later in the execution only to have their buffers read in the next cycle.
If the modules are run in the right order, the processed signal from the first module's output (A) should be observable at the last module (E) within the same poll from the sound library. A must run before B, B before C, and C before E. D must be run after A and before E.
To fix this problem, I wrote a test case describing something similar to the above ordering example and I introduced topological sorting. But this introduced another problem, as you can only topologically sort acyclical graphs and I wanted the ability to create cycles within my synth patches, even if they introduced a small delay.
In the graph above, there is a cycle which we must break before we topologically sort. I decided that to do this the planner would start a depth first search at the output module E. For every module encountered in the search, the planner starts a new depth first search at the module and if the innner search encounters the module that the inner search started at, the edge that lead to the module is removed. The module is tested for additional loops until none exist. Then the outer search continues. Finally, any modules in disconnected subgraphs are checked for and disconnected from cycles.
In the graph above, first E is checked to see if it is a part of a cycle. (It is not.) Then D is checked. D is part of a cycle (DCBF) and the edge between F and D is detected as leading to D being in the cycle, so that edge (for the purposes of execution planning) is removed. The rest of the nodes are checked, and because we disconnected the edge causing the cycle that D was a part of, none of the other nodes checked in the search are determined to be a part of cycles.
This satisfies the test case I wrote where cycles are disconnected as close to the output as possible and in casual play with the software, this appears to keep all signals in sync.
Serialization and Deserialization
Within the past week, I also added the ability to save and load patches
using Serde and rmp-serde (MessagePack).
Anticipating breaking changes, I intend to keep older versions of module structs
around and write From/Into implementations so that they can be migrated
forward.
More modules
I implemented a small collection of modules, including VCA, ADSR, a sequencer, and a sampler.
Removing pain around Arc<RwLock<T>> with closures
I noticed that I was spending a few lines of code read locking all of the input
buffers and write locking all of the output buffers -- for every buffer!
Inspired by egui's use of closures, I made an AudioBuffer type that holds
buffers, and has several methods that take functions to wrap with the read or
write locking with the buffers. For example, here is the "Mono Mixer" module's
calc function, which uses AudioBuffer::with_read_many and the with_write
method:
use itertools::Itertools; // provides .collect_vec
impl SynthModule for MonoMixerModule {
// ...
fn calc(&mut self) {
AudioBuffer::with_read_many(
(0..self.get_num_inputs())
.into_iter()
.map(|n| self.resolve_input(n).unwrap())
.collect_vec(),
|bufs| {
self.buf.with_write(|output| {
let output = output.unwrap();
output.fill(0.0);
for (buf, gain) in bufs.into_iter().zip(self.gain.iter()) {
if buf.is_none() {
continue;
}
let buf = buf.unwrap();
for (src, dst) in buf.iter().zip(output.iter_mut()) {
*dst += src * gain;
}
}
});
},
);
}
}
Closing thoughts
This is the first project I made in Rust that amounted to anything, and through trial and error, I feel I'm really starting to grasp the language and understand why people love it so much. That said, I'm still learning the idioms and I'm already seeing ways in which the code I wrote weeks ago is naive or could be written better.
I think a current weakness with Rust is that some of the "Rusty" ways of writing code are left to be discovered instead of collected in documentation. Python for example is well known for its opinion that there is a "right way" to do something: some ways of writing code are more "Pythonic" -- and are well documented. With Rust, on the other hand, I feel I've had to discover and absorb these kinds of idioms by reading lots of Rust source code.
There's still a lot to do on the project, but I've implemented the fundamentals of a nodal UI for a modular synthesizer, so I'm not sure I'll have too many more written updates here going forward.
Follow the project at https://github.com/sharph/s-rack!