Synchronized Synthesis

A music synthesizer enabled by the synchronization of *many* (~O(10^3)) coupled oscillators

Synchronization Synthesis

This is a synthesizer made using *many* (~O(10^3)) coupled oscillators. The idea is to create a synthesized sound that can 'organically' *morph between noise and pure tones*, using mechanisms inspired by biological fluid dynamics of ciliated surfaces. Specifically, each oscillator goes around an elliptical trajectory under prescribed speed profiles, with their motion coupled together due to hydrodynamic interactions. For different parameter values, oscillators can move together in a single traveling wave, in waves with local defects, without much coordination (noisy) or with other higher order patterns. The phase of each rotor are read-out in a wavetable synthesizer fashion to produce the final sound, or made to filter the input stream.

The research behind the biological fluid dynamics behind this work is published in our PNAS paper

A. V. Kanale, F. Ling, H. Guo, S. Fürthauer, E. Kanso, Spontaneous coordination of beat patterns in carpets of rotary cilia (2022) https://www.pnas.org/doi/full/10.1073/pnas.2214413119

Arxiv version: https://arxiv.org/abs/2208.10811

Fluid solver is based on the regularized stokeslet method:

J. Ainley, S. Durkin, R. Embid, P. Boindala, R. Cortez, The method of images for regularized Stokeslets (2008) J. Comp. Phys. 227(9):4600-16

Video explanation:

https://www.youtube.com/watch?v=iccluAT5gwE

https://drive.google.com/file/d/14k-dnNt4iV-dgKk8gtQpsFtku44MU0l-/view?usp=sharing

#Package includes:

(1) MATLAB audioplugin source code

(2) MATLAB code for an interactive GUI synthesizer with a built-in keyboard

(3) demo sound track

#Usage:

(1) run in MATLAB 'audioTestBench(synchronizedSynthesisPlugin)'

compilation tested in R2021a with VC++ 2017 Community Edition 15.9.36 (need Windows SDK 10.0.15063.0)

backward compatiable version for R2018a also included (synchronizedSynthesisPlugin_backcompat)

running plugin in MATLAB also provide a rudimentary visualization of all oscillator phases

the generated plugin has been tested on Windows, both as *.dll in DAW and *.exe

on Ubuntu 18.04, *.dll is tested to work with kxstudio Carla and *.exe with wine-stable

in the default additive mode, change the "Play Note" slider to hear the current plugin state

for more detailed documentation of the parameter interface, see below

(2) run in MATLAB 'synchronizedSynthesis' to open the main window

to open the help menu, press '?' key or press the '?' button

to open the synth sound envelope control, press '+/=' key or press the '+' button

(3) short demo track "RotoryWaves.mp3" created by recording (3) using the default pad envelope (Shift-2)

#Plugin Parameter Guide:

Dry/Wet - controls bypass between input stream and output stream

Stepsize - changes the integration time step for the fluid solver

larger value should produce steady states faster in absence of noise/jitter

Angle - modifies the angle maximum distortion faces for each oscillator

Strength - controls the range of interaction/synchronization between oscillators

under the hood, it corresponds to the distance between the oscillator plane

and the no-slip wall, thus modifies the hydrodynamic decay from O(1/r^3) to O(1/r^2)

Circularity - controls the circularity of the oscillator trajectory, 0 = rower, 1 = circle

h1 - h5 - encodes the amplitude for each forcing distortion applied to the oscillators

with only h1 > 0, the steady state yields short wavelength traveling waves with defects

with only h2 > 0, the steady state yields clean long wavelength waves

positive h3 to h5 give higher order synchronization patterns (e.g. spiraling waves)

Jitter - the main parameter that dictates how much noise is built in the system

higher jitter introduces noise in oscillator position, angle, frequnecy

Density - changes the number of oscillators running at one time

running outside MATLAB, each change will also reset the phases to uniform noise

Scale - changes the frequency of generated sound according to a selected scale/sequence

'chroma' - the equal tempered chromatic scale

'major' - major scale

'minor' - minor scale

'dharm' - double harmonic scale

'queue' - a simple two chord progression

Frequency - changes the baseline frequency of the generated sound

Stereo Spread - addes a phase lag between left and right output channels for the generated sound

Play Note - changes the output frequency according prescribed scale

Reset - toggles between 'noise' and 'waves', each press will reset rotor phases to

either uniformly distributed noise or a randomized traveling wave, respectively

Freeze - freeze the interaction between oscillators so they only evolve acoording to

their individual parameters (governed by stepsize, angle, h1-5, circularity, jitter)

Filter - when 'on', the input stream is applied as an envelope to the

generated sound; otherwise the output is weighted sum of input and phase read-outs

this acts as a work-a-round for the creation of more complicated sound / using MIDI

e.g. string any VSTi before this plugin in filter mode transfers the lo-fi timebre

from this synth engine to the original midi generated output

(NOTE: currently there is no pitch correction so there will be frequency shifts)

Decay - automatically adds a decay to the generate sound when NOT in filter mode

Direction - changes the default oscillator spinning direction from ccw to cw

#To-do / Known Issues:

VST plugin could face performance issues in DAW deployment (esp. through Carla & wine in Linux)

after running for a long time with Freeze toggle turned off (synchronization in action).

pitch correction is needed as currently frequency is changed based on time shifting only

Neither the plugins nor the MATLAB keyboard synth have been throroughly tested on MacOS

Feng Ling © 2021/05/20