Making music together

If you're not a musician, and if you've ever tried to play an instrument, you might be familiar with the feeling of wanting to bring a musical idea to life but not being able to do so. Music is something that most of us enjoy as listeners, but only some of us have the experience and skills required to make it. These considerations are at the heart of this project, which investigates a way to make collaborative instruments that makes it easy for inexperienced users to compose music together in real time.

How it works

The network is composed of shapes that serve two purposes: two shapes work as instruments, the other two function as filters that change various attributes of the composition. The pyramid controls the drums. The user triggers a series of looping drum patterns by flipping the pyramid to each of its sides. The cube controls the synth. In this case, when the user moves the cube a "one-time" pattern is triggered. This means that the user controlling the cube must continuously flip the cube to other sides to play the synth.

The effects are controlled by two cylinders. The taller cylinder lays on the round side and can be rolled back and forth to control how bright the overall composition sounds. Under the hood, this shape is controlling a low-pass filter. The shorter cylinder is can be used as a big knob and controls the tempo of the music.

Hardware setup

The hardware setup consists of four Puck.js configured to advertise accelerometer data through BLE. The BLE advertisements are picked up by an Arduino Nano 33 BLE Rev2, which has the tasks of converting the BLE advertisements into MIDI signals. The MIDI signals are then read by VCV Rack, an analog synth emulator that is running on the laptop.

This setup was chosen partly because of its simplicity and partly due to the resources available to us. A big limitation of this setup is latency, especially when a lot of Bluetooth devices are in the surroundings. Luckily, during our testing we were able to mitigate interferences both by playing in isolated environments but also by configuring the VCV Rack patch to mitigate latency issues.