The full scientific paper associate to this project is available here.
This project documents the Neurorack, which is a deep AI-based synthesizer based on the Jetson Nano development kit in a EuroRack format. The following diagram briefly explains the overall structure of the module and the relations between the hardware and software (green) components.
The hardware part features 4 CV and 2 Gates (along with a screen, rotary and button for handling the menus), which all communicate with specific Python libraries. Note that the behavior of these controls (and the module itself) is highly dependent on the type of deep model embedded. For this first version of the Neurorack, we implemented a descriptor-based impact sounds generator, described in the software section, later in this document.
Everything in this project is under the CC NC-BY-SA 4.0 licence, which means you can adapt, share, tweak, dance with, destroy anything here as long as there is no commercial use involved.
You can find in the board/ folder the board design and different schematics for wiring the hardware prototype. More detailed informations and tips are available in the wiki. We provide here just a quick BOM to help out.
- Jetson Nano developer kit
- Digital Analog Converter (DAC)
- Analog-Digital Converter (ADC) - Pimoroni ADS1015
- Screen - Pimoroni LCD (240x240)
- Rotary Encoder - Pimoroni RGB Breakout
- Jack Connectors (x6)
- 40pins jumper cable
- 40pins connector kit (fc40p)
- Soldering header pins kit
You can find in the code/ folder all the necessary Python code for running the Neurorack, along with the deep model based on a modified Neural Source-Filter (NSF) architecture. At this time, the first CV controls interpolation between different points of the latent descriptor space, and the three remaining CVs directly control high-level descriptors (loudness, centroid, inharmonicity). The first gate is used to output the corresponding current impact.
You can find our cryptic demonstration video on YouTube
https://www.youtube.com/watch?v=64VpQenCHVs
- Esling, P., & Devis, N. (2020). Creativity in the era of artificial intelligence.. Keynote paper at the JIM Conference 2020 arXiv preprint arXiv:2008.05959.
- Esling, P., Devis, N., Bitton, A., Caillon, A., & Douwes, C. (2020). Diet deep generative audio models with structured lottery. International Digital Audio Effects (DaFX 2020) Conference. arXiv preprint arXiv:2007.16170.