Audio Signal Processing for Music Applications

Audio signal processing is an engineering field that focuses on the computational methods for intentionally altering sounds, methods that are used in many musical applications.

We have tried to put together a course that can be of interest and accessible to people coming from diverse backgrounds while going deep into several signal processing topics. We focus on the spectral processing techniques of relevance for the description and transformation of sounds, developing the basic theoretical and practical knowledge with which to analyze, synthesize, transform and describe audio signals in the context of music applications. 

The course is based on open software and content. The demonstrations and programming exercises are done using Python under Ubuntu, and the references and materials for the course come from open online repositories. We are also distributing with open licenses the software and materials developed for the course.

Course Syllabus

Week 1: Introduction; basic mathematics 
Week 2: Discrete Fourier transform
Week 3: Fourier transform properties
Week 4: Short-time Fourier transform
Week 5: Sinusoidal model
Week 6: Harmonic model
Week 7: Sinusoidal plus residual modeling
Week 8: Sound transformations
Week 9: Sound/music description
Week 10: Concluding topics; beyond audio signal processing

Instructors

Xavier Serra, Professor, Dept. of Information and Communication Technologies, Universitat Pompeu Fabra of Barcelona

Prof Julius O Smith, III, Professor of Music and (by courtesy) Electrical Engineering, CCRMA, Stanford University

Recommended Background

The course assumes some basic background in mathematics and signal processing. Also, since the assignments are done with the programming language Python, some software programming background in any language is most helpful. 

Suggested Readings

The main software tools used are in https://github.com/MTG/sms-tools and the sounds to be studied come from https://freesound.org. Most of the external references come from Julius O Smith website, https://ccrma.stanford.edu/~jos, or from https://www.wikipedia.org.

FAQ

• How much programming background is needed for the course?
  All the assignments start from some existing Python code that the student will have to understand and modify. Some programming experience is necessary.
• What is the coolest thing I'll learn if I take this class?
  You will play around with sounds a lot, analysing them, transforming them, and making interesting new sounds.