My Projects

or Speech has a Pitch, but what is Pitch Anyway?

My dissertation started from the design of a pitch estimation algorithm, originally destined for quite a different task. Surprisingly, I found the true scientific challenge not the algorithm itself but how to evaluate its accuracy: Traditionally, pitch estimation algorithms are evaluated against a ground truth that is just assumed to be accurate. Pitch is a human perception, not a measurable, physical quantity. As it turns out, these ground truths are based on pitch estimation algorithms themselves, and often ones derived from laryngograph measurements instead of acoustic recordings, which are not ideal for the task. Furthermore, the error measures used for evaluating pitch estimation algorithms are woefully inadequate.

So I set out to design a better ground truth, which does not rely on a single pitch estimation algorithm, but a large number of them, and is based on acoustic recordings. This does not solve the categorical mismatch between an algorithm's estimate and a human perception, but at least it sidesteps the biases of any one algorithm and laryngographs.

With that, I conducted a huge comparison study between a very large number of pitch estimation algorithms and speech databases, and introduced a number of new evaluation methods. This study was unprecedented in the area of research, and produced a number of interesting results. Chiefly among them, that the differences between speech and noise databases used for evaluation far outweigh any minute differences between algorithms. In other words, a strong case of overfitting across the entire area of research.

In the end, the work was more about data science than signal processing. Regrettably, I was not able to publish my results in journals.

My second attempt at getting cross-platform audio to work in Python. Uses CFFI and portaudio, much like SoundFile. But it just didn't work out between portaudio and me. Use SoundCard for a better solution, or SoundDevice for a maintained portaudio bridge. (Status: starting to smell)

Show an asynchronously-loading, interactive map in a Matlab figure. This is an example for my lecture on programming Matlab. (Status: educational)

Draw a smooth, interactive, zooming spectrogram using WebGL and a Python web server. Built as a technology demonstration for WebGL. (Status: probably doesn't even start any more)

Parse and criticize Matlab code. This was meant as a tool for students to help them improve their code quality, but turned out to be too nit-picky to be useful. (Status: works)

A typing tutor in pure Matlab. See how fast you can type in Matlab. Writing a text editing widget from scratch was fun. (Status: fun hack)

Plot violin plots in Matlab. Because everyone is using box plots and thus don't realize that their data is non-gaussian and too sparse. Violin plots show the same thing, but more beautifully and less wrong. (Status: useful)

Annotate arbitrary files as a minor mode in Emacs. Didn't turn out to be quite as useful as I'd hoped, but I learned a lot about Emacs. Install using (package-install 'annotate). (Status: works)

It was literally faster to write my own XML parser for Matlab than to wait for Matlab's own parser to parse my 150 Mb XML file. No support for text nodes, cdata nodes, or any kind of schema validation. (Status: useful)

JSON parsing turned out to be the largest performance bottleneck in Transplant, so I use MsgPack instead. But Matlab lacked a MsgPack parser/dumper, so here is one. (Status: tested and complete)

Call Python from Lua, or Lua from Python. I ported this to Python 3, back in the day. But the original developer was nowhere to be found, so it ended up on my Github. Nowadays it is maintained by a dedicated group of wonderful volunteers. (Status: alive and kicking)

I often work with cyclic phase data, which wraps around such that its highest value is also its lowest value. Matplotlib didn't have a convenient color map for this kind of data, so I created one. And now this color map is part of Matplotlib. (Status: awesome!)

My first attempt at getting cross-platform audio to work in Python. Back then, PyAudio was the best choice, but it didn't support Python 3 yet. So I helped out, and ported it to Python 3. (Status: still maintained)

Analyze audio files with waveform, spectrogram and phase-spectrogram in the browser. Developed as part of a one-semester student project I supervised.

Run psychoacoustic experiments in the browser. A port of Psylab for Python, by @stvol, @zngguvnf, and Nils. Developed as part of a one-semester student project I supervised.

A cross-platform audio mex file for Matlab, based on portaudio. I worked on this back when I was a student. (Status: still alive)