Basti's Scratchpad on the Internet
22 Apr 2015

Matlab and Audio Files

So I wanted to work with audio files in Matlab. In the past, Matlab could only do this with auread and wavread, which can read *.au and *.wav files. With 2012b, Matlab introduced audioread, which claims to support *.wav, *.ogg, *.flac, *.au, *.mp3, and *.mp4, and simultaneously deprecated auread and wavread.

Of these file formats, only *.au is capable of storing more than 4 Gb of audio data. But the documentation is actually wrong: audioread can actually read more data formats than documented: it reads *.w64, *.rf64, and *.caf no problem. And these can store more than 4 Gb as well.

It's just that, while audioread supports all of these nice file formats, audiowrite is more limited, and only supports *.wav, *.ogg, *.flac, and *.mp4. And it does not support any undocumented formats, either. So it seems that there is no way of writing files larger than 4 Gb. But for the time being, auwrite is still available, even though deprecated. I tried it, though, and it didn't finish writing 4.8 Gb in half an hour.

In other words, Matlab is incapable of writing audio files larger than 4 Gb. It just can't do it.

15 Apr 2015

Unicode and Matlab on the command line

As per the latest Stackoverflow Developer Survey, Matlab is one of the most dreaded tools out there. I run into Matlab-related trouble daily. In all honesty, I have never seen a programming language as user-hostile and as badly designed as this.

So here is today's problem: When run from the command line, Matlab does not render unicode characters (on OSX).

I say "(on OSX)", because on Windows, it does not print a damn thing. Nope, no disp output for Windows users.

More analysis: It's not that Matlab does not render unicode characters at all when run from the command line. Instead, it renders them as 0x1a aka SUB aka substitute character. In other words, it tries to render unicode as ASCII (which doesn't work), and then replaces all non-ASCII characters with SUB. This is actually reasonable if Matlab were running on a machine that can't handle unicode. This is not a correct assessment of post-90s Macs, though.

To see why Matlab would do such a dastardly deed, you can use feature('locale') to get information about the encoding Matlab uses. On Windows and OS X, this defaults to either ISO-8859-1 (when your locale is pure de_DE or en_US) or US-ASCII, if it is something impure. In my case, German dates but English text. Because US-ASCII is obviously the most all-encompassing choice for such mixed-languages environments.

But luckily, there is help. Matlab has a widely documented (not) and easily discoverable (not) configuration option to change this: To change Matlab's encoding settings, edit %MATLABROOT%/bin/lcdata.xml, and look for the entry for your locale. For me, this is one of

<locale name="de_DE" encoding="ISO-8859-1" xpg_name="de_DE.ISO8859-1"> ...
<locale name="en_US" encoding="ISO-8859-1" xpg_name="en_US.ISO8859-1"> ...

In order to make Matlab's encoding default to UTF-8, change the entry for your locale to

<locale name="de_DE" encoding="UTF-8" xpg_name="de_DE.UTF-8"> ...
<locale name="en_US" encoding="UTF-8" xpg_name="en_US.UTF-8"> ...

With that, Matlab will print UTF-8 to the terminal.

You still can't type unicode characters to the command prompt, of course. But who would want that anyway, I dare ask. Of course, what with Matlab being basically free, and frequently updated, we can forgive such foibles easily…

12 Apr 2015

Decisions in Pillars of Eternity

Early on in Pillars of Eternity, you are tasked to kill one of two characters: Either you kill King Rethoric, who executed many innocent people, or you kill Rolsc, the leader of the rebellion. This is an interesting moral choice, but it is also profoundly sad that the game presents killing either of them as the only resolution to this conflict.

It got me thinking. In the real world, I would not consider killing to be an option, ever. But in the game, you have to play by the game's rules. And furthermore, you can't just walk away and have the two characters duke it out amongst themselves: The whole game is built around the player, and without the player's interaction, the game world doesn't evolve.

Thus, this conflict won't resolve itself, and the player is forced to kill. I wish there were a diplomatic option, or a way of fixing the underlying problem so the two characters are not at odds any more. I fear how these choices in video games might influence our perceptions of everyday choices, and crave for video games that offer interesting choices that do not revolve around murder.

From what I hear, Planescape Torment did offer such choices. As did many of my favorite games of all times.

12 Jan 2015

Julia First Impressions

Julia is a high-level dynamic programming language designed to address the requirements of high-performance numerical and scientific computing while also being effective for general purpose programming. —Wikipedia

In other words, it is supposed to be as fast as C, as practical as Python, and as scientific as Matlab. The next step in mainstream scientific computing.

However, Julia is still very young, and still evolving rapidly. But if any of the above is true, I am very interested!

So, over the last few days, I re-implemented an algorithm in Julia. The previous version of the algorithm was written in Python. The algorithm spends most of it's time in FFTs, thus I didn't expect big performance gains.

It is actually nice to have a language that is built for scientific computation. Coming from Python, it is refreshing to have array literals, ranges and mathematics available without importing anything. Much like Python, Julia also has a proper module system, comprehensions, and more than one function per file.

On the other hand, there are a few questionable design decisions as well. Julia uses 1-based, inclusive indexing (range[1:3] = [1 2 3] as opposed to Python's range[:3] = [0, 1, 2]). In practice, my experience is that I rarely need to add ±1 when indexing in Python, but I frequently need it in Julia/Matlab.

Also, Julia has no docstrings, which makes me sad. The whole documentation story is sad, really: Documentation is often incomplete, or missing altogether. Unit testing is not widespread at all, and still crude. 0.4 will apparently add docstrings using macros. This is ugly and doesn't work for one's own code, but it's certainly a step in the right direction. Those error messages would need some work, too.

All of that is hopefully just a symptom of Julia being young, and will improve over time. Similarly, signal processing functions are missing entirely, and I had to re-implement some. The community is growing rapidly, though, and a lot of missing functionality can be installed through the package manager already.

When it comes to writing code, there is a lot to like about Julia. Julia's type system does impose a bit of overhead, but it also grants immediate benefits: Many of Python's runtime errors happen at evaluation time and functions can easily document their arguments' types. Also, thinking strictly about types actually improved performance by a good 20% in my algorithm.

And finally, Performance is surprisingly good! Although my algorithm spends almost all of its time doing FFTs, Julia performed about twice as fast as Python. This is pretty much exactly twice as much as I had expected! It also exposed more of the features of the underlying libraries, which could be used for another speed-up of some 10%.

At the end of the day, Julia clearly isn't mature yet, but very promising. Documentation and libraries will no doubt grow, and performance is already excellent. I'll definitely keep an eye on it, and will experiment further when the opportunity presents itself.

01 Jan 2015

The 31. Chaos Communication Congress

Every year, between Christmas and New Year's Eve, a very special convention is held: The Chaos Computer Club hosts the Chaos Communication Congress. Among my peers, this is just "The Congress", the real highlight at the end of our year.

Don't be fooled by the name though, The Congress is not just some conference, with talks and meetings and business cards. The Congress is where every internet citizen, technologist, hacker, DIYer, net-politician, programmer, computer scientist, or geek convenes to exchange ideas. It is a place of levity, amazement and enlightening.

CCH.jpg

The Congress lasts four days, and every day is packed full of exciting talks. My favorites:

All of these talks and many more are also available online, and I highly recommend you watch some of them. But The Congress is not just talks; this is where all the German hackspaces and user groups meet and present their projects. Walking through the halls, there are laser shows, quadrocopters, 3D printers, stitching machines, automatic cocktail mixers, mechanical and human musicians, and so much more.

hand.jpg

At its core though, this is a place where a bunch of like-minded people meet and take over a convention center. The Congress happens in the CCH, the Hamburg Convention Center. Over the course of the four days of The Congress, this building transforms from a sterile business location to a colorful geek-party. Within a day, all the lights in the building are somehow turned colorful, a pneumatic tube network sends messages across the building, there are lounges and coffee rooms, and strange and beautiful sculptures of light and electronics show up all over the place. Just walking across this cornucopia of light and sound is a treat for the senses.

And this is just the natural thing that happens if you let a bunch of creative people do their thing. It is really a crowd like no other: Nowhere else have I seen such an open-minded and approachable crowd. There are no unpleasant drunk people, no fights, no harassment. All over the place, people talk to strangers, adults and children build things together, and men and women interact as equals.

The Congress is really unique. I can't recommend it enough. If you can free up those four days between Christmas and New Year's Eve at all, go there. You won't regret it!

Older posts