Posts tagged "macos":
OS Customization and MacOS
I was an Apple fanboy some years ago. Back then, whenever something was odd on my computer, I was surely just using it wrong. Nowadays, I see things the other way around: We're not "holding it wrong", the computer is just defective. Computers should do our bidding, not vice versa. So here's a bunch of things that I do to my computers to mold them to my way of working.
Keyboard Layout
I switch constantly between a German and English keyboard layout, and regularly between various machines. My physical keyboards are German, and my fingers are used to the Windows-default German and (international) US keyboard layout. These are available by default on Windows and Linux, but MacOS goes its own way.
However, keyboard layouts on MacOS are saved in relatively simple text files, and can be modified
with relative ease. The process goes like this: Download Ukelele (free) to create a new keyboard
layout bundle for your base layout1. Inside that bundle, there's a *.keylayout file,
which is an XML file that defines the characters that each key-modifier combination produces. I
changed that to create a Windows-like US keyboard layout. And I replaced the keyboard icon with
something sane (not "A") by creating a 256x256 pixel PNG, opening it in Preview, holding alt while
saving to select the ICNS format. Save the keyboard bundle to ~/Library/Keyboard Layouts
and
reboot. Then I remove the unremovable default ("A") German layout by selecting another one, then
plutil -convert xml1 ~/Library/Preferences/com.apple.HIToolbox.plist
, and delete the entry from
AppleEnabledInputSources
. Now reboot. Almost easy. Almost.
One the one hand, this was quite the ordeal. On the other, I have tried to do this sort of thing on Windows and Linux before, and for the life of me could not do it. So I actually think this is great!
Keyboard Shortcuts
My main text editor is Emacs, and I am very used to its keyboard shortcuts. Of particular note are CTRL-A/E for going to the beginning/end of a line, and Alt-B/F for navigating forward/backwards by word. I have long wanted to use these shortcuts not just in Emacs and readline-enabled terminal applications, but everywhere else, too. And with MacOS, this is finally possible: Install BetterTouchTool ($22), and create keyboard shortcuts that maps, e.g. Alt-B/F to Alt-←/→. Ideally, put this in a new activation group that excludes Emacs. It may be necessary to remove the keyboard character for Alt-B/F from your keyboard layout before this works. I've spent an embarrassing number of hours trying to get this to work on Windows and Linux, and really got nowhere2. Actually, however, most readline shortcuts such as Ctrl-A/E/B/F/K/Y already work out of the box on MacOS!
Mouse Configuration
I generally use a trackpad, and occasionally a traditional mouse for image editing, and have used a trackball. I find that any one specific device will lead to wrist pain if used constantly, so I switch it up every now and then. The trackpad and trackball, however, need configuration to be usable.
After experimenting with many a trackpad device, I have found Apple touch pads the best trackpads on the market3. On MacOS, they lacks a middle mouse click. So I created a trackpad shortcut in the aforementioned BetterTouchTool ($22) to map the middle click on a three-finger tap (can also be had for free with MiddleClick (OSS)). For Windows, Magic Utilities ($17/y) provides a wonderful third-party driver for Apple devices that also supports the three-finger tap. I have not gotten the Apple touch pad to pair reliably on Linux, and have generally found their touch pad driver libinput a bit lacking.
My trackball is a Kensington SlimBlade. To scroll, you rotate the ball around its vertical axis. This is tedious for longer scroll distances, however. But there's an alternative scrolling method called "button scrolling", where you hold one button on the trackball, and move the ball to scroll. You need to install the Kensington driver to enable this on Windows and MacOS. Button scrolling is available on Linux as well using xinput4, but I haven't gotten the setting to stick across reboots or sleep cycles. So I wrote a background task that checks xinput every five seconds, which does the trick.
Window Management
Frankly, Windows does window management correctly. Win-←/→ moves windows to the left and right edge of the screen, as does dragging the window to the screen border. Further Win-←/→ then moves the window to the next half-screen in that direction, even across display boundaries. KDE does this correctly out of the box as well, Gnome does not do the latter, and it drives me mad. MacOS doesn't do any of these things. But Rectangle (OSS) does. Easy fix. (BetterTouchTool can do it, too, but Rectangle is prettier)
Furthermore, I want Alt-Tab to switch between windows. Again, MacOS is the odd one out, which uses CMD-Tab to switch between apps, now windows. And then there's another shortcut for switching between windows of the same app, but the shortcut really doesn't work at all on a German keyboard. Who came up with this nonsense? Anyway, Witch ($14) implements window switching properly.
Application Management
In Windows and Linux, I hit the Windows key and start typing to select and start an app. In MacOS, this is usually Cmd-Space, but BetterTouchTool can map it to a single short Cmd, if you prefer.
More annoying are the various docks and task bars. I always shove the dock off to the right edge of the screen, where it stays out of the way. Windows 10 had a sane dock, but then 11 came and forced it to the bottom of the screen. Dear OS makers, every modern screen has plenty of horizontal space. But vertical space is somewhat limited. So why on earth would you make a rarely used menu such as the dock consume that precious vertical space by default? And Microsoft, specifically, why not make it movable? Thankfully, there's StartAllBack ($5), which replaces the Windows task bar with something sensible, and additionally cleans up the start menu if you so desire. On KDE, I fractionally prefer Latte (OSS) over KDE's native dock. The MacOS dock is uniquely dumb, offering no start menu, and allowing no window selections. But it's unobtrusive and can be moved to the right edge, so it's not much of a bother.
File Management
One of the most crucial tasks in computer work in general is file management. I am not satisfied with most file managers. Dolphin on KDE works pretty well, it has tabs, can bulk-rename files, can display large directories without crashing, and updates in real time when new files are added to the current directory. Gnome Nautilus is so bad it is the main reason I switched to KDE on my Linux machines. Finder on MacOS is passable, I suppose, although the left sidebar is unnecessarily restrictive (why can't I add a shortcut to a network drive?). Windows Explorer is really rather terrible, lacking a bulk-rename tool, and crucially, tabs. In Windows 10, these can be added with Groupy ($12) (set it to only apply to explorer.exe). Windows 11 has very recently added native tabs, which work OK, but can't be detached from the window.
The sad thing is that there are plenty of very good file manager replacements out there, but none of the OSs have a mechanism for replacing their native file manager in a consistent way, so we're mostly stuck with the defaults.
Oh, and I always remove the iCloud/OneDrive sidebar entries, which is surprisingly tedious on Windows.
Hardware Control
On laptops, you can control screen brightness from your keyboard. On desktops, you can not. However, some clever hackers have put together BetterDisplay (OSS for screen brightness), which adds this capability to MacOS. That's actually a capability I have wanted for quite a while, and apparently it is only available in MacOS. Great stuff!
Less great is that MacOS does not allow volume control on external sound cards. SoundSource ($47) adds this rather crucial functionality back, once you go through the unnecessarily excruciating process of enabling custom kernel extensions. Windows and Linux of course natively support this.
Another necessary functionality for me is access to a non-sucky (i.e. no FAT) cross-platform file system. At the moment, the most portable file system seems to be NTFS, of all things. Regrettably, MacOS only supports reading NTFS, but no writing. Paragon NTFS (€20) adds this with another kernel extension, and promptly kernel-panicked my computer. Oh joy. At least it's only panicking for file transfers initiated by DigiKam, which I can work around. Paragon Support says they're working on it. I'm not holding my breath. Windows and Linux of course natively support NTFS.
System Management
I have learned from experience not to trust graphical backup programs. TimeMachine in particular has eaten my backups many times already, and can not be trusted. But I have used Borg (OSS) for years, and it has so far performed flawlessly. Even more impressive, my Borg backups have a continuous history despite moving operating systems several times. It truly is wonderful software!
On Windows, I run Borg inside the WSL, and schedule its backups with the Windows Task Scheduler. On Linux, I schedule them with systemd units. On MacOS, I install Borg with Homebrew (OSS) and schedule the backups with launchd tasks. It's all pretty equivalent. One nice thing about launchd, however, is how the OS immediately pops up a notification if there's a new task file added, and adds the task to the graphical system settings.
I have to emphasize what a game-changer the WSL is on Windows. Where previously, such simple automations where a pain in the neck to do reliably, they're now the same simple shell scripts as on other OSes. And it perfectly integrates with Windows programs as well, including passing pipes between Linux and Windows programs. It's truly amazing! At the moment, I'd rate Windows a better Unix system than MacOS for this reason. Homebrew is a passable package manager on MacOS, but the way it's ill-integrated into the main system (not in system PATH) is a bit off-putting.
App Compatibility
I generally use my computer for three tasks: General document stuff, photo editing, and video games.
One major downside of Apple computers is that video games aren't available. This has become less of a problem to me since I bought a Steam Deck, which has taken over gaming duties from my main PC. Absolutely astonishingly, the Steam Deck runs Windows games on Linux through emulation, which works almost flawlessly, making video games no longer a Windows-only proposition.
What doesn't work well on Linux are commercial applications. Wine generally does not play well with them, and frustratingly for my photo editing, neither VMWare Workstation Player (free) nor VirtualBox (OSS) support hardware-accelerated VMs on up-to-date Linux5. So where MacOS lacks games, Linux lacks Photoshop. Desktop applications in general tend to be unnecessarily cumbersome to manage and update on Linux. Flatpak is helping in this regard, by installing user-facing applications outside of the OS package managers, but it remains more work than on Windows or MacOS. The occasional scanner driver or camera interface app can also be troublesome on Linux, but that's easily handled with a VirtualBox VM (with the proprietary Extension Pack for USB2 support), and hasn't really bothered me too much.
Luckily for me, my most-used apps are generally OSS tools such as Darktable (OSS) and DigiKam (OSS), or cross-platform programs like Fish (OSS), Git (OSS), and Emacs (OSS). This is however, where Windows has a bit of a sore spot, as these programs tend to perform noticeably worse on Windows than on other platforms. Emacs and git in particular are just terribly slow on Windows, taking several seconds for routine operations that are instant on other platforms. That's probably due to Windows' rather slow file system and malware scanner for the many-small-files style of file management that these Unix tools implement. It is very annoying.
Conclusions
So there's just no perfect solution. MacOS can't do games, Linux can't run commercial applications, and Windows is annoyingly slow for OSS applications. Regardless, I regularly use all three systems productively. My job is mostly done on Windows, my home computer runs MacOS, and my Steam Deck and automations run Linux.
Overall, I currently prefer MacOS as my desktop OS. It is surprisingly flexible, and more scriptable than I thought, and in some ways is actually more functional than Linux or Windows. The integrated calendar and contacts apps are nice, too, and not nearly as terrible as their Windows/Linux counterparts. To say nothing of the amazing M1 hardware with its minuscule power draw and total silence, while maintaining astonishing performance.
Linux is where I prefer to program, due to its sane command line and tremendously good compiler/debugger/library infrastructure. As a desktop OS, it does have some rough edges, however, especially for its lack of access to commercial applications. While Linux should be the most customizable of these three, I find things tend to break too easily, and customizations are often scattered widely between many different subsystems, making them very hard to get right.
Theoretically, Windows is the most capable OS, supporting apps, OSS, and games. But it also feels the most user-hostile of these three, and the least performant. And then there's the intrusive ads everywhere, its spying on my every move, and at work there's inevitably a heavy-handed administrator security setup that gets in the way of productivity. It's honestly fine on my home computer, at least since they introduced the WSL. But using it for work every day is quite enough, so I don't want to use it at home, too.
Footnotes:
if there's an easier way to create a keyboard layout bundle, please let me know. I didn't use ukelele for anything but the bundle creation.
It occurs to me that it might actually be possible to do something like on Windows this with AutoHotkey (free). I'll have to try that!
The Logitech T650 is actually not bad, but the drivers are a travesty. Some Wacom tablets include touch, too, but they palm rejection is abysmal and they don't support gestures.
xinput set-prop "Kensington Slimblade Trackball" "libinput Scroll Method Enabled" 0, 0, 1 # button scrolling
xinput set-prop "Kensington Slimblade Trackball" "libinput Button Scrolling Button" 8 # top-right button
VirtualBox does not have a good accelerated driver, and VMWare does not support recent kernels. Qemu should be able to solve this problem, but I couldn't get it to work reliably.
I Miss My Mac
There was a time, maybe from 2009 to 2011, when Apple computers were glorious. They were elegant devices, beautiful to me, and powerful. Never before, or since, have I loved a machine as I loved that MacBook Pro early 2011.
I miss that, being able to love the elegance and power a machine.
Nowadays I mostly use KDE Linux, which I can tolerate. Things occasionally break, stuff sometimes doesn't work as intended, but I can generally work around these quirks, so they don't bother me too much. Today, off the top of my head, Firefox crashed, Thunderbird couldn't display one folder, Zotero had to be re-installed, one monitor had to be power-cycled when waking the computer, and the laptop battery died way too quickly again.
I also use a Windows Surface tablet, which is a wonderful little device. It currently shows an error message every time it boots that I can't get rid of, has to be rebooted every few days to stay fast, and sometimes just won't react until restarted. And it always wants to install some update, and just refuses to do it overnight as intended. But generally it works pretty well.
And yesterday, in a bout of nostalgia, I opened my macOS laptop again and endeavored to write a bit on it. There were no updates, then there were updates but they couldn't be installed, then they could be installed, then the download stalled, and hat to be un-paused for some reason. Then my project files randomly couldn't be downloaded to the laptop, and the window management in macOS is a horrid mess. Also the update took a full hour, and didn't change anything noteworthy. I didn't end up actually getting to write on it.
In the evening, my camera corrupted its file system, as it sometimes does if I forget to format the SD card as FAT instead of its default exFAT. The recovery process was tedious but worked. I've done it a few times now and I know what to do.
My phone, meanwhile, can't use its fingerprint reader any longer ever since I "up"graded to Android 10. It's a Pixel 2, as google as Google can be. And apps are sometimes unresponsive and have to be restarted to become usable again. And it still can't undo text edits in the year 2020, and probably spies on my every move.
My amplifier nominally supports Bluetooth, but it only works some of the time. Currently it doesn't, so we use a portable Bluetooth speaker instead. Its Spotify integration has never worked even once. But at least it now reliably sends an image to the projector ever since we vowed to only ever touch the one reliable HDMI port on the front of it.
And as much as I love my ebook reader, it sometimes crashes when there's too much stuff on the page. One time I even had to completely reset it to get it to work again. And every time I install an update, the visual layout changes. Most of the time, mostly for the better. But it just erodes trust so much, to be at the mercy of the whims of an obviously demented software designer somewhere.
As I said, I miss my Mac, back when technology was magical, and just worked. And any error could be clearly explained as user error instead of terrible programming. When I was young and foolish, in other words.
I should go play with my child now, and run around in the yard, or maybe read a paper book. There's no joy in technology any more.
How to set up rsnapshot instead of Time Machine
(This blog post was changed since my initial strategy of disabling the lockfile didn't work. Turns out, the lockfile is required, and backups have to be stacked.)
Yesterday, I wrote about how Time Machine has failed me. Time Machine keeps regular backups, going back as far as your hard drive space permits. In theory. In practice, every year or so it messes up somehow and has to start over, thereby deleting all your older backups. A backup that is not reliable is not a backup.
Luckily, there are alternatives. Probably the easiest is rsync[1], a very cool tool that copies files and directories from one place to another. You could simply run this once a day, and have a new backup every day. You can even configure rsync so it doesn't need to copy unchanged files, and instead hard-links them from an older backup. rsnapshot automates this process to keep a number of tiered copies, for example ten hourly backups, seven daily backups, four weekly backups, and a hundred monthly backups. Each backup is then simply a directory that contains your files. No fancy starfield-GUI, but utterly reliable and trivial to understand [2].
Setting up rsnapshot on macOS is not quite as straight-forward as I'd like, and I couldn't find a great guide online. So, without further ado, here's how to configure rsnapshot on macOS:
Install rsnapshot
brew install rsnapshot
Write the config file
You can copy a template from homebrew:
cp /usr/local/Cellar/rsnapshot/1.4.2/etc/rsnapshot.conf.default /usr/local/etc/rsnapshot.conf
And then configure the new configuration file to your liking (preserve the tabs!):
config_version 1.2 # default verbose 2 # default loglevel 3 # default # this is where your backups are stored: snapshot_root /Volumes/BBackup/Backups.rsnapshot/ # make sure this is writeable # prevent accidental backup corruption: lockfile /Users/bb/.rsnapshot.pid # use this if you back up to an external drive: no_create_root 1 # don't back up if the external drive is not connected # configure how many tiers of backups are created: retain hourly 10 retain daily 7 # dailies will only be created once 10 hourlies exist retain weekly 4 # weeklies will only be created once 7 dailies exist retain monthly 100 # monthlies will only be created once 4 weeklies exist # the list of directories you want to back up: backup /Users/bb/Documents localhost/ backup /Users/bb/eBooks localhost/ backup /Users/bb/Movies localhost/ backup /Users/bb/Music localhost/ backup /Users/bb/Pictures localhost/ backup /Users/bb/Projects localhost/ backup /Users/bb/Projects-Archive localhost/
Instead of
localhost
, you can use remote machines as well. Checkman rsync
for detailsMake sure it works and create initial backup
rsnapshot -c /usr/local/etc/rsnapshot.conf hourly
The first backup will take a while, but subsequent backups will be fast. A normal backup on my machine takes about two minutes and runs unnoticeably in the background.
Write launchd Agent
Next, we have to tell macOS to run the backups in regular intervals. Conceptually, you do this by writing a launchd agent script[3], which tells launchd when and how to run your backups. In my case, I create four files in
/Users/bb/Library/LaunchAgents/
, calledrsnapshot.{hourly,daily,weekly,monthly}.plist
. Apple's documentation for these files is only mildly useful (as usual), butman launchd.plist
andman plist
should give you an idea how this works.Here is my hourly launchd agent (I'll explain the bash/sleep thing later):
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>Label</key> <string>rsnapshot.hourly</string> <key>ProgramArguments</key> <array> <string>/bin/bash</string> <string>-c</string> <string>sleep 0 && /usr/local/bin/rsnapshot -c /usr/local/etc/rsnapshot.conf hourly</string> </array> <key>StartCalendarInterval</key> <dict> <key>Minute</key> <integer>0</integer> </dict> </dict> </plist>
For the other four scripts, change the two occurrences of
hourly
to{daily,weekly,monthly}
and change the<dict>
portion at the end todaily:
<key>Minute</key> <integer>0</integer> <key>Hour</key> <integer>0</integer>
weekly:
<key>Minute</key> <integer>0</integer> <key>Hour</key> <integer>0</integer> <key>Weekday</key> <integer>1</integer>
monthly:
<key>Minute</key> <integer>0</integer> <key>Hour</key> <integer>0</integer> <key>Day</key> <integer>1</integer>
However,
rsnapshot
can only ever run one backup at a time without stepping on its own toes. This is a problem when the computer wakes up, and more than one backup was scheduled during its sleep, since launchd will then happily launch all missed backups at the same time. But only one of them will succeed.To fix this, I delay the later backup tiers using the
sleep 0
directive. I usesleep 900
(15 minutes later) for daily,sleep 1800
(30 minutes), andsleep 2700
(45 minutes) for the lower tiers[4]. It seems that there should be a more elegant solution than this, but I haven't found one.From the documentation, you might think that
<key>Program</key>
would be more succinct than supplying the binary as the first argument of<key>ProgramArguments</key>
, but this apparently uses a different syntax and does not in fact work as expected.Load launchd agents
launchctl load ~/Library/LaunchAgents/rsnapshot.*
Test launchd agent
launchctl start rsnapshot.hourly
If it doesn't work, Console.app might show a relevant error message.
Remove backup directory from Spotlight
Go to System Preferences → Spotlight → Privacy → Add your
snapshot_root
directory from earlierDisable TimeMachine and delete your existing backup (if you want)
Start Time Machine, right-click any directory you want to delete, and select "delete all backups of $dir"
[1] rsync is one of those reliable tools I talked about. It is rock solid, incredibly versatile, and unapologetically single-minded. A true gem!
[2] This works great for local backups. If you need encrypted backups or compressed backups (maybe on an untrusted remote machine), this post recommends Borg instead of rsnapshot, but you will lose the simplicity of simple directories.
[3] I use launchd instead of cron since launchd will re-schedule missed backups if the computer was asleep.
[4] This will fail if the hourly backup takes longer than 15 minutes. This is rather unlikely, though, or at least should not happen often enough to be of concern.
Caveats
The configuration file of rsnapshot says that you might experience data corruption if you run several copies of rsnapshot at the same time (and you can use the lockfile to prevent this). This is a problem if your computer is asleep while rsnapshot is scheduled to run, since launchd will then re-schedule all missed tasks at once when the computer wakes up. If you enable the lockfile, only one of them will run.
On the other hand, only the hourly task will actually create a new backup. All higher-level backup tiers merely copy existing backups around, so in theory, they shouldn't step on each other's toes when run concurrently. I have opened an issue asking about this.
There are other possible solutions: ① You could modify the launchd entry such that backups only trigger after a few minutes or, better yet, only once all other instances of rsnapshot have finished. I am not sure if launchd supports this, though. ② You could schedule the hourly task using cron instead of launchd, since cron will not reschedule missed tasks. This would only work for two tiers of backups, though. ③ You could just ignore the issue and hope for the best. After all, if a daily or hourly backup gets corrupted every now and then, you still have enough working backups…
Audio APIs, Part 1: Core Audio / macOS
This is part one of a three-part series on the native audio APIs for Windows, Linux, and macOS. This first part is about Core Audio on macOS.
It has long been a major frustration for my work that Python does not have a great package for playing and recording audio. My first step to improve this situation were a small contribution to PyAudio, a CPython extension that exposes the C library PortAudio to Python. However, I soon realized that PyAudio mirrors PortAudio a bit too closely for comfort. Thus, I set out to write PySoundCard, which is a higher-level wrapper for PortAudio that tries to be more pythonic and uses NumPy arrays instead of untyped bytes
buffers for audio data. However, I then realized that PortAudio itself had some inherent problems that a wrapper would not be able to solve, and a truly great solution would need to do it the hard way:
Instead of relying on PortAudio, I would have to use the native audio APIs of the three major platforms directly, and implement a simple, cross-platform, high-level, NumPy-aware Python API myself. This effort resulted in PythonAudio, a new pure-Python package that uses CFFI to talk to PulseAudio on Linux, Core Audio on macOS, and WASAPI[1] on Windows.
This series of blog posts summarizes my experiences with these three APIs and outlines the basic structure of how to use them. For reference, the singular use case in PythonAudio is playing/recording of short blocks of float
data at arbitrary sampling rates and block sizes. All connected sound cards should be listable and selectable, with correct detection of the system default sound card (a feature that is very unreliable in PortAudio).
[1]: WASAPI is part of the Windows Core Audio APIs. To avoid confusion with the macOS API of the same name, I will always to refer to it as WASAPI.
CoreAudio, or the Mac's best kept secret
CoreAudio is the native audio library for macOS. It is known for its high performance, low latency, and horrible documentation. After having used the native audio APIs on all three platforms, CoreAudio was by far the hardest one to use. The main problem is lack of documentation and lack of feedback, and plain missing or broken features. Let's get started.
The basic unit of any CoreAudio program is the audio unit. An audio unit can be a source (aka microphone), a sink (aka speaker) or an audio processor (both sink and source). Each audio unit can have several input buses, and several output buses, each of which can have several channels. The meaning of these buses varies wildly and is often underdocumented. Furthermore, every audio unit has several properties, such as a sample rate, block sizes, and a data format, and parameters, which are like properties, but presumably different in some undocumented way.
In order to use an audio unit, you create an AudioComponentDescription
that describes whether you want a source or sink unit, or an effect unit, and what kind of effect you want (AudioComponent is an alternative name for audio unit). With the description, you can create an AudioComponentInstance
, which is then an opaque struct pointer to your newly created audio unit. So far so good.
The next step is then to configure the audio unit using AudioUnitGetProperty
and AudioUnitSetProperty
. This is surprisingly hard, since every property can be configured for every bus (sometimes called element) of every input or output of every unit, and the documentation is extremely terse on which of these combinations are valid. Some invalid combinations return error codes, while others only lead to errors during playback/recording. Furthermore, the definition of what constitutes an input or output is interpreted quite differently in different places: One place calls a microphone an input, since it records audio; another place will call it an output, since it outputs audio data to the system. In one crazy example, you have to configure a microphone unit by disabling its output bus 0, and enabling its input bus 1, but then read audio data from its ostensibly disabled output bus 0.
The property interface is untyped, meaning that every property has to be given an identifier, a void pointer that points to a matching data structure, and the size of that data structure. Sometimes the setter allocates additional memory, in which case the documentation does not contain any information on who should free this memory. Most objects are passed around as opaque struct pointers with dedicated constructor and destructor functions. All of this does not strike me as particularly C-like, even though CoreAudio is supposedly a native C library.
Once your audio unit is configured, you set a render callback function, and start the audio unit. All important interaction now happens within that callback function. In a strange reversal of typical control flow, input data to the callback function needs to be fetched by calling AudioUnitRender
(evoked on the unit itself) from within the callback, while output is written to memory provided as callback function arguments. Many times during development, AudioUnitRender
would return error codes because of an invalid property setting during initialization. Of course, it won't tell which property is actually at fault, just that it can't fulfill the render request at the moment.
Error codes in general are a difficult topic in CoreAudio. Most functions return an error code as an OSStatus
value (aka uint32
), and the header files usually contain a definition of some, but not all, possible error codes. Sometimes these error codes are descriptive and nice, but often they are way too general. My favorite is the frequent kAudioUnitErr_CannotDoInCurrentContext
, which is just about as useless an error description as possible. Worse, some error codes are not defined as numeric constants, but as int err = 'abcd'
, which makes them un-searchable in the source file. Luckily, this madness can be averted using https://osstatus.com/, which is a dedicated database for OSStatus
error codes.
By far the worst part of the CoreAudio API is that some properties are silently ignored. For example, you can set the sample rate or priming information on a microphone unit, and it will accept that property change and it will report that property as changed, but it will still use its default value when recording (aka "rendering" in CoreAudio). A speaker unit, in contrast, will honor the sample rate property, and resample as necessary. If you still need to resample your microphone recordings, you have to use a separate AudioConverter
unit, which is its own bag of fun (and only documented in a remark in one overview document).
Lastly, all the online documentation is written for Swift and Objective-C, while the implementation is C. Worse, the C headers contain vastly more information than the online documentation, and the online documentation often does not even reference the C header file name. Of course header files are spread into the CoreAudio framework, the AudioToolkit framework, and the AudioUnit framework, which makes even grepping a joy.
All of that said, once you know what to do and how to do it, the resulting code is relatively compact and readable. The API does contain inconsistencies and questionable design choices, but the real problem is the documentation. I spent way too much time reading the header files over and over again, and searching through (often outdated or misleading) example projects and vague high-level overviews for clues on how to interpret error messages and API documentation. I had somewhat better luck with a few blog posts on the subject, but the general consensus seems to be that the main concepts of CoreAudio are woefully under-explained, and documentation about edge cases is almost nonexistent. Needless to say, I did not enjoy my experience with CoreAudio.
OS X Finder Woes
The Mac. It used to be the most streamlined, thought-through general computing device on the market.
Even it's file management used to be top-notch. There were many cool little touches. One particularly useful feature was the Proxy Icon–if a window displayed a file's content, that file's icon would show up in the window's title. And you could drag that icon directly onto a thumb drive or email, without having to use the Finder. But the Finder, too, had many neat little features. I loved the fact that when you renamed a file in an alphabetically sorted file list, Finder would not immediately re-shuffle it to its new location, but would wait half a second before doing so. When renaming multiple files, this was really useful, since you could go through them one by one and rename them, simply by pressing arrow keys and return.
But as you might have guessed from my use of the past tense, these golden days are gone. The Finder used to know a JPEG from a ZIP regardless of file extension. Now it doesn't any more. The Proxy Icon is still draggable, but it will create an alias instead of a copy–perfectly useless on a thumb drive or in an email.
And with the newest version of OS X, El Capitan, they finally blew it for me. Before, even though the Finder inexplicably never had the ability to cut and paste files, you could always install programs like TotalFinder to fix that. Not so with El Capitan. The Finder now is holy land, and can not be touched any more by third parties. So no more cut and paste, no more un-hiding system files. No more side-by-side Finder tabs. And brand new with El Capitan as well: No more waiting after renaming. Now, when you rename a file, it is immediately re-sorted to its new position, thus making renaming multiple files terribly inconvenient.
So, good bye OS X. I updated my work laptop first, and I regret it. I never regretted an OS X update before. My home machine is not going to get the update. It is honestly sad to see my once-beloved Mac platform becoming worse and worse and worse with every new release.
Changing File Creation Dates in OSX
On my last vacation, I have taken a bunch of pictures, and a bunch of video. The problem is, I hadn't used the video camera in a long time, and it believed that all it's videos were taken on the first of January 2012. So in order for the pictures to show up correctly in my picture library, I wanted to correct that.
For images, this is relatively easy: Most picture libraries support some kind of bulk date changes, and there are a bunch of command line utilities that can do it, too. But none of these tools work for video (exiftool claims be able to do that, but I couldn't get it to work).
So instead, I went about to change the file creation date of the actual video files. And it turns out, this is surprisingly hard! The thing is, most Unix systems (a Mac is technically a Unix system) don't even know the concept of a file creation date. Thus, most Unix utilities, including most programming languages, don't know how to deal with that, either.
If you have XCode installed, this will come with SetFile
, a command line utility that can change file creation dates. Note that SetFile
can change either the file creation date, or the file modification date, but not both at the same time, as any normal Unix utility would. Also note that SetFile
expects dates in American notation, which is about as nonsensical as date formats come.
Anyway, here's a small Python script that changes the file creation date (but not the time) of a bunch of video files:
import os.path import os import datetime # I want to change the dates on the files GOPR0246.MP4-GOPR0264.MP4 for index in range(426, 465): filename = 'GOPR0{}.MP4'.format(index) # extract old date: date = datetime.datetime.fromtimestamp(os.path.getctime(filename)) # create a new date with the same time, but on 2015-08-22 new_date = datetime.datetime(2015, 8, 22, date.hour, date.minute, date.second) # set the file creation date with the "-d" switch, which presumably stands for "dodification" os.system('SetFile -d "{}" {}'.format(new_date.strftime('%m/%d/%Y %H:%M:%S'), filename)) # set the file modification date with the "-m" switch os.system('SetFile -m "{}" {}'.format(new_date.strftime('%m/%d/%Y %H:%M:%S'), filename))
Kindle.app not starting on a case-sensitive file system
So Kindle.app was updated through the App Store and did not start any more. It just crashed and the crash reporter came up.
A quick look at Console.app turns up a link to the actual crash report. And the crash report starts with
Dyld Error Message: Library not loaded: @executable_path/../Frameworks/libWEbCoreKRF.dylib Referenced from: /Applications/Kindle.app/Contents/Frameworks/libWebCoreViewer.dylib Reason: image not found
libWEbCoreKRF.dylib
? With a capitalized E
? That looks very much like a spelling error. So, point your terminal to /Applications/Kindle.app/Contents/Frameworks/
, type
sudo ln -s libWebCoreKRF.dylib libWEbCoreKRF.dylib
to create a symlink with the misspelled name. Done.
Kindle works again.
Spelling is hard, it seems.
Using a Raspberry Pi as a Time Capsule for Mountain Lion
A while ago, I bought a Time Capsule to take care of my backups. I can't say it has been smooth sailing. Every now and then, the Time Capsule would claim that the backup did fail. Sometimes a reboot would help, sometimes not. Sometimes hdiutil
would be able to salvage the backups, sometimes not. Sometimes, the backup disk image would simply be corrupted and the only option would be to delete it and start over.
This might be bad luck or it might be due to a defective Time Capsule or it might be due to my computer. I have no idea. But the thing is, if I have to hack on my backup system anyway, lets do it in style, at least. So here goes:
Ingredients: A Raspberry Pi, an external hard drive, some patience
Format an SD card as described in the wiki. I just installed the version of Debian that is provided on the official website. Now just boot up.
Next, I was stumped because I only have an Apple LED display and no convenient way of connecting the Raspberry Pi's HDMI output to the LED display's Mini Display Port. After some searching and a combination of three adapter cables, I finally got it connected and could see it boot. Really, I have no use whatsoever for the HDMI port on the Raspberry pi. So the first thing I did was to enable SSH, which luckily is available right there in the configuration utility that starts when you boot the thing for the first time.
After that, I disconnected the display and immediately was stumped because I now had no way of finding the Pi's IP address. Actually, I did not even have a network to connect it to. So I strung an ethernet cable from my laptop to the Pi and enabled Internet Sharing in order to (1) start the DHCP server and (2) give the Pi internet access. The IP address was then easily found using arp -a
.
Setting up the hard drive
First off, I needed to format and mount my external hard drive to be usable as a Time Machine volume. ls /dev
showed the hard drive as /dev/sda
. Thus, I installed parted
using sudo apt-get install parted
and used it sudo parted
. In parted, select /dev/sda
sets it up to modify the external hard drive, rm 1
deleted its main partition, q
to quit parted. Next, creating a new partition: sudo fdisk
, then in there n
with p
and 1
to create a new primary partition, then w
to apply the changes and exit. Lastly, I created the file system with sudo mkfs -t ext4 /dev/sda1
with the whole partition as its size. Now lastly, I created a mount point for it using mkdir ~/TimeMachine
(don't use sudo
!) and auto-mounted it by appending this to /etc/fstab
/dev/sda1 /home/pi/TimeMachine ext4 rw,auto,user,exec,sync 0 0
Note: sync
specifies that all file system changes have to be written to disk immediately, without caching. This might be bad for performance, but on the other hand, this behavior is probably a good idea for a backup system. I once read something somewhere that Apple is enforcing a similar behavior on their Time Capsules and that this is the reason why they won't allow any other network drive as Time Capsules.
Setting up the shared folder
First up, this requires netatalk
, so I did sudo apt-get update
and sudo apt-get install netatalk
to install it. Next, netatalk has to be configured to actually share the drive on the network. This is accomplished by appending this line to /etc/netatalk/AppleVolumes.default:
/home/pi/TimeMachine TimeMachine allow:pi cnidscheme:dbd options:upriv,usedot,tm
Also, the afp daemon should be configured to use the proper authentification schemes. Thus, add this to /etc/netatalk/afpd.conf:
- -transall -uamlist uams_randnum.so,uams_dhx.so,uams_dhx2.so -nosavepassword -advertise_ssh
(maybe append
mdns
to the hosts in /etc/nsswitch.conf? Probably not necessary.)
I am also not quite sure whether I actually had to create a new file /etc/avahi/services/afpd.service and write into it:
<?xml version="1.0" standalone='no'?><!--*-nxml-*--> <!DOCTYPE service-group SYSTEM "avahi-service.dtd"> <service-group> <name replace-wildcards="yes">%h</name> <service> <type>_afpovertcp._tcp</type> <port>548</port> </service> <service> <type>_device-info._tcp</type> <port>0</port> <txt-record>model=Xserve</txt-record> </service> </service-group>
And maybe, you have to create an empty file that signifies the drive as Time Machine compatible using
touch ~/TimeMachine/.com.apple.timemachine.supported
.
Edit: Turns out, all these were not necessary. Thank you, reader Philipp, for trying them out!
I certainly did all that, but I am not quite sure which of these steps were strictly necessary. If you know, please let me, too.
Anyway, with all that done, restart both the netatalk and the Bonjour daemon using sudo /etc/init.d/netatalk restart
and sudo /etc/init.d/avahi-daemon restart
.
Setting up the Time Machine
Now, back to the Mac. In order to make Time Machine accept the new network share, run
defaults write com.apple.systempreferences TMShowUnsupportedNetworkVolumes 1
Edit: Turns out, this setting is not necessary. OSX just picks the Raspberry Pi as a usable Time Machine drive by default.
Finally, the TimeMachine folder on the Raspberry Pi was available as one of the backup drives. Halleluja!
Now transfer speeds for the initial backup are not exactly what I would call fast, but this might not be the Pi's fault. For one thing, the Pi is reporting to only run at half load. For another thing, the external hard drive and its USB connection is probably not very speedy. And lastly, I seem to remember that initial backups always were slow. But really, only time will tell how well this thing can do the job of a Time Capsule.
Further testing shows that transfer speeds are very comparable to the Time Capsule. Thus, I declare this a raging success!
This article heavily steals from these fine folks on the internet:
How Apple is Failing Me
I bought my first Apple computer in 2007 after a long time of gaming on Windows and a few years of Linux. In the beginning, I was just amazed at the consistency, practicality and sanity of the whole experience. I think I had a bad case of tinkeritis and neophilia in my Linux days, so this was probably to be expected.
Later, I marveled at the level of polish in the third party software ecosystem around Macs. Then I was just glad to have a slick GUI with actual Unix underpinnings. For reference, this happened in the time span from OS X 10.5 Leopard until 10.7 Lion. However, in that same time frame, I also noticed a gradual increase in problems. The computer seems to crash slightly more often, programs seemed to become more buggy, behavior less logical. At first, I marked this up as mostly a matter of my own increasing knowledge of the system, but as time went on, I became more and more convinced that it was actually the system getting worse, not myself becoming more sensitive.
So here are some of those problems:
Lack of Uninstallers
Computer people like to try new software. Some new software is crap. You should delete crap from your computer. Amazingly however, the Mac does not have any way of uninstalling some software.
At the moment, there are three ways of getting software on your Mac.
- Install from the App Store. Installation and un-installation work through the App Store.
- Install by dragging n
*.app
bundle onto your computer. Uninstall by deleting the bundle. Works fine unless the app installed some stuff outside the bundle. - Install by executing a
*.pkg
file. No way to uninstall whatsoever unless the developer provides a separate uninstall mechanism.
Really, this is beyond ridiculous. There is no way of uninstalling software that has been installed from a *.pkg
. How the heck am I supposed to use my computer without uninstalling software? Seriously, Apple? Is this a joke?
(So as a rule, I will only install *.pkg
if there is absolutely no other way. Most of the time, I use or abuse homebrew to mitigate this problem.)
Failing Time Machine
So the Mac comes with this awesome backup solution called Time Machine. Just plug in your external hard drive, and it will magically keep all your data save, at all times. Better yet, buy a Time Capsule and this will happen over the wireless network so you don't even need to plug in that hard drive any more. Magic!
Except, every few weeks the Time Machine icon in the menu bar will have a small exclamation point, which indicates that something went wrong. Usually, it means that Time Machine somehow lost track of what it was doing and silently corrupted all backups beyond repair. This happens across several computers and Time Machines. The only solution is to delete all backups and start anew.
So, backups that are not backing up. How amazing is that?
Living in the Cloud
OS X is this astonishingly modern operating system, where all your email, contacts, calendars and settings are synced in the cloud, accessible from anywhere and any device or service you use. In theory.
Synchronization is cool. I used to live within the Googleverse, and all my contacts, calendars and email accounts would sync through Google to all my Apple devices. Life was good! Until things started to go bad. At some point, Mail.app lost an email or two. Not a big deal. Then iCal would not connect any more, and would need to be restarted every other day. And finally, Address Book deleted all my contacts from Google. Actually, it did not delete the whole contacts, just all the information inside them, and leave empty husks with only a name and maybe a partial email address.
I wish I was making this up. I tested this quite thoroughly. Whenever Contacts was set up to sync with Google, after a few days or weeks it would delete all my contacts. Of course, this being cloud backed, it would delete them everywhere on all my devices, without any backup or anything. Thankfully, Address Book on iOS seems to not have this problem.
Things can go wrong, I understand that. But deleting all my contacts? Not cool. Not. Cool.
iPhoto
My grandfather has this G4 PowerBook. He stores all his photos on it. So at one point, we just could not stand it any more to support his software problems and set him up with a newer MacBook (2007) just so we could at least roughly recreate his problems on our machines.
But to do that, we had to convert his old library from iPhoto 1876 to iPhoto 2008. iPhoto itself claims to be able to do that, and lo and behold, it magically imported the old library, started an update program, then another, then another, then another, then another, and updated the old library through the ages up to the most recent version. And everything seemed to work fine, all the albums were there, all the thumbnails looked right, just perfect. Except if you opened any image, it would only show black. I mean, I can understand that an import of a library that old does not work. But at least tell me so. Don't silently corrupt everything!
And then there was my own library, started in late 2007 and handed through different revisions of iPhoto to today. It countains about 25 Gb worth of photos. Funny enough, its size on disk is actually 65 Gb. What. The. Heck?
At this point, a giant thank you to the iPhoto Library Manager by Fat Cat Software, which saved both my library and my sanity! Those guys rock!
OS X Lion with Auto Save and Versions
The newest and brightest in the land of Apple is OS X 10.7 Lion (at the time of writing). This amazing piece of software has a life-changing new feature: It friggin breaks one of the oldest tradition in computer history: Saving files.
Now normally, I would applaud this. Just save everything automatically and give me a sensible undo mechanism instead. Great!
However, in a stroke of genius, they also got rid of the possibility of save as…, which is just an easy way of duplicating a document. Say, because you received it in an email and want to file it somewhere else on your hard drive. Or because the document is saved on a thumb drive and you want to copy it over to your hard drive. No sir, you have to navigate to the document in the finder and copy it by hand from there. Thank you for your patience.
But the worst offender is Preview. Every program on the Mac has this very handy proxy icon in the window title. Whenever a window is representing a file on your computer, it will show that very file in the title, so you can drag and drop it somewhere else. Except preview won't give you the actual document. Instead, it will give you an alias to that document. Which is worth nothing at all if you drag it into an email, your Dropbox or a thumb drive.
Or try to enable File sharing on your local network, which works sometimes, but not all the time. Or Internet Sharing, which might share your internet connection, or break your DNS settings. Or take Bluetooth, which for the life of me, I can not make work for tethering with my iPhone, even though the same works just fine with the iPad and other Macs. Or take networking in general, which every once in a while just completely breaks down on my company's network and just refuses to send or receive any data over the Ethernet cable until I reboot my computer.
I could go on. Let's just say that Lion increased my WTF-per-minute rate significantly.
Performance, Memory and File System
Lets talk performance. I can't really complain here. In my MacBook Pro I have four cores, eight Gb of memory and a fast, Apple-sanctioned SSD. Life is good. Except that I happen to use virtual machines a whole lot and regardless of whether I use Linux (Ubuntu, Fedora, OpenSuse) or Windows (7, 8) on there, they feel faster.
I mean, you don't feel the pain of starting applications much on OS X, because in general, you simply keep your programs running all the time. But even so, Firefox starts up on Windows in, like, no time at all, but takes noticeable time on OS X (same configuration and plugins). I have an SSD, so I mostly don't care, but still.
Or take boot times. Even in a virtual machine, both Windows 7 and 8 and Ubuntu 12.04 and Fedora 17 boot up in less than half a minute. In comparison, OS X Lion takes significantly longer, sometimes minutes. Again, you usually don't feel that pain too much because you don't reboot your Mac all that often (unless you run Bootcamp for gaming regularly). Still, OS X boot and shutdown times are downright shameful these days.
Lets run a few virtual machines. I have eight Gb of memory. Lets boot up a Windows VM and give it two Gb. Do anything you want, run Eclipse and Visual Studio and Firefox with loads of tabs and Steam. Windows will do just fine. Those two Gb are plenty for anything except gaming and photo/video editing. Do the same thing with Linux, same result. In fact, Linux will work just fine with even less memory. But OS X? Don't even think of running it with less than two Gb. Four Gb will give you a workable system if you don't run too many memory intensive applications. But start up XCode and Eclipse and you are done for. This is a huge waste.
And lastly, let's talk HFS+. Did you know that HFS+ has a global lock that prevents any program from accessing the hard drive while another program is doing so? That explains why OS X is so friggin slow on non-SSD drives. And did you ever notice that HFS+ is slowly corrupting your data, even if there are no power failures or anything? I fire up Disk Utility every so often and it will always find some file system corruptions. It rarely breaks stuff, but really, the file system is the very last part of my system I want to see failing randomly.
So, what now?
With all this love, why am I still using a Mac? Well, for one thing, the hardware is seriously great. I would love to see some laptop manufacturer build something equally sturdy, light and good looking as a MacBook Pro. I would love to see someone build a great touch pad into a non-Apple device. I would love to see Retina displays everywhere1. But I don't.
And then there is software. Where is Tower for Windows or Linux? Or Sparrow or Tweetbot or Reeder or Pixelmator or iPhoto? I'm not saying there are no alternatives, but I certainly have not been able to find any that were really up to the same level of polish.
That said, I have recently been playing around with Windows 8, Ubuntu 12.04 and Fedora 17 and I must say, I am quite taken with them. All of them. As for the next computer I am going to buy, I am doubtful if it will be an Apple computer again.
Footnotes:
As a matter of fact, I would love to see well enough to see Retina at all, but that is beside the point
How to gem install rlua on OSX/homebrew
If you want to use rlua, you need to have Lua installed. However, OSX does not come with Lua preinstalled, so you install it using
#+begin src sh brew install lua #endsrc
Oh, would it be nice if everyone agreed on how to install stuff like this. Case in point, homebrew installs liblua (quite reasonable) in /usr/local/lib/liblua.[5.1[.4]].dylib
rlua however expects it to be called liblua5.1.dylib
(notice the missing .
).
Similarly, the headers are installed plainly into /usr/local/include
, whereas rlua expects them to be in a folder called lua5.1
.
Hence, here is how you get rlua to install:
ln -s /usr/local/Cellar/lua/5.1.4/lib/liblua.5.1.4.dylib /usr/local/lib/liblua5.1.dylib ln -s /usr/local/Cellar/lua/5.1.4/include/ /usr/local/include/lua5.1
Not exactly a beautiful solution, but it works.
Adobe AIR Application Crashes
Adobe, oh Adobe. Why?
Your updater pops open. Adobe AIR 2.5, if I remember correctly. It runs, and after that, all AIR applications stop working. They boot, then crash and barf up a crash report.
Turns out, this is a problem with case sensitive file systems. If your file system is not case sensitive, you are experiencing some other problem and this fix is not for you.
See, Adobe has this library called WebKit.dylib
. However, their code actually looks for Webkit.dylib
(note the capitalisation of the K). Well, spelling is hard, I presume.
At least this is an error that can easily be fixed. Fire up your terminal, type
cd /Library/Frameworks/Adobe\ AIR.framework/Version/Current/Resources/ sudo ln -s WebKit.dylib Webkit.dylib
Then enter your password and your AIR applications should work again.
Dear Adobe, is it too much to ask to pretty please make your software work on a case sensitive file system? Spelling is primary school stuff. It is really not that hard!
Compiling Scipy and Matplotlib using pip on Lion
So I upgraded to Lion. Predictably, some things went wrong. This time, the main thing that bit me was that for some reason, pip
stopped working. After a bit of messing around with brew
, pip
and easy_install
, I found out it was almost entirely my own fault. I messed up my PATH
.
In the meantime, I had uninstalled all of brew
's Python, so I had to reinstall. For me, that entails Python, Numpy, Scipy and Matplotlib. Only this time, Scipy would not build. Some obscure error in some veclib_cabi_c.c
would report errors. A quick round of googling reveals:
In order to get Scipy to compile, you need to insert #include <complex.h>
in
./scipy/lib/blas/fblaswrap_veclib_c.c.src ./scipy/linalg/src/fblaswrap_veclib_c.c ./scipy/sparse/linalg/eigen/arpack/ARPACK/FWRAPPERS/veclib_cabi_c.c
That done, Scipy compiles perfectly fine.
But, that is not enough yet. As this blogpost outlines, Matplotlib is not currently compatible with libpng
1.5, which ships with Lion. Fortunately, this is already fixed in the most recent source on the Matplotlib repo, so you just have to checkout that:
pip install -e git+https://github.com/matplotlib/matplotlib.git#egg=matplotlib
By doing that, Matplotlib should install just fine.
Seriously though, these PyPi repos are in a very sorry state. Every time I install one of these packages, I have to jump through hoops and spend hours debugging packages that really should work right out of the box. After all, brew
, rvm
and gem
can do it just fine. Why is pip
such a horrible mess?
Installing Pygame using Homebrew
So I want to do audio development on the Mac without using Matlab. An alternative to Matlab is Python, or rather, Numpy, Scipy and Matplotlib. They are awesome for working with audio data. What they don't do however is playing back audio. There are several packages out there that would afford audio playback. If you are serious about this though, you not only want audio playback, you want asynchronous audio playback. That is, you want to send some audio data to the sound card and continue with your program without waiting for the audio to finish playing. This allows continuous audio playback of computer-generated sound.
Pygame is one package that allows this. (I will submit a patch to Pyaudio soon that will enable it there, too). There are pre-built binaries on the Pygame website that you can install easily. But then there would be no easy way to uninstall them, so what I would rather want is to install Pygame using package managers that allow easy updating and uninstallation. My tool of choice on the Mac is of course Homebrew.
Note that although I am mostly interested in audio playback, this post will detail the installation of all modules of Pygame, not just pygame.mixer
.
Homebrew won't install Pygame, but it will install all the prerequisites for Pygame. So, let's do that.
brew install sdl, sdl_mixer, sdl_ttf, libpng, jpeg, sdl_image, portmidi
This will install most packages for you. Note that libpng
is also available as a system library, so it is installed keg_only
, that is, without linking it in your path. We will need to compile against it though, so the next step is
brew link libpng
Now there is still one package missing, smpeg
. Sadly, smpeg
does not install its headers, so you can't compile against it. To fix that, type
brew edit smpeg
and add the following line just above the two end at the end of the file
include.install Dir["*.h"]
Then save the file. (I submitted a bug to have this fixed, so you might not need to do this when you read this). Now you can install smpeg
with the usual
brew install smpeg
and you will get the headers, too. Isn't Homebrew great?
Now that all the prerequisites are met, lets look at Pygame itself. This is rather more difficult, as it will not build properly against Homebrew libraries on its own. First, download the source package of Pygame from the [official website](http://www.pygame.org/download.shtml). Unpack it to some directory.
Now open a terminal and navigate to that directory. Me, I like [iTerm](http://iterm.sourceforge.net/), but Terminal.app will do just fine, too. In there, run python config.py
to create an initial setup file.
At this point, the setup file is mostly useless since config.py
failed to find any homebrew-installed library. It is also strangely garbled, so there is some manual labor to do. Open the file Setup
(no extension) in your favourite text editor. After the first comment block, you will see a line that looks like this
SDL = -I/NEED_INC_PATH_FIX -L/NEED_LIB_PATH_FIX -lSDL
Obviously, this is lacking the paths to the SDL library. If you installed Homebrew to its default directory, this will be in /usr/local…
. Hence, change this line to
SDL = -I/usr/local/include/SDL -L/usr/local/lib -lSDL
The next lines are strangely garbled. They say, for example
FONT = -lS -lD -lL -l_ -lt -lt -lf
Where they actually should say
FONT = -lSDL_ttf
Instead of having one -l
and then the library name SDL_ttf
, they put -l
in front of every single letter of the name. This is strange, and certainly wrong. So, correct it for FONT
, IMAGE
, MIXER
and SMPEG
.
Note that I did not tell you to do this for PORTTIME
, too. Actually, PORTTIME
is already correctly linked in PORTMIDI
, so you don't need that at all any more. Just delete or comment the PORTTIME
line.
Now that all the dependencies are corrected, lets enable the features. A few lines further down, there will be a block of lines, where most lines begin with a #
except for the ones beginning with _numericsurfarray…
and _camera…
, These are the different features of Pygame: The ones with the #
are disabled, the other two are enabled.
With all the stuff we installed earlier, you can now enable all features (remove the #
in front of imageext…
, font…
, mixer…
, mixer_music…
, _minericsndarray…
, movie…
, scrap…
and pypm…
).
Remember we disabled PORTTIME
a while ago? Right, so we have to remove that dependency: In the line starting with pypm…
, delete the part that says $(PORTTIME)
. Great. That was easy, right? Now save that file and go back to the Terminal.
We are now going to compile and install Pygame. The nice thing is, even though we are installing it manually, it will go in the right directories and it will be registered with pip
or easy_install
, so you can just invoke them if you want to uninstall it later by typing pip uninstall pygame
. This is something I love about Python!
Alright, now without further ado, install Pygame by typing
python setup.py install
Great! That's it! Everything should work now!
Installing Python/Numpy/Scipy/Matplotlib on OSX
For numerical analysis and signal processing prototyping, you would use Matlab. However, Matlab has some downsides that might make it unsuitable for your project. It might be too expensive. You might be a snobbish programmer that can't stand less-than-elegant programming languages. I certainly am.
So, you look for alternatives. You could take Octave, which is free, but that would not solve that ugly-code issue. You could take any scripting language you fancy, but Ruby, Perl and Python are too slow to do serious number crunching.
Then, you stumble upon that Python package called Numpy, which seems to be nearly as fast as Matlab when it comes to matrix processing and linear algebra. You then discover SciPy, which would add all that signal processing prowess of Matlab (do quick transformations, random numbers, statistics) to your toolbox. Last but not least, you need plotting. That would be Matplotlib then, which provides quick plotting facilities in Python.
And the best thing is, these three systems work really well together. They seem to be the perfect replacement for Matlab that could even be superiour to it in many regards.
Next up, you need to install all that stuff. If you are like me, you naturally want to do all that on a Mac. Also, you kind of dislike all these installer-thingies, which install stuff to unknown places and are nigh impossible to uninstall or update cleanly. Even though, you could of course just go to the individual websites, download Python, Numpy, SciPy and Matplotlib, run them installers, and be done. You would save yourself a lot of trouble that way.
But since you allegedly are like me, you instead fire up brew
and try to install all that stuff using that. Again, you could use MacPorts or Fink instead, but you probably had some bad experiences with them and you generally love the hackishness of Homebrew, so this is your natural first try.
So you set about this, you believe in packet managers and trust them to take care of every obstacle that might be lying in your way. First of all, install the latest developer tools from developer.apple.com. You might need to register (for free) to get them. Also, you need to install Homebrew.
To cut this short, here is what you need to get that Python running:
brew install python
This one should be obvious. At the time of writing, it will install Python 2.7.1. You could take Python 3, but matplotlib is not compatible to it, so you kind of have to stick with 2.7.1 instead.
You also need to put /usr/local/bin
and /usr/local/sbin
in the beginning of your path to make sure the new Python gets loaded instead of the pre-installed one. You do that by writing
export PATH=/usr/local/bin:/usr/local/sbin:$PATH
in your \~/.bash_profile
. (Create it if its not there–it is just a simple text file).
Now, if you type python --version
, you should get Python 2.7.1
as a response.
Alright, next up, install the python package manager:
brew install distribute brew install pip
This will come preconfigured for your newly installed Python. In an ideal world, this should be all. The world being as it is, the pip package of Matplotlib is severely broken and has one other unstated dependency:
brew install pkg-config
Also, SciPy is using some FORTRAN sources, so you need a fortran compiler:
brew install gfortran
Alright. That was enough. Now on to pip. With all these dependencies cleared, pip should be able to download Numpy and Scipy without trouble:
pip install numpy pip install scipy
Matplotlib, on the other hand, is more difficult to install. You see, pip is looking at the Python package repository PyPi for each package. PyPi then provides a URL. Pip then scans that website for links to suitable package files. But, Sourceforge changed its links a while ago, so pip gets confused and will download an outdated version. Sourceforge says, its new links are way better and no way we will change them back; Pip says, well, if Sourceforge can't provide proper links, that's not our problem. Oh My. Silly children.
So we have to do this manually:
pip install -f http://sourceforge.net/projects/matplotlib/files/matplotlib/matplotlib-1.0.1/matplotlib-1.0.1.tar.gz matplotlib
That URL comes straight from Sourceforge. Look for the latest version of Matplotlib, search for the download link to the source distribution (*.tar.gz
), copy that link and strip any trailing '/download'.
UPDATE:
It seems the matplotlib package was updated in the meantime, so you can just run pip install matplotlib
now.
This should now download and install matplotlib.
Thank you for reading.
Get a Mac
Bei meinem üblichen, Samstagmorgendlichen, bettlägrigen Web-Rundgang habe ich heute drei sehr nette Artikel von David Alison gefunden, einem Windows-Programmierer und selbsternanntem Microsoft-Fanboy, der sich einen Mac zulegt und unverhofft glücklich damit wird. So glücklich sogar, dass er inzwischen mehr oder weniger ein Mac-only-User geworden ist.
Ziemlich genau wie ihm ging es mir auch.
- Teil 1: A hardcore Windows guy gets a Mac
- Teil 2: After two months of Mac, here's why I switched
- Teil 3: Switching from Windows to Mac - One Year Later
Wens interessiert: Das ist eine sehr hübsche Einführung darin, warum Macs toll sind. Viel besser als alles, was ich dazu schreiben würde oder bereits getan habe.