Basti's Scratchpad on the Internet

Posts tagged "computers":

16 Jun 2024

Python Inception

At most companies I have worked for, there was some internal Python code base that relied on an old version of Python. But especially for data science, I'd often want to execute that code from an up-to-date Jupyter Notebook, to do some analysis on results.

When this happened last time, I decided to do something about it. Here's a Jupyter cell magic that executes the cell's code in a different Python, pipes out all of STDOUT and STDERR, and imports any newly created variables into the host Python. Use it like this:

%%py_magic /old/version/of/python
import this
truth = 42

When this cell executes, you will see the Zen of Python in your output, just as if you had import this in the host Python, and the variable truth will now be 42 in the host Python.

To get this magic, execute the following code in a preceding cell:

import subprocess
import sys
import pickle
import textwrap
from IPython.core.magic import needs_local_scope, register_cell_magic
 
@register_cell_magic
@needs_local_scope
def py_magic(line, cell, local_ns=None):
    proc = subprocess.Popen([line or 'python'],
                            stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
                            encoding='UTF8')
    # send a preamble to the client python, and remember all pre-existing local variable names:
    proc.stdin.write(textwrap.dedent("""
        import pickle as _pickle
        import types as _types
        _names_before = [k for k, v in locals().items()] + ['_f', '_names_before']
        try:
    """))
    # send the cell's contents, indented to run in the try:
    for line in cell.splitlines():
        proc.stdin.write("    " + line + "\n")  # indent!
    # send a postamble that pickles all new variables or thrown exceptions:
    proc.stdin.write(textwrap.dedent("""
        # save results to result.pickle
        except Exception as exc:
            with open('result.pickle', 'wb') as _f:
                _pickle.dump({'type':'error', 'value': exc}, _f)
        else:
            with open('result.pickle', 'wb') as _f:
                _values = {k:v for k, v in locals().items()
                               if not isinstance(v, _types.ModuleType) 
                                  and not k in _names_before}
                _safe_values = {}  # skip any unpickleable variables
                for k, v in _values.items():
                    try:
                        _pickle.dumps(v)
                    except Exception as _exc:
                        print(f'skipping dumping {k} because {_exc}')
                    else:
                        _safe_values[k] = v
                _pickle.dump({'type':'result', 'value': _safe_values}, _f)
        finally:
            quit()
    """))
    # print any captured stdout or stderr:
    stdout, stderr = proc.communicate()
    if stdout:
        print(stdout, file=sys.stdout)
    if stderr:
        print(stderr, file=sys.stderr)

    # load new local variables or throw error:
    try:
        with open('result.pickle', 'rb') as f:
            result = pickle.load(f)
        if result['type'] == 'error':
            raise result['value']
        elif result['type'] == 'result':
            for key, value in result['value'].items():
                try:
                    local_ns[key] = value
                except Exception as exc:
                    print(f"skipping loading {key} because {exc}")
    finally:
        pathlib.Path('result.pickle').unlink()  # remove temporary file
  
del py_magic  # otherwise the function overwrites the magic

I love how this sort of trickery is relatively easy in Python. Also, this is the first time I've used a try with except, else, and finally.

Tags: computers python
09 May 2024

AI Predictions

Meta just invested 30 Billion Dollars into AI accelerators1. That's roughly equivalent to one Manhattan Project worth of money. Meta must expect a comparable return on that investment. But with that kind of money, that return must be disruptive.

And yet, AI does not feel disruptive. In my life, I have witnessed a few consumer technology disruptions: portable computers, portable telephones, internet-connected "smart" telephones, always-available GPS. Perhaps even tablet computers, smart watches, and electric cars? They all felt exciting! They all felt immediately obviously useful, if perhaps not always to me. But AI does not excite me. So if it's not for me, where is that $30B market that Meta is envisioning?

The best I can think of is a "command line for the common man". But the power of the command line comes from unflinchingly powerful commands, and behaving deterministically. Both of these characteristics are the antithesis of current AI technology.

We will not give an AI assistant the power to execute "format my hard drive", or "delete my Google account", even though a command line interface clearly would. Yet without that power, the AI assistant is toothless, and less useful. Even if we did, how could we trust the AI assistant to actually do what we want, and not misunderstand us? When interacting with LLMs, I am reminded of Terry Pratchett's gods of the Discworld, who wield absolute power, but you don't want to ask them for help as they're too likely to do what they think you wanted instead of what you actually asked.

But without power, and without deterministic behavior, you can't have a command line experience.

I keep coming back to that question: What is the disruptive use case of AI? Sure, we'll outsource some tasks to AI that we'd previously outsource overseas. This will result in rampant tragedy, and is already known to be difficult to pull off successfully. We'll enhance many tasks with short snippets of AI, to speed up menial programming tasks, and writing tasks, translation, image generation. But that's a feature-addition to existing software, not a disruption, let alone a $30B disruption.

Perhaps I'm wrong. Only time will tell. But I hereby predict that disruptive AI is a bubble.

Footnotes:

1

: For reference, one Billion Dollars is a 1km stack of $100 bills

Tags: computers
21 Apr 2023

OS Customization and MacOS

I was an Apple fanboy some years ago. Back then, whenever something was odd on my computer, I was surely just using it wrong. Nowadays, I see things the other way around: We're not "holding it wrong", the computer is just defective. Computers should do our bidding, not vice versa. So here's a bunch of things that I do to my computers to mold them to my way of working.

Keyboard Layout

I switch constantly between a German and English keyboard layout, and regularly between various machines. My physical keyboards are German, and my fingers are used to the Windows-default German and (international) US keyboard layout. These are available by default on Windows and Linux, but MacOS goes its own way.

However, keyboard layouts on MacOS are saved in relatively simple text files, and can be modified with relative ease. The process goes like this: Download Ukelele (free) to create a new keyboard layout bundle for your base layout1. Inside that bundle, there's a *.keylayout file, which is an XML file that defines the characters that each key-modifier combination produces. I changed that to create a Windows-like US keyboard layout. And I replaced the keyboard icon with something sane (not "A") by creating a 256x256 pixel PNG, opening it in Preview, holding alt while saving to select the ICNS format. Save the keyboard bundle to ~/Library/Keyboard Layouts and reboot. Then I remove the unremovable default ("A") German layout by selecting another one, then plutil -convert xml1 ~/Library/Preferences/com.apple.HIToolbox.plist, and delete the entry from AppleEnabledInputSources. Now reboot. Almost easy. Almost.

One the one hand, this was quite the ordeal. On the other, I have tried to do this sort of thing on Windows and Linux before, and for the life of me could not do it. So I actually think this is great!

Keyboard Shortcuts

My main text editor is Emacs, and I am very used to its keyboard shortcuts. Of particular note are CTRL-A/E for going to the beginning/end of a line, and Alt-B/F for navigating forward/backwards by word. I have long wanted to use these shortcuts not just in Emacs and readline-enabled terminal applications, but everywhere else, too. And with MacOS, this is finally possible: Install BetterTouchTool ($22), and create keyboard shortcuts that maps, e.g. Alt-B/F to Alt-←/→. Ideally, put this in a new activation group that excludes Emacs. It may be necessary to remove the keyboard character for Alt-B/F from your keyboard layout before this works. I've spent an embarrassing number of hours trying to get this to work on Windows and Linux, and really got nowhere2. Actually, however, most readline shortcuts such as Ctrl-A/E/B/F/K/Y already work out of the box on MacOS!

Mouse Configuration

I generally use a trackpad, and occasionally a traditional mouse for image editing, and have used a trackball. I find that any one specific device will lead to wrist pain if used constantly, so I switch it up every now and then. The trackpad and trackball, however, need configuration to be usable.

After experimenting with many a trackpad device, I have found Apple touch pads the best trackpads on the market3. On MacOS, they lacks a middle mouse click. So I created a trackpad shortcut in the aforementioned BetterTouchTool ($22) to map the middle click on a three-finger tap (can also be had for free with MiddleClick (OSS)). For Windows, Magic Utilities ($17/y) provides a wonderful third-party driver for Apple devices that also supports the three-finger tap. I have not gotten the Apple touch pad to pair reliably on Linux, and have generally found their touch pad driver libinput a bit lacking.

My trackball is a Kensington SlimBlade. To scroll, you rotate the ball around its vertical axis. This is tedious for longer scroll distances, however. But there's an alternative scrolling method called "button scrolling", where you hold one button on the trackball, and move the ball to scroll. You need to install the Kensington driver to enable this on Windows and MacOS. Button scrolling is available on Linux as well using xinput4, but I haven't gotten the setting to stick across reboots or sleep cycles. So I wrote a background task that checks xinput every five seconds, which does the trick.

Window Management

Frankly, Windows does window management correctly. Win-←/→ moves windows to the left and right edge of the screen, as does dragging the window to the screen border. Further Win-←/→ then moves the window to the next half-screen in that direction, even across display boundaries. KDE does this correctly out of the box as well, Gnome does not do the latter, and it drives me mad. MacOS doesn't do any of these things. But Rectangle (OSS) does. Easy fix. (BetterTouchTool can do it, too, but Rectangle is prettier)

Furthermore, I want Alt-Tab to switch between windows. Again, MacOS is the odd one out, which uses CMD-Tab to switch between apps, now windows. And then there's another shortcut for switching between windows of the same app, but the shortcut really doesn't work at all on a German keyboard. Who came up with this nonsense? Anyway, Witch ($14) implements window switching properly.

Application Management

In Windows and Linux, I hit the Windows key and start typing to select and start an app. In MacOS, this is usually Cmd-Space, but BetterTouchTool can map it to a single short Cmd, if you prefer.

More annoying are the various docks and task bars. I always shove the dock off to the right edge of the screen, where it stays out of the way. Windows 10 had a sane dock, but then 11 came and forced it to the bottom of the screen. Dear OS makers, every modern screen has plenty of horizontal space. But vertical space is somewhat limited. So why on earth would you make a rarely used menu such as the dock consume that precious vertical space by default? And Microsoft, specifically, why not make it movable? Thankfully, there's StartAllBack ($5), which replaces the Windows task bar with something sensible, and additionally cleans up the start menu if you so desire. On KDE, I fractionally prefer Latte (OSS) over KDE's native dock. The MacOS dock is uniquely dumb, offering no start menu, and allowing no window selections. But it's unobtrusive and can be moved to the right edge, so it's not much of a bother.

File Management

One of the most crucial tasks in computer work in general is file management. I am not satisfied with most file managers. Dolphin on KDE works pretty well, it has tabs, can bulk-rename files, can display large directories without crashing, and updates in real time when new files are added to the current directory. Gnome Nautilus is so bad it is the main reason I switched to KDE on my Linux machines. Finder on MacOS is passable, I suppose, although the left sidebar is unnecessarily restrictive (why can't I add a shortcut to a network drive?). Windows Explorer is really rather terrible, lacking a bulk-rename tool, and crucially, tabs. In Windows 10, these can be added with Groupy ($12) (set it to only apply to explorer.exe). Windows 11 has very recently added native tabs, which work OK, but can't be detached from the window.

The sad thing is that there are plenty of very good file manager replacements out there, but none of the OSs have a mechanism for replacing their native file manager in a consistent way, so we're mostly stuck with the defaults.

Oh, and I always remove the iCloud/OneDrive sidebar entries, which is surprisingly tedious on Windows.

Hardware Control

On laptops, you can control screen brightness from your keyboard. On desktops, you can not. However, some clever hackers have put together BetterDisplay (OSS for screen brightness), which adds this capability to MacOS. That's actually a capability I have wanted for quite a while, and apparently it is only available in MacOS. Great stuff!

Less great is that MacOS does not allow volume control on external sound cards. SoundSource ($47) adds this rather crucial functionality back, once you go through the unnecessarily excruciating process of enabling custom kernel extensions. Windows and Linux of course natively support this.

Another necessary functionality for me is access to a non-sucky (i.e. no FAT) cross-platform file system. At the moment, the most portable file system seems to be NTFS, of all things. Regrettably, MacOS only supports reading NTFS, but no writing. Paragon NTFS (€20) adds this with another kernel extension, and promptly kernel-panicked my computer. Oh joy. At least it's only panicking for file transfers initiated by DigiKam, which I can work around. Paragon Support says they're working on it. I'm not holding my breath. Windows and Linux of course natively support NTFS.

System Management

I have learned from experience not to trust graphical backup programs. TimeMachine in particular has eaten my backups many times already, and can not be trusted. But I have used Borg (OSS) for years, and it has so far performed flawlessly. Even more impressive, my Borg backups have a continuous history despite moving operating systems several times. It truly is wonderful software!

On Windows, I run Borg inside the WSL, and schedule its backups with the Windows Task Scheduler. On Linux, I schedule them with systemd units. On MacOS, I install Borg with Homebrew (OSS) and schedule the backups with launchd tasks. It's all pretty equivalent. One nice thing about launchd, however, is how the OS immediately pops up a notification if there's a new task file added, and adds the task to the graphical system settings.

I have to emphasize what a game-changer the WSL is on Windows. Where previously, such simple automations where a pain in the neck to do reliably, they're now the same simple shell scripts as on other OSes. And it perfectly integrates with Windows programs as well, including passing pipes between Linux and Windows programs. It's truly amazing! At the moment, I'd rate Windows a better Unix system than MacOS for this reason. Homebrew is a passable package manager on MacOS, but the way it's ill-integrated into the main system (not in system PATH) is a bit off-putting.

App Compatibility

I generally use my computer for three tasks: General document stuff, photo editing, and video games.

One major downside of Apple computers is that video games aren't available. This has become less of a problem to me since I bought a Steam Deck, which has taken over gaming duties from my main PC. Absolutely astonishingly, the Steam Deck runs Windows games on Linux through emulation, which works almost flawlessly, making video games no longer a Windows-only proposition.

What doesn't work well on Linux are commercial applications. Wine generally does not play well with them, and frustratingly for my photo editing, neither VMWare Workstation Player (free) nor VirtualBox (OSS) support hardware-accelerated VMs on up-to-date Linux5. So where MacOS lacks games, Linux lacks Photoshop. Desktop applications in general tend to be unnecessarily cumbersome to manage and update on Linux. Flatpak is helping in this regard, by installing user-facing applications outside of the OS package managers, but it remains more work than on Windows or MacOS. The occasional scanner driver or camera interface app can also be troublesome on Linux, but that's easily handled with a VirtualBox VM (with the proprietary Extension Pack for USB2 support), and hasn't really bothered me too much.

Luckily for me, my most-used apps are generally OSS tools such as Darktable (OSS) and DigiKam (OSS), or cross-platform programs like Fish (OSS), Git (OSS), and Emacs (OSS). This is however, where Windows has a bit of a sore spot, as these programs tend to perform noticeably worse on Windows than on other platforms. Emacs and git in particular are just terribly slow on Windows, taking several seconds for routine operations that are instant on other platforms. That's probably due to Windows' rather slow file system and malware scanner for the many-small-files style of file management that these Unix tools implement. It is very annoying.

Conclusions

So there's just no perfect solution. MacOS can't do games, Linux can't run commercial applications, and Windows is annoyingly slow for OSS applications. Regardless, I regularly use all three systems productively. My job is mostly done on Windows, my home computer runs MacOS, and my Steam Deck and automations run Linux.

Overall, I currently prefer MacOS as my desktop OS. It is surprisingly flexible, and more scriptable than I thought, and in some ways is actually more functional than Linux or Windows. The integrated calendar and contacts apps are nice, too, and not nearly as terrible as their Windows/Linux counterparts. To say nothing of the amazing M1 hardware with its minuscule power draw and total silence, while maintaining astonishing performance.

Linux is where I prefer to program, due to its sane command line and tremendously good compiler/debugger/library infrastructure. As a desktop OS, it does have some rough edges, however, especially for its lack of access to commercial applications. While Linux should be the most customizable of these three, I find things tend to break too easily, and customizations are often scattered widely between many different subsystems, making them very hard to get right.

Theoretically, Windows is the most capable OS, supporting apps, OSS, and games. But it also feels the most user-hostile of these three, and the least performant. And then there's the intrusive ads everywhere, its spying on my every move, and at work there's inevitably a heavy-handed administrator security setup that gets in the way of productivity. It's honestly fine on my home computer, at least since they introduced the WSL. But using it for work every day is quite enough, so I don't want to use it at home, too.

Footnotes:

1

if there's an easier way to create a keyboard layout bundle, please let me know. I didn't use ukelele for anything but the bundle creation.

2

It occurs to me that it might actually be possible to do something like on Windows this with AutoHotkey (free). I'll have to try that!

3

The Logitech T650 is actually not bad, but the drivers are a travesty. Some Wacom tablets include touch, too, but they palm rejection is abysmal and they don't support gestures.

4

xinput set-prop "Kensington Slimblade Trackball" "libinput Scroll Method Enabled" 0, 0, 1 # button scrolling
xinput set-prop "Kensington Slimblade Trackball" "libinput Button Scrolling Button" 8 # top-right button

5

VirtualBox does not have a good accelerated driver, and VMWare does not support recent kernels. Qemu should be able to solve this problem, but I couldn't get it to work reliably.

Tags: computers linux macos windows
04 Dec 2021

Fixing AMD OpenCL on Windows

When I recently installed Windows 11 on my desktop, my photo editor Darktable suddenly got much slower than it used to be. When I looked into its preferences, I noticed OpenCL was no longer available.

As it turns out, some versions of the AMD graphics driver apparently no longer ship with OpenCL support on Windows. However, they do ship with the necessary libraries, it's just that these libraries are not registered any longer.

To register them, open the Registry Editor (aka regedit.exe), navigate to the key HKEY_LOCAL_MACHINE\SOFTWARE\Khronos\OpenCL\Vendors (if the key does not exist, create it), and create a new DWORD of Value 0. Now rename the DWORD to be the path to your amdocl64.dll. Mine is C:\Windows\System32\DriverStore\FileRepository\u0344717.inf_amd64_d38cec78c83eee99\B343886\amdocl64.dll. Search C:\Windows\System32\ for amdocl64.dll to find the correct path on your computer.

With that, OpenCL was once again recognized by Darktable and the rest of my photo editing programs.

(source)

Tags: computers windows
27 Dec 2020

A review of iOS, from an Android user's perspective

My Pixel 2 phone received its final software update the other day, and its battery took the occasion to really show its age. So it was time for a new phone. Which is not a decision to make lightly, considering that I spend multiple hours per day using it. And this time, I was inclined to give the iPhone another go, mostly because of Google's gross lack of human decency in the last few years. So, after years of using various Android devices, I bought a used, red, iPhone SE (2020). I made this choice with some trepidations, as my last experience with iOS was an iPad 3 from 2012. These are my experiences.

As a seasoned Android user, my first impressions of iOS were a bit of a mess. There are things that swipe up from the bottom, swipe down from the top, to the left of the home screens, to the right of the home screens, and at various permutations of pressing/holding the home button. Everything animates, takes a long time, and moves the viewport around. Particularly annoying is the placement of the back button in the top left corner, i.e. the most-inconvenient place on the entire screen for the arguably most-important gesture of the entire UI1. A close second are the context menus that slide out from a long-pressed item, with the menu thus in a different position on the screen every time you long press anything. These things may be less flashy on Android, but are seriously simpler.

I also immediately missed Android's customizeable home screen, with freely-positionable app icons and a plethora of useful widgets. iOS is very restrictive in this regard, and seemingly for no good reason. Why is the list of all apps (right-of-homescreen) sorted arbitrarily into nonsensical folders instead of a plain list? Why are app widgets allowed, but only on that weird left-of-the-home-screen screen? Why can't I have a currently-playing widget for my podcast player, a weather radar, or my groceries list? Apparently, iOS 14 has a brand new API that does now allow Android-like widgets in iOS, but at the moment they were only available for Apple's own (useless) apps.

My second big stumbling block was the iOS on-screen keyboard. At first glance, I thought it a terribly clunky thing. Actions as simple as inserting a comma require multiple taps, and positioning the cursor seemed almost comically difficult. But then I discovered that the "123" button in the bottom left can be swiped instead of tapped, which makes commas and periods and hyphens available to a quick swipe. That is seriously cool, if slightly hampered by my accidentally activating Control Center instead of swiping from "123" a bit too often. And precise cursor positioning is hidden behind a long-press of the spacebar. Very cool indeed. With these gestures, the iOS on-screen keyboard is actually not bad at all.

Autocorrect seems capable as well, and multi-language aware (hear that, Android?). And, mind-blowingly, a triple-swipe on the content area engages undo and redo in the current text area. Albeit a nigh-undiscoverable gesture, this is miles better than Androids undo/redo system (there isn't one). Actually, Android text fields do support undo and redo if you press Ctrl-Z on the keyboard, it's just that no on-screen keyboard has a Ctrl key. This is my number one grievance with Android at the moment (although there are workarounds).

As a summary to the keyboard situation, I came to respect the iOS keyboard. I still miss a German "ß" key, more control about the autocorrect system, and a more modern design. But it works reasonably well. Better than Android's stock keyboard for sure.

My second big hurdle with iOS was the camera. Just like my Pixel's camera, the iPhone takes perfectly acceptible photos. But everything else is just plain worse than on Android. It starts with the way you get access to the camera, either by swiping right on the lock screen, or by tapping the camera app on the home screen. On Android, you double-click the lock button to start the camera, then hit the volume button to take a photo. Not only is this significantly faster, it also requires no ungloved finger, and no look onto the screen. And it doesn't make that abominable shutter sound, either, that the iPhone requires (unless the entire phone is silenced or you take video).

When I take a picture with my phone, it is because I don't have a dedicated camera at hand. Considering the number of cameras I own, this is most likely to happen when I need a camera fast. Thus the speed from pants to picture is extremely important to me, and this is a clear victory for Android. And additionally, Pixel phones have supported stacked RAWs since the original Pixel2, and offer quite a comprehensive image editing suite right in the stock camera app. And I prefer the Pixel's relatively neutral rendering over the iPhone's oversharpened, over-denoised, waxy images. But that might be just me.

At this point, I had more or less given up on iOS, and bought a Pixel 4a, which is probably Android's closest competitor to the iPhone SE.

Beyond the camera, there were a number of annoyances with iOS that I found hard to get used to. Like the fingerprint reader on the SE being very unreliable for me (50% miss rate), and awkwardly requiring a distinct push to activate the home button, where the Pixel's is quicker and in a more convenient location. The home button is in fact not a button, but an immovable touch pad that fakes a button-like behavior with a little rumble effect when pressed uncomfortably hard. Pressing a hard surface really hard did not get comfortable to me, especially not when invoking the multi-tasking switcher with a double-press. And it's awkwardly placed at the very bottom of the device, where my thumb does not rest with any kind of force in normal usage. Another real problem is the lack of third-party browsers3, which just leaves you in the cold should a website not work in Safari, or should you dislike Safari's rendering or its very limited ad-blockers. I was particularly annoyed by Safari's context menu for links, which slowly slides out from the link's location, often extending far outside of the screen. Thus opening links in new tabs is just awkwardly slow and annoying in Safari, where Android's more utilitarian menu gets the job done much quicker. Although a phone with a bigger screen might mitigate this problem somewhat.

On the topic of animations in general, one of my favorite features in Android is the animation speed setting in the Developer settings. If you want to feel like you got a new, faster phone, just set the animation speed to 0.5x, and marvel at the newfound snappiness of everything.

Apps in general feel annoyingly restrictive on iOS. Where is a true Firefox, with addons? Where is a Youtube player that can play a YouTube video in the background while the screen is locked, or block ads (NewPipe is the bee's knees!). Or a file synchronization tool that can actually synchronize useful data outside of its own sandox? In fact, my original hopes for iOS were driven by my memory of the fabled higher quality apps available only on the App Store. Looking at some of my old favorites on the SE however, I was forced to take off those rose-tinted glasses. These apps might have been radical back in the day, but the world has moved on. I don't see a pervasive difference in app quality between iOS and Android any longer. Of course iOS apps do still cost a more money on average, and often can't be test-driven without paying. That two-hour free return policy on the Play Store is seriously genius.

Additionally, there were a number of odd problems with the iPhone SE that I found hard to make sense of. For example, apps were very frequently getting booted out of memory. So much so in fact, that often my RSS feeds would not refresh in the background, and podcasts would not download. Even though the iPhone SE sure does have a lot less memory than the Pixel 4a, I hadn't expected this to be an actual problem4. And the iPhone would frequently misunderstand a vertical swipe for a horizontal one, or initiate an edge swipe when my finger was still clearly on the screen; My bluetooth headphones sounded noticeably worse, with a strange clipping artifact that should not be there; Significantly worse cell reception and voice quality; Low contrast and tiny text on lock screen notifications; And only barely adequate battery life. And let's not even talk about the stupid Lightning cable, the (comparatively) laggy and tiny TFT screen, the lack of a podcast client I like, and my horrible experiences during the setup process.

So, in summary, the iPhone SE was not for me. Don't get me wrong, it's a nice enough phone, and probably enough of a smartphone for most people. But there were a number of issues with both its hardware and its software, where I found Android and the Pixel 4a plainly more productive. Which is surprising, as my mind still had iOS pegged as the premium alternative, that I would totally buy if I wasn't too cheap for it. But in actual use I was annoyed at its preference of form over function, with many a distracting animation and a number of glaring ergonomic mistakes. Well, another lesson learned. The only truly significant advantage of iOS remains its vastly superior software support story: The SE will probably receive software updates until 2025 or 2027, where the best-in-class Pixel 4a will definitely expire in 2023.

Footnotes:

1

in some places, a right swipe can be used instead of the back button. But the behavior is too inconsistent to be useful

2

what Apple calls "ProRAW", and only makes available in the iPhone 12. iOS does support ordinary RAW, but only in third-party camera apps, and anyway non-stacked RAW files are rather useless from a tiny phone sensor

3

there are different browsers, but they are all required to use Safari's web renderer.

4

I hear this might have been a bug in iOs 14.3? What this says about the software quality of iOS in general is even more troubling, however.

Tags: computers ui
17 Sep 2020

Dear Computer, We Need to Talk

After years of using Linux on my desktop, I decided to install Windows on my computer, to get access to a few commercial photo editing applications. I'll go into my grievances with Linux later, but for now:

I tried to install Windows, you won't believe what happened next

Like I have done many times with Linux, I download a Windows image from my university, and write it to a USB drive, then reboot into the USB drive. The USB drive can't be booted. A quick internet search leads me to a Microsoft Support page on how to Install Windows from a USB Flash Drive, which says that

Instead, one has to format the stick as FAT32, make it active, then copy the files from the image to it. So I follow the instructions and open the Disk Management program. It does not offer an option of FAT32, nor for making the partition active. I settle on (inactive) exFAT instead. It doesn't boot.

I switch over to Linux, where I can indeed make a FAT32 partition, and I can mark it as bootable, which I take as the equivalent of active. But Linux can not open the Windows image to copy the files onto the USB stick. So back to Windows, for copying the files. Except they can't be copied, because some of them are larger than 4Gb, which can't be written to a FAT32 partition. What now?

While researching how to download a different version of Windows 10, I stumble upon the Media Creation Tool, which automatically downloads Windows 10 and writes it to the USB stick correctly. Why was this not pointed out in the article above? Who knows. At any rate, it works. I can finally install Windows.

The installation process requires the usual dozen-or-so refusals of tracking, ads, privacy intrusions, and voice assistants. I wish I could simply reject them all at once. And then the install hangs, while "polishing up a few things". Pressing the helpful back button, and then immediately the forward button unhangs it, and the installation completes.

Next up are drivers. It feels anachronistic to have to install drivers manually in this day and age, but oh well. The new GPU driver to make screen tearing go away, a driver for my trackball to recognize the third mouse button, a wacom driver, ten or so Intel drivers of unknown utility. The trackball driver is not signed. I install it anyway. The GPU driver does not recognize my GPU and can't be installed. A quick Internet search reveals that my particular AMD/Intel GPU/CPU was discontinued from support by both AMD and Intel, and does not have a current driver. But fora suggest that up to version 20.2.1 of the AMD driver work fine. They don't, the driver crashes when I open images in my photo editor. An even older version published by Intel does work correctly. So now I am running an AMD GPU with an Intel driver from 2018.

Installing and setting up Firefox and my photo editors works without issue, thank goodness. Emacs has a Windows installer now, which is greatly appreciated. OpenCL and network shares just work. This is why I'm installing Windows next to my Linux.

But Windows is still not activated. I copy my university's product key in the appropriate text box, but hesitate: That's for Windows Enterprise, and I'd be just fine with Home. So I cancel the activation without activating. A helpful link in the activation systems sends me to the Microsoft Store to get my very own version of Windows Home for €145, which normally retails for around €95, so that's a no-go. Whatever, I'll go with my university's Enterprise edition. Except the activation box now says my product key is invalid. And the Store now literally says "We don't know how you got here but you shouldn't be here" instead of selling me Windows. After a restart it installs and activates Windows Enterprise, even though I never actually completed the activation.

I install Git, but in order to access my Github I need to copy over my SSH key from the Linux install. Which I can't boot at the moment, because installing Windows overwrites the boot loader. This is normal. So I download Ubuntu, write it to the USB stick, boot into it, recover the bootloader, boot into my old install, reformat the stick, copy the files to the stick, boot back into Windows, and the files aren't on the stick. Tough. Boot back into Linux, copy the files onto the stick, eject the stick, boot back into Windows, copy the files to the computer. Great user experience.

Now that I have my SSH key, I open a Git Bash to download a project. It says my credentials are incorrect. I execute the same commands in a regular CMD instead of Git Bash, and now my credentials are correct. Obviously.

There are several programs that claim to be able to read Linux file systems from Windows. They do not work. But Microsoft has just announced that you will be able to mount Linux file systems from WSL in a few weeks or months. So maybe that will work!

I set my lock screen to a slideshow of my pictures. Except my pictures do not show up, and I get to see Window's default pictures instead. An internet search reveals that this is a wide-spread problem. Many "solutions" are offered in the support fora. What works for me is to first set the lock screen to "Windows Spotlight", then to "Slideshow". Only in that order will my pictures be shown.

I will stop here. I could probably go on ad infinitum if I wanted to. This was my experience of using Windows for one day. I consider these problems relatively benign, in that all of them had solutions, if non-obvious ones.

Why install Windows in the first place?

Part of the reason for installing Windows was my growing frustration with Linux. I have been a happy user of KDE of various flavors for about seven years now. But ever since I got into photo editing, things began to become problematic:

My photo editor requires OpenCL, but the graphics driver situation on Linux is problematic, to say the least. I generally managed to get RocM running most of the time, but kernel updates frequently broke it, or required down- or upgrading RocM. It was a constant struggle.

I wanted to work with some of my data on a network share, but KDE's implementation of network shares does not simply mount them for applications to use, but instead requires each application to be able to open network locations on their own. Needless to say, this worked almost never, requiring many unnecessary file copies. Perhaps Gnome handles network shares better, but let's not open that can of worms.

Printing photos simply never worked right for me. The colors were off, photo papers were not supported, the networked printer was rarely recognized. Both for a Samsung printer and an Epson and a Canon. One time a commercial printer driver for Linux printed with so much ink it dripped off the paper afterwards. Neither Darktable nor Gimp nor Digikam have a robust printing mode. I generally resorted to Windows for printing.

I ran that Windows in a virtual machine. With Virtualbox, the virtual machine would be extremely slow, to the point where it had a delay of several seconds between typing and seeing letters on the screen. VMWare did better, but would suddenly freeze and hang for minutes at a time. Disabling hugepages helped sometimes, for a short while. The virtual machine network was extremely unreliable. Some of these issues were probably related to my using a 4K screen.

Speaking of screens, I have two screens, one HighDPI 4k and one normal 1440p. Using X, the system can be either in HighDPI mode, or in normal mode. But it can't drive the two displays in different modes. Thus the second monitor was almost useless and I generally worked only on the 4k screen. With Wayland I would have been able to use both screens in different modes, but not be able to color-calibrate them or record screen casts. Which is completely unacceptable. So I stuck with one screen and X. In Windows, I can use both screens and calibrate them.

Additionally, Linux hardware support is still a bit spotty. My SD card reader couldn't read some SD cards because of driver issues. It would sometimes corrupt the SD card's file systems. USB-connected cameras were generally not accessible. The web cam did not work reliably. The CPU fan ran too hot most of the time.

So there had been numerous grievances in Linux that had no solutions. Still I stuck with it because so many more smaller issues were actually fixable if I put in the work. In fact I had accumulated quite a number of small hacks and scripts for various issues. I feared that Windows would leave me without recourse in these situations. And it doesn't. But at least the bigger features generally work as advertised.

Where do we go from here?

Just for completion's sake, I should really find an Apple computer and run it through its paces. From my experience of occasionally using a Macbook for teaching over the last few years, I am confident that it fares no better than Linux or Windows.

Were things always this broken? How are normal people expected to deal with these things? No wonder every sane person now prefers a smartphone or tablet to their computers. Limited as they may be, at least they generally work.

There is no joy in technology any more.

Tags: computers linux windows
27 May 2020

How to Write a Dissertation

Assembling scientific documents is a complex task. My documents are a combination of graphs, data, and text, written in LaTeX. This post is about combining these elements, keeping them up to date, while not losing your mind. My techniques work on any Unix system on Linux, macOS, or the WSL.

Text

For engineering or science work, my deliverables are PDFs, typically rendered from LaTeX. But writing LaTeX is not the most pleasant of writing environments. So I've tried my hand at org-mode and Markdown, compiled them to LaTeX, and then to PDF. In general, this worked well, but there always came a point where the abstraction broke, and the LaTeX leaked up the stack into my document. At which point I'd essentially write LaTeX anyway, just with a different syntax. After a few years of this, I decided to cut the middle-man, bite the bullet, and just write LaTeX.

That said, modern LaTeX is not so bad any more: XeLaTeX supports normal OpenType fonts, mixed languages, proper unicode, and natively renders to PDF. It also renders pretty quickly. My entire dissertation renders in less than three seconds, which is plenty fast enough for me.

To render, I run a simple makefile in an infinite loop that recompiles my PDF whenever the TeX source changes, giving live feedback while writing:

diss.pdf: diss.tex makefile $(graph_pdfs)
	xelatex -interaction nonstopmode diss.tex

We'll get back to $(graph_pdfs) in a second.

Graphs

A major challenge in writing a technical document is keeping all the source data in sync with the document. To make sure that all graphs are up to date, I plug them into the same makefile as above, but with a twist: All my graphs are created from Python scripts of the same name in the graphs directory.

But you don't want to simply execute all the scripts in graphs, as some of them might be shared dependencies that do not produce PDFs. So instead, I only execute scripts that start with a chapter number, which conveniently sorts them by chapter in the file manager, as well.

Thus all graphs render into the main PDF and update automatically, just like the main document:

graph_sources = $(shell find graphs -regex "graphs/[0-9]-.*\.py")
graph_pdfs = $(patsubst %.py,%.pdf,$(graph_sources))

graphs/%.pdf: graphs/%.py
	cd graphs; .venv/bin/python $(notdir $<)

The first two lines build a list of all graph scripts in the graphs directory, and their matching PDFs. The last two lines are a makefile recipy that compiles any graph script into a PDF, using the virtualenv in graphs/.venv/. How elegant these makefiles are, with recipe definitions independent of targets.

This system is surprisingly flexible, and absolutely trivial to debug. For example, I sometimes use those graph scripts as glorified shell scripts, for converting an SVG to PDF with Inkscape or some similar task. Or I compile some intermediate data before actually building the graph, and cache them for later use. Just make sure to set an appropriate exit code in the graph script, to signal to the makefile whether the graph was successfully created. An additional makefile target graphs: $(graph_pdfs) can also come in handy if you want ignore the TeX side of things for a bit.

Data

All of the graph scripts and TeX are of course backed by a Git repository. But my dissertation also contains a number of databases that are far too big for Git. Instead, I rely on git-annex to synchronize data across machines from a simple webdav host.

To set up a new writing environment from scratch, all I need is the following series of commands:

git clone git://mygitserver/dissertation.git dissertation
cd dissertation
git annex init
env WEBDAV_USERNAME=xxx WEBDAV_PASSWORD=yyy git annex enableremote mywebdavserver
git annex copy --from mywebdavserver
(cd graphs; pipenv install)
make all

This will download my graphs and text from mygitserver, download my databases from mywebdavserver, build my Python environment with pipenv, recreate all the graph PDFs, and compile the TeX. A process that can take a few hours, but is completely automated and reliable.

And that is truly the key part; The last thing you want to do while writing is being distracted by technical issues such as "where did I put that database again?", "didn't that graph show something different the other day?", or "I forgot to my database file at work and now I'm stuck at home during the pandemic and can't progress". Not that any of those would have ever happened to me, of course.

Tags: computers emacs workflow
21 Jan 2020

I Miss My Mac

There was a time, maybe from 2009 to 2011, when Apple computers were glorious. They were elegant devices, beautiful to me, and powerful. Never before, or since, have I loved a machine as I loved that MacBook Pro early 2011.

I miss that, being able to love the elegance and power a machine.

Nowadays I mostly use KDE Linux, which I can tolerate. Things occasionally break, stuff sometimes doesn't work as intended, but I can generally work around these quirks, so they don't bother me too much. Today, off the top of my head, Firefox crashed, Thunderbird couldn't display one folder, Zotero had to be re-installed, one monitor had to be power-cycled when waking the computer, and the laptop battery died way too quickly again.

I also use a Windows Surface tablet, which is a wonderful little device. It currently shows an error message every time it boots that I can't get rid of, has to be rebooted every few days to stay fast, and sometimes just won't react until restarted. And it always wants to install some update, and just refuses to do it overnight as intended. But generally it works pretty well.

And yesterday, in a bout of nostalgia, I opened my macOS laptop again and endeavored to write a bit on it. There were no updates, then there were updates but they couldn't be installed, then they could be installed, then the download stalled, and hat to be un-paused for some reason. Then my project files randomly couldn't be downloaded to the laptop, and the window management in macOS is a horrid mess. Also the update took a full hour, and didn't change anything noteworthy. I didn't end up actually getting to write on it.

In the evening, my camera corrupted its file system, as it sometimes does if I forget to format the SD card as FAT instead of its default exFAT. The recovery process was tedious but worked. I've done it a few times now and I know what to do.

My phone, meanwhile, can't use its fingerprint reader any longer ever since I "up"graded to Android 10. It's a Pixel 2, as google as Google can be. And apps are sometimes unresponsive and have to be restarted to become usable again. And it still can't undo text edits in the year 2020, and probably spies on my every move.

My amplifier nominally supports Bluetooth, but it only works some of the time. Currently it doesn't, so we use a portable Bluetooth speaker instead. Its Spotify integration has never worked even once. But at least it now reliably sends an image to the projector ever since we vowed to only ever touch the one reliable HDMI port on the front of it.

And as much as I love my ebook reader, it sometimes crashes when there's too much stuff on the page. One time I even had to completely reset it to get it to work again. And every time I install an update, the visual layout changes. Most of the time, mostly for the better. But it just erodes trust so much, to be at the mercy of the whims of an obviously demented software designer somewhere.

As I said, I miss my Mac, back when technology was magical, and just worked. And any error could be clearly explained as user error instead of terrible programming. When I was young and foolish, in other words.

I should go play with my child now, and run around in the yard, or maybe read a paper book. There's no joy in technology any more.

Tags: computers macos
29 Dec 2019

Efficiency is not Intuitive

A few days ago, a new version of Darktable was released. Darktable is an Open Source RAW image processor, and one of my favorite pieces of software of all time. It may be THE most powerful RAW processor on the market, far exceeding its proprietary brothers. But in the social media discussions following the release, it got dismissed for being unintuitive.

Which is silly, ironic, sad; why should Intuition be a measure of quality for professional software? Because in contrast to a transient app on your phone, RAW processing is a complex task that requires months of study to do well. Like most nontrivial things, it is fundamentally not an intuitive process. In fact, most of my favorite tools are like this: Efficient, but not Intuitive; like Darktable, Scientific Python, the Unix Command Line, Emacs.

These tools empower me to do my job quickly and directly. They provide sharp tools for intelligent users, which can be composed easily and infinitely. Their power derives from clean metaphors and clear semantics, instead of "intuitive", "smart", magics. They require learning to use well, but reward that effort with efficiency. They are professional, and treat their users as adults.

In my career as a software developer, I know these ideas as traits of good APIs: Orthogonal, minimal, composable. APIs are great not when they are as expansive and convenient as possible, but when they are most constrained and recombinable. In other words, "don't optimize for stupid". That's what I understand as good design in both programming, and in professional UIs.

Which is quite the opposite of Intuitive. Quite the opposite of "easy to grasp without reading the manual". That's a description of a video game, not a professional tool. As professionals, we should seek and build empowering, efficient tools for adults, not intuitive games maskerading as professional software.

Tags: computers ui
09 Dec 2014

Choosing a German Email Provider

Why not just use Gmail1? Because if you're not paying for the service, you are the service. Google gives you Gmail for free, and in return they analyze and market your data. That's a fair deal, as far as I am concerned. But not a deal I want to make.

For one thing, I just don't like the idea of someone else reading through my email. Furthermore, I'd like my emails to be hosted in the same country I live in2, so that all the laws that protect the privacy of my letters are just as valid for my emails. So a German email provider it is.

First, I tried mail.de. They show ads, even if you pay them. Also, their web interface is surprisingly ugly. No thank you.

Then, I found posteo.de, and they seemed to be doing everything right! Hosted in Germany, a buck a month, with a focus on security and privacy, and seemingly developed by friendly people. They even try to be environmentally friendly. In summary: perfect!

After half a year of using Posteo, though, I have found a few niggles. The web interface can't search mailboxes with a lot of emails–I have an archive directory with 14k emails and the search wouldn't work. Their support says that's because there are too many messages in that directory. Also, they have had a bit too much down time for my taste: I experienced three outages of about two hours each in the last half year. Not a deal breaker, but it doesn't exactly instill confidence in their infrastructure.

Enter mailbox.org. Also German, also a buck a month, also friendly and safe and eco-conscious. But with a much nicer web interface, support for custom domains, and a working search box. They even support ActiveSync, which you'll like if you use Outlook or Windows Phone. In other words, Posteo done right. I'm sold.

Footnotes:

1

Or outlook.com, which is the same deal from Microsoft. Use this if you don't like Google's "priority" inbox and wonky IMAP support.

2

For the record, Microsoft has made a point of having servers in each user's country and abiding by that country's law.

Tags: computers
28 Mar 2009

Synchronisieren von Google, Äpfeln, Fenstern und Telefonen

cloud.jpg

Ich hatte lange Zeit zwei Computer, einen Desktop und einen Laptop, jeweils mit verschiedenen Betriebssystemen und Datensätzen. Um dennoch immer mit den selben Daten arbeiten zu können, verwendete ich eine externe Festplatte. Obwohl sehr low-tech, funktionierte diese Lösung absolut tadellos: Meine Linux-Kisten mounteten die Festplatte automatisch in ihr jeweiliges home-Verzeichnis und so konnte ich auf verschiedenen Computern arbeiten, ohne mich um die Synchronizität der Daten kümmern zu müssen.

Fast-forward ein Jahr, tausche Linux gegen Apple und finde es jetzt doch sehr anstrengend, immer eine externe Festplatte mit mir herumzuschleppen – Apple-Snob, der ich bin. Sieht auch unelegant aus, dieses schwarze Kästchen an den hübschen Laptop zu klemmen. Es muss also eine andere Lösung her, um immer auf beiden Rechnern mit aktuellen Dateien arbeiten zu können. Es bietet sich an: MobileMe (damals noch .Mac), genauer, die iDisk, also ein Stück online-Speicher bei Apple, auf dem man von mehreren (Apple-) Rechnern aus arbeiten kann. Das Angebot ist verlockend, aber leider erfüllt MobileMe meine Erwartungen nicht, es gehen Daten verloren und ich ärgere mich, jemals Geld für diesen Dreck ausgegeben zu haben.

Eine Alternative finde ich in Dropbox, welches die Vision des immer synchronen Datenspeichers "in the cloud" endlich erfüllt, wenn auch als Ordner und nicht als Laufwerk. Inzwischen erbringt auch Syncplicity diese Leistung, wenn auch mit einem eigenen Set an Einschränkungen (Es ist aber noch Beta, also kein Grund zur Sorge).

Dennoch: ganz zufrieden bin ich nicht, einfach, weil zwei Rechner immer eine gewisse Menge "mental overhead" bedeuten. Allein, nicht immer am selben Gerät zu sitzen stellt einfach eine Irritation dar, die im Grunde nicht nötig ist. Na gut, und dieses neue "Unibody"-MacBook Pro ist einfach sexy. Also, tausche iMac + MacBook gegen MacBook Pro. Das löst -logisch- auch alle Synchronizitätsprobleme.

Aber ich wäre nicht der Sohn meines Vaters, wenn ich nicht immer noch mehr technischen Schnickschnack haben müsste, enter: the iPhone. Dank Apple und iTunes ist es natürlich kein Problem, Kalender, Email, Kontakte und Musik immer synchronisiert zu halten; Kabel reinstecken, iTunes machen lassen und fertig. Perfektioniert wird das alles aber erst durch Beihilfe von Google, welches durch ActiveSync (sprich: Exchange) nun auch alle meine Kontakte, Kalendereinträge, Emails und Dokumente auf allen Geräten zur Verfügung stellt. Dieses Setup ist nun endlich wirklich extrem zufriedenstellend. Es ist zwar ein Haufen Kleinkram, den man durcharbeiten muss, bis man das alles richtig konfiguriert hat, aber hat man das einmal getan funktioniert es wirklich tadellos! Und Syncplicity und Dropbox laufen auf dem Rechner einfach nur noch als Backup weiter.

Tags: computers
19 Mar 2009

Western Digital Festplatten-Umtausch mit Überraschungen

festplatte.jpg

Ich hatte mir vor einer Weile eine externe Festplatte nur für TimeMachine gekauft. Denn TimeMachine ist super, die einzige Backup-Lösung die ganz bewusst nie in Erscheinung tritt es sei denn man braucht sie. TimeMachine läuft leise und unbeachtet im Hintergrund, und gibt mir dieses flauschige Gefühl von Sicherheit, quasi das unsicht- und spürbare Kondom der Computerwelt (bzw. Apple-Welt).

…Bis ich einmal den fatalen Fehler beging, meine externe Festplatte HOCHZUHEBEN. Nicht ruckartig, nicht gewaltsam, sondern tatsächlich sehr sanft, aber wohl gerade zu einem ungünstigen Zeitpunkt, denn die Festplatte gab ein leises Klick von sich und hörte auf zu funktionieren.

Es handelt sich hierbei um eine externe Festplatte der Marke Western Digital MyBook mit 500 Gigabytes, gekauft bei Norskit. Ein Anruf beim Verkäufer ergab, dass dies eine alte Bestellung (weltbewegende 9 Monate) sei und ich daher bei einer anderen Nummer anrufen sollte. Eine sehr freundliche Mitarbeiterin teilte mir dort mit, dass die Firma leider Insolvent sei und ich meine Supportanfrage daher an den Hersteller richten sollte. Und was soll ich sagen? EIN GLÜCK, dass sie das sagte!

Denn auf der Webseite von Western Digital gibt es nicht nur eine Support-Telefonnummer, sondern gleich ein komplettes austausch-Programm für Festplatten, welches vollkommen automatisiert abläuft: Man gibt die Modellnummer seiner Festplatte und -für Notfälle- seine Email-Adresse an und bekommt sofort eine neue Festplatte zugeschickt, mit der einzigen Auflage, die alte, defekte Platte innerhalb von 30 Tagen einzuschicken. Und tatsächlich wurde die neue Platte prompt am nächsten Tag verschickt und kam heute pünktlich per UPS bei mir an.

Faszinierend, so wünsche ich mir Kundenservice. Der einzige Wermutstropfen ist, dass Western Digital offenbar nicht sehr überzeugt von der Ausfallsicherheit seiner Festplatten ist, wenn sie so viel Infrastruktur für den einfachen Austausch bereitstellen… Na egal, ich will mal nicht meckern ;-)

Nachschlag: Es stellt sich heraus, die neue Festplatte unterscheidet sich ein wenig von der alten: Sie ist silber statt schwarz und hat zwei Firewire 800 Anschlüsse anstatt Firewire 400 – ein Glück, dass auch Kabel mitgeliefert wurden, sonst könnte ich sie jetzt nicht anschließen!

Tags: computers
Other posts
Other posts
Creative Commons License
bastibe.de by Bastian Bechtold is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.