Time moves too quickly

I have many things to blog about and ideas in my queue - a few half-written drafts and one almost finished final copy. But none of that is important right now.

I'm incredibly late in finding this out, perhaps a side-effect of spending the last two months of my life packing to move to a much better, happier place than where I live now. It was reason enough to drop off of some of the places I frequent, I thought, because it's not like the same people won't be there when I have finished my move.

But I've just learned that Telsa Gwynne has died. Possibly not a whole lot of people who read my blog will know her, but she was quite active in the open-source community when I was growing up. I loved to read her "diary entries", what today we would call a blog. It was during a re-read of her diary last year that I became inspired to create this blog, and indeed, my musing posts are basically my own version of it. It is directly as a result of her writing that this blog exists, and that I have been able to help others whether it be in FreeBSD, Gentoo, Python, or elsewhere. I am grateful to her and I am quite sorry that I never was able to tell her about it. I had considered it, at one time, but felt it would be silly, especially since I am not very well known yet outside of some FreeBSD circles. Who am I to bother someone whom I greatly respect, with a silly story of a small blog? Regret does not begin to express the emotions I feel from not telling her anyway.

Fare thee well, Telsa. I did not know you personally, but your writing style would betray that in my heart and mind, and those of many like me. You may have been removed from the open source communities you were once a large part of, but your legacy lives on, and will always live on. Your perseverance inspired me. And I give all of my deepest condolences to your family and friends, who will surely miss you more than I ever could.

Web browsers, music players, workarounds, and PulseAudio

As security researchers have discovered yet another horrible security bug in Chrome, and Google yet again decides to put off fixing it, I decided to finally give up Chrome entirely. I had dwindled down my usage of it from primary browser in 2009; to secondary browser for Flash and videos in 2013; and finally using it solely for streaming Google Play Music and Spotify, along with the occasional site compatibility test for my work, in 2015. Firefox's inspector tools and Firebug are good enough, and I have a Mac running Safari if I need a WebKit test, so I decided it was no longer important to test on Chrome. That left the issue of music streaming.

Can't handle it, can't handle it

Google Play Music, however, has a fatal flaw. It is a mess of terrible "one page" JavaScript. After only a few hours of music streaming, it had already leaked 150 MB(!) worth of orphan DOM nodes, and 282 MB(!!!!) worth of uncollectable JavaScript objects. This basically means it created buttons, links, and so on, and didn't properly remove them when it was done, so that memory is leaked out and I would have to restart Firefox to get that memory back. Restarting Firefox multiple times a day is not an option for me.

What's worse is that one of Firefox's best and most unknown features was also making my life worse. Every 10 seconds, it scans its memory to see if any of it can be reclaimed, to make sure that it does not use too much memory. Since Google Play Music's interface had leaked so much memory, the scan was taking about 2 seconds - during which the browser became completely unresponsive. That means that for about 20% of the entire time it was open, it was unresponsive (frozen, locked, etc), all because Google has no idea how to write JavaScript.

My mother (bless her soul, she's openly embraced Debian) suggested I try Rekonq, but it could not even load Google Play Music's user interface. I also tried Opera Classic (pre-Blink), and it too could not load Google Play Music. At this point I am very upset at Google; why did you write such a cluster#*$@ of terrible code instead of writing a simple multi-page player like YouTube? YouTube does not suffer from any of these issues, and is a Google product!* Anyway, my next goal was to see what I could do for streaming music that did not require a Web browser.

* I am aware that YouTube has a single-page mode, but I found a way to disable it except while using playlists. It works great and does not leak half a gigabyte of memory.

Done, done, and I'm on to the next one

It turns out that Google Play Music has no official API and no non-browser clients. Even Spotify has unofficial ones that are of questionable quality and legality, but Google has done a very good job of making their API so hard to use that nobody bothers to even try with them. (Future project idea.)

Then I realised their Android app is pretty reliable and certainly better than having my browser locked for 20% of the time it's open. However, I still need to be able to hear other things on my computer (if someone links me to a video or presentation, for instance), and I don't want to have to keep flipping back and forth between my phone and desktop.

My work-provided desktop did not come with a sound card (even though we use sound a lot internally...), so I am using a USB Griffin iMic as my sound "card". It works fantastically in Linux/ALSA, but one thing I could not figure out was how to make it play line in as a monitor (i.e. playthrough, listening to line in/mic with headphones/out, whatever you like to call it). Thankfully, I found a very helpful blog post about this very issue, and a solution involving PulseAudio: pactl load-module module-loopback was all it took to listen to crystal-clear, low-latency, glorious Nexus audio on my desktop!

Final thoughts

• While it certainly is great that PulseAudio offers the same great passthrough functionality that OS X had since Jaguar (and lost in Mavericks), they really need to document PulseAudio modules better.
• Google needs to rethink making their music player in one page JavaScript. A native app would be amazing and make me a much happier catfox.
• It just feels like... if Google hadn't royally screwed up Chrome, and they hadn't royally screwed up their music player, then hours of my life would have been saved because then I would not have had to learn how to monitor line in within Linux. It was interesting learning all this, but I still have this feeling that it should be entirely unnecessary, and like this is a very unclean workaround for what amounts to "Google is terrible at writing code".

Oh well, at least Android 6.0 is good. (For now.)

The Joys of Unix Programming: MAP_ANON(YMOUS)

I was trying to do a little late-night hacking last night on SuperGameHerm, the Game Boy emulator my friends and I are writing, and I hit an error in the memory mapper. Specifically, certain OSes that used to be named after cats don't like calling mmap on /dev/zero (neither does Android). I thought it was odd that it was falling back to that code in the first place, though, because Apple's Mac OS X — I mean, certain cat themed OS — has always supported MAP_ANON, and I confirmed that by going to man mmap on a Mac.

What was going on? I dug deeper and saw MAP_ANON was guarded in sys/mman.h, so CMake wasn't finding it and it was instead compiling our fallback code. And so I started digging up other issues related to big endian machines and realised that I had only tested OS X and FreeBSD on big endian, and never tested OS X or FreeBSD on little endian. So was my big mistake.

This is a comprehensive guide to How to Make MAP_ANON(YMOUS) Visible, for every OS I could find information on:

Mac OS X

On 10.3 and below, this is easy; it's always there! It is not guarded by any #ifdef.

On 10.5 and above, it is slighly harder; you must define _DARWIN_C_SOURCE to cause MAP_ANON to be visible in a public scope.

On 10.4 only, it's much harder! It is only protected by #ifndef _POSIX_C_SOURCE, so to use MAP_ANON against the 10.4 SDK, you must completely undefine _POSIX_C_SOURCE. You don't have any other choice.

I suppose that means my overall advice then is to use the 10.5 SDK no matter what, if you have a Leopard computer handy, because it can target as low as 10.0. Otherwise, use the Panther SDK included with Tiger's Xcode Tools. Don't ever use Tiger's SDK if you want MAP_ANON.

FreeBSD

Before 5.0, it's always visible, just as in OS X 10.3. There are no preprocessor options to show or hide MAP_ANON.

On 5.0 or above, the only way to cause MAP_ANON to be visible is to define __BSD_VISIBLE somewhere. Undefining _POSIX_C_SOURCE won't save you here.

Other BSDs (NetBSD, OpenBSD, DragonFly BSD)

It's never guarded. MAP_ANON is always available.

Solaris

I could only get my hands on OpenSolaris, but considering the header having a copyright date of 1989 (by AT&T), I can't imagine it's any different on Real Solaris (or Oracle Solaris). There are no guards here, either; that's to be expected since they invented the damn thing.

Linux

glibc: I don't understand /usr/include/bits in the slightest. It seems to be always available no matter what options I toss to clang, but it is guarded by.. __USE_MISC? I presume this is some sort of feature macro buried deep in glibc that I don't care about or understand.

musl: It's always available, at least on 1.1.11 which is what I have on my test box.

Android: After searching through their spaghetti of includes to get to the actual file that defines constants, it appears they are all completely unguarded, though that isn't surprising since it is Linux and embedded.

In conclusion

Perhaps it's best to avoid anonymous mmap(2) in applications that you want to actually be portable.

Musings: More Python 3 compat, Project Sunrise, InspIRCd modules and Portage

Some good news: as I eix-sync'd this morning, I noticed that dev-python/ndg-httpsclient and dev-python/ipaddress now have Python 3 compatibility. That means two of the packages I had thought had no chance of being upgraded actually have been. As for my own efforts, I have been very busy with work and musl support patches lately, but I have been looking at fixing up the htop package next.

I've found Project Sunrise, a way for me to be able to contribute ebuilds to Gentoo in hopes of someday getting them in the master Portage repository. I'm hoping to add a few Python libraries first, then moving up to packaging SuperGameHerm and PyIRC once they've matured enough to be useable by external users.

While testing PyIRC, I needed to be able to use a few modules that are not a part of InspIRCd's main package. Since Portage didn't allow any way of including them in the installed package, I simply checked out the source code package, ran modulemanager to add the modules, then built only those modules. I copied them to the /usr/lib64/inspircd/modules directory and added them to modules.conf, and voila! Now I can do more IRCv3.2 testing.

Foxtoo: Gentoo + musl C library on 100MHz Pentium laptop

I haven't yet finished up writing all the content I want to write about the process needed to get this working, but I do have a little teaser picture to share:

That is a Compaq LTE 5150 with a 100MHz Pentium CPU and 40 MB EDO RAM running Gentoo Linux! The kernel version is 4.2-rc1 (because I'm an incorrigible ricer), it was all built with GCC 4.9.3 and it is using the venerable musl C library instead of glibc. Boot up takes only about 15 seconds off the 5400 RPM laptop IDE disk, and once booted, the minimal kernel I have + bash use only about 3 MB of the 40 MB total.

It may not seem like a very useful thing to have done, but I had a lot of fun building it up, and I've ended up finding and closing various bugs in everything from procps to the kernel itself. So I feel that not only has this been a fun personal project across two weekends, but it has been productive for the entire community :)

Python 2 -> 3 upgrade: status update

This is a small update on my bringing packages in Gentoo to Python 3.

I haven't had time to contribute as much to this effort as I had hoped, but I have successfully finished with two packages and the patches are now in the hands of upstream maintainers. I've been toying with the musl C library as an alternative to glibc (and I'll be posting about my experiences with that later), which has distracted me a bit from Python 3 work.

app-misc/ca-certificates

Required a bit of effort. Debian #789753 filed. Maintainer seems happy enough with it, but it's not in master yet.

dev-libs/evdev

This was simple enough; libevdev has had upstream support for Python 3 since 2013. Gentoo #553110 filed with a patch to update the ebuild accordingly. No response as of post time.

Musings: SSH key types, PayPal API documentation, and more

I have a few random musings to arf about today.

Sleep is miserable. I don't know why I even bother pretending to have a schedule to myself. Between work demands, personal issues, and the fact that I tend to favour going to sleep around the 05:00 hour naturally, it's impossible. Sure, I can hold a "normal person" schedule indefinitely... if I have no external factors. But my life is full of external factors that make it impossible. And there are some people in my life that try and make me feel guilty for not being able to hold a schedule. It feels miserable.

In other news, PayPal's API documentation for their SetExpressCheckout call lies. They say the xsi:type for the SolutionType is "ebl:SolutionTypeType", but I found out the hard way that passing that as the type causes a SOAP Fault (and leaks the API password and signature out in the error message)! The only way I can find to do it properly is to set xmlns="urn:ebay:apis:eBLBaseComponents" on the SolutionType node and then set xsi:type="SolutionTypeType" (no ebl: namespace). Then the API accepts it fine. Who knows why their systems do what they do.

I investigated making a new SSH key that would be stronger than my current one. Unfortunately, it doesn't seem I really have a choice in the matter. The only common denominator is RSA, as shown in this matrix:

Key Type	Mac OS X	NetBSD	Debian
RSA	Yes	Yes	Yes
DSA	Yes	Yes	Yes
ed25519	No	No	No
ECDSA	No	Yes	Yes
SSH Version / OS Version	5.6 / 10.8.5	5.9 / 6.1.5/i386.	6.0 / 8.0 "Jessie".

Note that the following systems supported all listed key types:

• Gentoo 20150623
• FreeBSD 10.1
• OpenBSD 5.6
• Alpine Linux 3.2
• Windows 2000

Truly a sad day when a 16 year old Windows OS has more SSH key types available (via mingw) than Mac OS X, NetBSD, and Debian combined. Looks like I'm sticking with RSA keys for the foreseeable future.

Removing Python 2.7 from Gentoo, one port at a time.

I'm highly motivated, as a Pythonista who loves Python 3 and all of the new features, syntax, and improvements, to help the rest of the community upgrade to Python 3 and enjoy all of its benefits.

To that end, I would like to end the dependence of Python 2 in the general community. Since I feel I am a fairly average power user of Gentoo, I figured I would start with the packages on my own system, and this is what I found:

  dev-lang/python-2.7.10 pulled in by:
    app-emulation/qemu-2.3.0-r2
    app-emulation/virtualbox-4.3.28
    app-misc/ca-certificates-20141019.3.19
    app-text/gnome-doc-utils-0.20.10-r1
    dev-lang/spidermonkey-1.8.5-r4
    dev-lang/spidermonkey-17.0.0-r3
    dev-lang/spidermonkey-24.2.0-r2
    dev-lang/yasm-1.3.0
    dev-libs/glib-2.44.1
    dev-libs/gobject-introspection-1.44.0
    dev-libs/libevdev-1.4
    dev-libs/libgamin-0.1.10-r5
    dev-libs/libnatspec-0.2.6-r1
    dev-libs/libxslt-1.1.28-r4
    dev-libs/protobuf-2.6.1-r3
    dev-libs/zziplib-0.13.62
    dev-python/google-apputils-0.4.0
    dev-python/ipaddress-1.0.7
    dev-python/librsvg-python-2.32.0-r1
    dev-python/m2crypto-0.22.3-r3
    dev-python/ndg-httpsclient-0.3.2
    dev-python/pygobject-2.28.6-r55
    dev-python/pygoocanvas-0.14.1-r1
    dev-python/python-gflags-2.0
    dev-python/twisted-core-15.2.1
    dev-python/wxpython-3.0.2.0
    dev-util/boost-build-1.56.0
    dev-util/scons-2.3.4
    dev-vcs/git-2.4.3
    dev-vcs/mercurial-3.4.1
    games-emulation/m64py-0.2.1-r1
    gnome-base/gconf-3.2.6-r3
    gnome-base/libglade-2.6.4-r2
    kde-apps/kajongg-4.14.3
    kde-base/krosspython-4.14.3
    kde-base/plasma-workspace-4.11.20
    media-gfx/gimp-2.8.14
    media-libs/alsa-lib-1.0.29
    media-libs/avidemux-plugins-2.6.8
    media-libs/libcaca-0.99_beta19
    media-libs/libgpod-0.8.3
    media-video/openshot-1.4.3
    net-analyzer/nmap-6.49_beta1
    net-dns/avahi-0.6.31-r7
    net-libs/farstream-0.1.2-r2
    net-libs/gupnp-0.20.14
    net-libs/libproxy-0.4.11-r2
    net-print/cups-2.0.2-r2
    net-wireless/crda-3.18
    sys-apps/usbutils-008-r1
    sys-devel/llvm-3.5.2
    sys-process/audit-2.4.1-r1
    sys-process/htop-1.0.3
    www-client/firefox-38.0.1
    x11-libs/xpyb-1.3.1-r3
    x11-plugins/purple-plugin_pack-2.7.0-r1

Not terrible for a world that has 1,277 packages. Now, we can wipe some of those off:

dev-lang/spidermonkey, www-client/firefox

The Mozilla build system...

dev-libs/protobuf, dev-python/google-apputils, dev-python/ipaddress, dev-python/python-gflags-2.0

I can't make Google care about Python 3. I'm just a lone fox.

dev-python/ndg-httpsclient

This is apparently only pulled in because I have 2.7 compatibility enabled on other packages, and it also appears abandoned upstream.

dev-python/m2crypto

All but deprecated upstream. dev-python/cryptography pretty cleanly replaces it. I simply need to change dependencies to use that instead.

dev-python/twisted-*

The Twisted project is large, and already has a Python 3 objective. No need to worry about that until later, since they have others working on it.

dev-python/wxpython

They, too, are working on an official Python 3 port already. I've even used it; though it is ridiculously unstable, there has been a lot of progress made.

dev-util/scons

As of August 2014, they are officially porting to Python 3. Same as Twisted, then.

dev-vcs/mercurial

This is a very large codebase with a lot of issues related to 2->3. Not something one fox can help with either, I'm afraid.

any gnome package

Not really interested in contributing to the ever-dying and unmaintained GTK 2, and definitely not interested in contributing to the abomination of GTK 3.

KDE 4

As KDE 5 nears stability, this will become less and less maintained, so it probably isn't worth it either.

net-print/cups, sys-devel/llvm

I doubt there is any hope of getting Python 3 code into Apple projects, since they seem to enjoy clinging to 1990-era development tech :(

After doing some sorting to show the order I'd like to work on these, this leaves us with:

    app-misc/ca-certificates-20141019.3.19
    app-emulation/virtualbox-4.3.28
    dev-libs/libevdev-1.4
    media-libs/libcaca-0.99_beta19
    media-libs/libgpod-0.8.3
    media-libs/alsa-lib-1.0.29
    dev-libs/libxslt-1.1.28-r4
    net-wireless/crda-3.18
    dev-vcs/git-2.4.3
    sys-apps/usbutils-008-r1
    dev-lang/yasm-1.3.0
    x11-plugins/purple-plugin_pack-2.7.0-r1
    sys-process/audit-2.4.1-r1
    net-dns/avahi-0.6.31-r7
    sys-process/htop-1.0.3
    games-emulation/m64py-0.2.1-r1
    net-libs/libproxy-0.4.11-r2
    dev-libs/zziplib-0.13.62
    dev-libs/libnatspec-0.2.6-r1
    app-emulation/qemu-2.3.0-r2
    net-analyzer/nmap-6.49_beta1
    dev-util/boost-build-1.56.0
    media-video/openshot-1.4.3
    media-libs/avidemux-plugins-2.6.8
    x11-libs/xpyb-1.3.1-r3

I would really like to do at least one per week, but there's no guarantees of course. Wish me luck!

Linux 4.1 kernel: Finally, a movement in the upwards direction

No musings yesterday because I was having quite a time getting Linux to play well. I spent yesterday configuring the rest of the kernel until around 21:00, when I started searching for and applying all the patches I wanted. Found a small bug in the CPU optimisation patch that Gentoo uses (via grsecurity); namely, it doesn't enable P6_NOP for anything higher than a Core 2. I manually edited the patch and applied it.

I built the kernel, ran # emerge @module-rebuild, signed the newly built modules, and rebooted in to my new system... Or I would have, if it had been able to read the init RAM disk. Doing the Apple EFI dance to switch between 3.18 to rebuild and 4.1 to test was not my idea of a good time, but after a while, I found the issue. There's something broken somewhere and it would not read XZ-compressed ones. I used my fallback algorithm, LZO (chosen because it's free and very fast), and it booted right up. Only now, I couldn't start KDE.

Typically, I start KDE by using $ X_SESSION=KDE-4 startx; I don't use KDM or XDM because I regularly test breaking changes to i915 that may cause X to hardlock, rendering my laptop useless. This time, however, it complained 'xterm: command not found'. Unsure why it was trying to load xterm, I checked around and I found no answer. I still haven't figured this out. The workaround I'm using is $ XINITRC=/etc/X11/Sessions/KDE-4 xinit which acts the same as the old command, only it's longer and less readable.

Then I had a world of i915 bugs that I will omit from my blog mainly because they were mostly configuration error and the ones that weren't were resolved by re-merging media-libs/mesa.

After all of that was over, I checked on my wireless chip. Lo and behold, for the first time since I've started using draft-n wireless networking on Linux in 2009, it was actually associating successfully to a 5GHz 802.11n AP! The speed is only 150mbit/s; the chip supports 300mbit, so I am not sure why that is, but I am still happy to be finally untethered from my Ethernet cord!

Some benchmarks.

Running emerge -1 openssl

Kernel / Sched	Wall Time	System Time
3.18 / CFQ	5m 3s	59s
4.1 / CFQ	4m 57s	55s
4.1 / BFQ	4m 53s	52s

That's a pretty nice win for BFQ, and a huge improvement over 3.18. Note this is all on the same hardware, on a fresh boot, with nothing in fscache.

Playing 720p MPEG-4 video in VLC

Kernel	On-Die Temperature	CPU % used
3.18	71° C	31.2%
4.1	63° C	46.3%

Each measurement was taken at 2 minutes into playing the video. Not sure why the CPU's a bit more active, but wow, that temperature reduction is serious!

Overall, I am very, very happy with this upgrade. The Linux 4 series seems to be all about making things smoother, faster, and more battery-friendly. I'll update later with a benchmark of battery life.

If you have a MacBook Pro from the 2011 era (or you're just curious), you can view my config file online. Special thanks to Elly and Horst for guidance, patchsets, and keeping me company while I was going insane with menuconfig. :)

Musings: Just another lazy Sunday.

Okay, I guess I'm doing this daily. Sundays are usually pretty lazy for me, so this will be short.

Found more interesting tidbits, still configuring the 4.1 kernel for my laptop. RC8 came out before I even finished, so now I have some oldconfiging to do.

Still felt dizzy from yesterday, so spent 20 minutes on the treadmill. Blew all the cobwebs out of my blood, I suppose. Almost completely better afterwards, so that's a good thing.

Found a really intricate resource pack for Minecraft, S&K Photo Realism. I'll have to try it out the next time I'm on the desktop with the Radeon 5700HD. My laptop can't handle resource packs; the poor little Sandy Bridge HD Graphics is too overwhelmed.

Helped Horst debug an i915_drm driver issue on 4.0.5. Not sure the root cause, will have to probe further tomorrow.

Started planning how to install Foxtoo, my own little brand of Gentoo, on to my Pentium-100 laptop. I think I have a way and it's going to make for quite an article if I manage to make it happen.

Actually excited for work tomorrow. Lots of cool things in the works there.

Musings: libxo, Python code quality, and Linux kernel shenanigans

I've been inspired to actually make this in to an actual bloggy journal thing instead of just writing occasional long-winded well thought out articles. I still want to do long-winded well thought out articles, but the entire reason I got this thing in the first place is to share my knowledge and experiences with the world. Might as well.

Caught up on the backlog of happenings on the freebsd-current mailing list. Lots of people are upset about libxo, including myself. May mean a Linux migration in my professional future. Disappointing, especially since there is a lot of opposition, but it's understandable that it's going forward anyway. Juniper pays to keep the lights on at the FreeBSD foundation, so they have a lot of pull. Sad to see they don't really care about the community or what we have to say.

Looked harder at GCC 5.1 code output. Then looked harder at switching default compiler from GCC to Clang on Gentoo. ;) It's not all there yet but it seems to be improving every day. Looking forward to trying out a Clang-compiled kernel if that patchset advances.

Played with Pylint and Pyflakes. Ran it on a few of the projects I work on and a few of the ones I write with Elly. PyIRC 3 got a rating of 8.5 out of 10. Really impressive. I hope to show Pylint to the people at work in the coming week and improve the quality and process there too.

Tried playing Minecraft. Already felt a bit dizzy before playing, so that was a mistake. Logged off after about 20 minutes from the ill feeling. Perhaps tomorrow.

Found out about a lovely years-old DoS vulnerability in the Linux kernel's UDP stack. Ran netstat on my servers to make sure they had no UDP listeners. Not sure what to do about the Minecraft server, perhaps the DDoS protection we already have on it is good enough.

Read a hilarious article comparing the Pentium 4 Northwood to various new CPUs, where the AMD "accelerated" notebook CPUs performed worse. I can't believe AMD has gone so downhill that a Pentium 4 from 2002 can spank its brand new designs.

Decided to start configuring a 4.1-rc7 kernel for my laptop. 3.18 is so old (by my own standards) and I really want to get off of this line since my checkout is still affected by that stupid exit race bug which sometimes causes Firefox to die. Found an interesting knob, CONFIG_EXT4_ENCRPYTION, which is apparently the same encryption stuff Android "M" will be using. Pretty neat but seems useless for my needs. Might try it on the Pentium II for the hilarity.

Tried to tinker with the internals of OS X Server 5.0 beta 1 before bed, but it's compressed with a weird algorithm (ADC?) that none of my Linux utilities can break open. I'll need to investigate that more tomorrow.

FreeBSD on Apple MacBook Pro 8,2: Epilogue.

It is with a fairly heavy heart that I write I am no longer running FreeBSD on my MacBook Pro.

What happened to improving?

Part of the problem is that I finally received gainful employment in March, and that work is almost impossible to do on FreeBSD. A lot of it involves Chrome (which I still have been unable to run on FreeBSD), Qt5-based applications (which crash due to known bugs in libv8 that Google do not care to resolve), and some Python libraries that have truly terrible performance on FreeBSD.

Why not run Linux in a VM for work?

Sure, I could have, if VirtualBox ever worked...

Weren't you excited to fix up FreeBSD?

I was. I still am, but something just feels different. For over a decade, FreeBSD has for me been the go-to operating system for any use case: servers, embedded projects, desktop systems, and more. But the current heading of development seems to strongly suggest this is no longer encouraged or desired.

When I first started out with Gentoo nine years ago, they were pretty much bent on making it for newer hardware only. Back then, Pentium computers were like the Pentium 4s of now - something you give your grandma or little sister for web browsing, but nothing too serious. And Gentoo developers did not really care if they broke compatibility with these older systems. I can understand that, given that compiling the entire system by hand is something that is pretty taxing for older hardware.

The nice thing about FreeBSD was their community never looked down on you for using these older machines, and realised they still have use. My first interactions with #FreeBSDHelp on EFnet were in 2006 and related to getting SLIP support working in sysinstall so I could remotely install FreeBSD 6 on my Pentium 90 laptop. They were happy to help.

The roles have largely reversed now. Running into issues with older hardware get me looks of disdain and "great, upgrade your hardware and try again" from the FreeBSD community. Meanwhile, the Gentoo team was happy to help me with an issue regarding my retro Intel486 box, in 2015. This computer has no business still functioning, and they were still willing to help me configure a kernel that would boot on it with its anaemic 20 MB RAM.

The other thing I have noticed is that even now, months later, none of my Ports bugs have been handled. In the same amount of time, I have filed three bugs against Portage packages... and all of them were closed within one week of being opened. I feel like my contributions matter to the Gentoo Linux team.

What have you learned?

FreeBSD is more fun to hack on than Gentoo. FreeBSD is harder to get things done on than Gentoo.

FreeBSD is lighter on resources than Gentoo. FreeBSD is heavier on bug backlog than Gentoo.

Ports updates for FreeBSD: Minecraft, HFS reading, and more…

I have been spending my time lately doing some maintenance with Ports, updating and fixing the software packages I use. I've submitted a couple of PRs to the FreeBSD team and hopefully my changes will be merged very soon.

Minecraft (games/minecraft-client)

The minecraft-runtime script wasn't working, and the JAVA_VERSION was, in my opinion incorrectly, bumped to 1.8 when it runs perfectly well on 1.7. I have submitted PR 197063 to rectify these issues, and additionally more correctly replace variables in the script templates.

HFSExplorer (sysutils/hfsexplorer)

The current version in Ports no longer exists on SourceForge, as they have moved to GitHub. With the move came a new release that brings some new features and enhanced compatibility. I have submitted PR 197209 to update the distfile location and version.

LWJGL (games/lwjgl)

This library, used by Minecraft, does not currently build with openjdk8. I am currently ironing out this bug so that I can submit a patch either upstream or to FreeBSD (or perhaps both), whichever one is necessary. I'll note further progress on this here on my blog.

Symbola (x11-fonts/symbola)

Elly has submitted PR 197090 to add the Symbola Unicode 7.0 font to FreeBSD. Look for this coming soon.

An article about more MacBook stuff is coming shortly, but for now I'm still working out some issues. Until then, happy hacking everyone!

FreeBSD on Apple MacBook Pro 8,2: Now with more features

As an update to my last post, I have been able to make more of the features of my 13" Late 2011 MacBook Pro work on FreeBSD 11.0-CURRENT.

Sensor Readings.

Simple: kldload coretemp. To have on boot, add coretemp_load="YES" in /boot/loader.conf.

Cool Temperatures and Power Saving.

You will need to ensure i915kms is loaded and add drm.i915.enable_rc6=7 to your /boot/loader.conf. This was pointed out by the FreeBSD wiki which I could not find, so thank you Elly for helping me find these docs.

Additionally, you can add hw.pci.do_power_nodriver=3 to power down unused PCI cards. They re-enable when you kldload the modules, so this will help if you do not use the SD card reader or FireWire port very much.

Touchpad Gestures.

The atp(4) driver is flaky, but does work. Just add atp_load="YES" to /boot/loader.conf. Two fingers = right-click, three fingers = middle-click. Three finger tap is somehow hilariously much more reliable than two fingers, but with a little persistence they work. Two-finger scroll is a bit wonky and scrolls at unpredictable speeds, but I have a feeling this is a sysctl that may be fixable.

Bluetooth.

This actually does work right out of the box. I can pair with my phone and use the nifty obexapp utility to transfer files to my Android phone, an HTC One (M8). Unfortunately I was unable to transfer any files or pictures from the phone; obexapp consistently reported "0 bytes streamed in 2 seconds". I think it may be a permissions error as I able to upload files to my phone's Download/ directory correctly but no other directory.

To establish a Bluetooth connection (with an Android phone at least), you have to do the following:

1. Add your phone to /etc/bluetooth/hcsecd.conf. The examples are quite good. Remember your PIN.
2. Enable hcsecd in /etc/rc.conf and set flags to ubt0.
3. Enable sdpd in /etc/rc.conf.
   Your /etc/rc.conf should now have at least the following lines:

   hcsecd_enable="YES"
   hcsecd_flags="ubt0"
   sdpd_enable="YES"
   

4. Start both services...
   

   /etc/rc.d/hcsecd start
   /etc/rc.d/sdpd start
   

5. Now run obexapp -a phone:bluetooth:mac -C FTRN. Your phone will ask you for the PIN you specified in /etc/bluetooth/hcsecd.conf. Enter it, and you can see files on your FreeBSD computer!

Some More Bumps.

With all of these things fixed, I have had two more snags. One was that my tilde key (the ~ next to the 1 key) on my MacBook's internal keyboard was not working. This was fixed by adding the following line to ~/.xmodmaprc and running xmodmap ~/.xmodmaprc:

keycode 94 = grave asciitilde

My other new bump is that when I plug in headphones, the sound still comes out of the external speakers (and there is no sound in the headphones). This is probably due to some weirdness in the Apple HDA and I will have to take a closer look at this.

In conclusion, I am still quite happy with this change and am looking forward to fixing the rest of these issues.

FreeBSD on Apple MacBook Pro (13", Late 2011): Initial thoughts

Background.

I rely on my laptop, an Apple MacBook Pro 8,2 (13", Late 2011), for most of my work. For the past year or so I have been using Gentoo Linux on it. Gentoo is probably the best distribution of Linux out there today, but that isn't really saying a lot about it. Underneath the beautiful and easy-to-use Portage system lies the same glibc, the same turmoil over a switch to a less-than-ideal init system, and the same kernel-level bugs that bring my productivity down.

I had considered sticking it out, and possibly even picking up maintaining OpenRC if it is abandoned as some people seem to want. However, I have been having many glibc-related bugs lately:

• Almost all GTK apps keep locking up in waitpid.
• gdb locks up when I try to attach to processes - even after checking /proc/sys/kernel/yama/ptrace_scope and the like (no error, just hang).
• Starting VirtualBox VMs cause the entire machine to lock up for up to two minutes before returning to normal. This isn't from starting the VM either; it occurs before the VM can even begin.

I don't take system-level changes lightly, but it was time to try something else. Trying to rebuild everything against something like musl seemed like an interesting idea, but it still wouldn't solve the eventuality of systemd. Additionally, there are still other compatibility issues using musl. And if I was having to rebuild everything anyway, I might as well try something completely different…

Enter FreeBSD.

I have been playing around with FreeBSD on my UltraSPARC system, as I wrote earlier. It is pretty snappy on there, even though it is a very old and slow system. Therefore, I decided to research about FreeBSD on MacBooks. There are a few requirements I have with a primary OS:

• It absolutely must boot natively via EFI. I do not have the time to wait 90+ seconds for BIOS boot, and some of the hardware in the MacBook won't even perform correctly under that mode; the DVD drive and the Bluetooth are very fickle, at least in my experience.
• It must be able to run a performant virtualisation solution, like VirtualBox, which is my go-to system emulator. I use virtualisation heavily, so this is a requirement.
• It must work with the hardware components I use on my computer. This is primarily Gigabit Ethernet, FireWire, the SD card reader, and the Bluetooth radio.

I had enough disk space free on my Mac OS X partition to resize it down and give FreeBSD a "comfortable" 64 GB. Since this is just for testing, I left my Gentoo install and its LVM PV alone. I resized the Mac OS X partition and moved down the LVM PV to put FreeBSD in the middle (otherwise the space would be oddly divided) using GParted running on the Gentoo install. I was horrified when it, too, was locked in waitpid. I waited over three hours before it finally became responsive again, and hilariously enough, it reported that the operations completed in just 49 minutes.

I created a 300MB HFS+ partition used for the EFI loader, since Apple's firmware really likes having EFI code on HFS+. This also allows it to show up native in Chooser. You can also use FAT16 or FAT32, but I prefer to use HFS+ for this since it is just a single file. The rest of the 64 GB was given to a single FreeBSD UFS partition.

Next, I did something that I absolutely do not recommend for any reason unless you are willing to accept permanent data loss: I ran a Qemu instance with /dev/sda as the virtual hard disk. This allowed me to manipulate the disk directly from Qemu, which was convenient since the FreeBSD UEFI USB install image does not yet support Apple's firmware. However, if I typed even one letter or number wrong, this could have erased my entire drive so please do not do this! I fetched the sets manually from FreeBSD's FTP archive and extracted them to the new FreeBSD UFS partition on my disk. I shut down the Qemu instance after I configured /etc/rc.conf to my liking.

Finally, back in Gentoo, I copied the /boot/boot1.efi file from the FreeBSD UFS partition to the HFS+ partition, using the name boot.efi. The directory tree on the HFS+ partition, to boot natively and show up in Chooser (and Startup Disk in Mac OS X), is as follows:

• /mach_kernel - 0 byte file, must be present to show in Startup Disk.
• /EFI - copy this directory from the ESP. It contains the encrypted firmware used to start the computer's components.
• /System/Library/CoreServices/boot.efi - this is the FreeBSD /boot/boot1.efi file.
• /System/Library/CoreServices/SystemVersion.plist - this is an XML file that you can put the details of your FreeBSD install into to show what version it is and such. This file isn't strictly necessary but adds a very nice touch of professionalism to the install.

Booting the FreeBSD: EFI on the First Try!

I snuck in a /.VolumeIcon.icns file with the FreeBSD Beastie head logo, then rebooted my system. I waved goodbye to Gentoo, but wasn't sure if my FreeBSD installation was fully successful. Lo and behold!

I pressed RETURN and hoped for the best... and the best is what I received!

I was overjoyed to see how fast it boots; under 20 seconds from Chooser to a login prompt. I spent the rest of the night installing binary packages using pkg for software which I could tolerate the default options, but actually installed most things from the Ports system. Xorg support was native and flawless, including DRM for acceleration using my laptop's Intel HD Graphics 3000. The next morning I had the entire KDE 4 base system built and working in just two hours! KWin's compositing looks just as good as it did in Linux, but now uses much less CPU and memory - the entire thing boots up (using KDM4 to start my KDE session) using just 290 MB RAM, and will actually idle at 0.0% CPU (Linux would idle around 1.1-1.5%).

Some Issues.

While FreeBSD appears to be a panacea for everything I had wrong in Gentoo Linux, all is not immediately well. Of course, as with any new system, there are some more things I need to fix and figure out:

WiFi/Bluetooth

These will need some configuring and driver twiddling, but I have high hopes based on what I read on the very useful FreeBSD forums and mailing lists.

Ext2FS crashes

I backed up my system before installing FreeBSD, of course, but my backup disk (along with my primary disk which was unharmed) is formatted as ext4 (Linux FS). FreeBSD has a module to be able to read it, but when I try and access almost any file, it has a kernel panic. I am currently looking in to what is causing this and how to fix it.

Sensor readings

Currently, my laptop thinks it is -273.1 °C. This is quite obviously impossible, so I will need to investigate how to fix KDE's system monitor widget :)

Touchpad

While the FreeBSD kernel has a great touchpad driver merged in to 11.0-CURRENT, and it shows up in my kernel log as I boot up, Xorg only sees my mouse as having one button with no gesture support either. This means no scrolling on the side or with two fingers, no right clicking, no middle click for pasteboards, and so on. This will also need some fixing.

Overall Thoughts.

FreeBSD is very performant on this hardware, and seems to be doing a good job driving it. There are some quirks to it, but this is a new system and the kernel I am running is a beta anyway, so it of course will not be perfect. I am happy with this switch though and probably will not go back to Linux any time soon unless they can sort out their issues. I hope they can, because the Linux kernel itself is actually decent, and has a lot of commercial support which is good for open-source going forward. However, FreeBSD's performance is amazing and gives me renewed hope for the quality of systems that can be made in open-source communities.

Now playing: Devil in a New Dress - Kanye West

Double entendre... FreeBSD's logo is Beastie, and my laptop is a devil of a computer wearing a new dress with FreeBSD.

FreeBSD 10.1 on UltraSPARC Sun Ultra 60: Booting and initial packages

As noted in my previous article, my friend Madeline and I set out yesterday to fix my Sun Ultra 60's boot problems so that it could run FreeBSD. The issue ended up being that the disk alias in OpenBoot had been scrambled (probably from the NVRAM reset I did to force it to boot), and was reading from a SCSI target that did not exist. I fixed this and it instantly booted up FreeBSD!

Now comes the fun part. I would really like to use pkg-ng and avoid building packages on a 360MHz computer if at all possible. Even though the UltraSPARC-II's 360MHz is closer to an Intel's 800-1000MHz, it's still a pretty poor platform for compiling some of the things I plan to use on it. I would like to run things like CMake and the new Clang 3.5 vs the very old, broken GCC 4.2.1 that FreeBSD on sparc64 ships.

I plan on setting up a buildbox on amd64 that will cross-compile ports for sparc64. There is only a single reference I can find for setting up such a system, and the instructions aren't too clear to me, so I will be writing up my experiences. For now, I have built ports-mgmt/pkg from source and it works. Unfortunately FreeBSD runs no mirrors, so I will have the "honour" of setting up my own local pkg mirror. If I'm able to make it work and compile enough useful ports I may make it public.

FreeBSD 10.1 on UltraSPARC Sun Ultra 60: The adventure begins

Today begins a new adventure for me. After trying to like Solaris 8 and finding it too old, and trying to like Solaris 11 and finding it too Enterprisey EnterpriseWare™, and unsuccessfully trying to boot Linux plenty of times, I am going to install FreeBSD 10.1 on my Sun Ultra 60. Some specs...

• UltraSPARC-II 360MHz (supposedly equivalent to a Pentium III, which at the time was huge)
• 512MB RAM (again, at the time it was released, this was more than anyone would ever need)
• 18.6GB Fujitsu SCSI disk, which doesn't give me a whole lot of room but is sufficient (I've run FreeBSD 9 on a P4 with a 20GB disk, with KDE and X, so...)
• Creator 3D UPA graphics - full 1080p, with 3D and 2D acceleration - in 1999.
• OpenBoot 3.31

You may be asking why I would use this computer. I have a few reasons. One is that I love utilising different CPU architectures and computers for what they do best. I'm thinking about playing around with some older graphics demos and seeing what I can get OpenGL/SDL2 to do with this. Considering that at the time it was 10x more powerful than any PC around, it may be enough to get some small-ish demos running at good speed. Another reason is that it's just a fun project, and another big-endian architecture besides PowerPC that I can use for endianness testing. I also am looking forward to porting SuperGameHerm to the SPARC architecture if I am able.

I'll be blogging about using and maintaining this system as I continue to use it, but for now, some installation notes!

Booting the System.

This computer is notoriously fickle to boot. I had to hold the STOP+N keys to clear the NVRAM to be able to see video at all. STOP+D is also a useful key combo, as it turns on the POSTs and ensures all the devices are working properly. All devices passed in my unit, so I proceeded at the OpenBoot "

ok

" prompt to type boot cdrom and watched the system boot up.

Booting the Kernel.

I watched with glee as the system booted the FreeBSD 10.1 kernel, but the glee quickly turned to dismay as I saw...

cd0: cd present [269120 x 2048 byte records]
Trying to mount root from cd9660:/dev/iso9660/10_1_RELEASE_SPARC64_CD [ro]...
warning: no time-of-day clock registered, system time will not be set accurately
|

No disk activity. The CD drive was quiet. The fans weren't even spun up, so there was no CPU activity. It was quiet... Too quiet... Eerily quiet for what was supposed to be a noisy, happy FreeBSD install. I pressed STOP+A to try and break into OpenBoot. The system was locked solid and it would not respond. Fortunately, it seems to be a glitch, and resetting the system (power down and power up) fixed this issue.

Installing the FreeBSD.

Since the Linux install failed and I already saved the contents of the SPARC slices, I used the Automatic Disk partitioning, which created a 17 GB / and 800 MB swap slice. This should be enough for most builds. It may not be enough for Qt, but I will probably not build Qt on here anyway. Now the fun part: Installing the distributions (somewhat akin to package sets on Linux, I guess). I downloaded the full Disc 1 ISO and burned it to a CD so I did not have to configure the network or wait for the - sadly only 10Mbit - Happy Meal Ethernet to download any files. It verified the checksums and immediately began installing to my local disk drive.

Note that I saw a few odd errors: "sym0: unexpected disconnect". This didn't seem to affect the installation any, so I'm not sure what exactly is going on here.

After that was finished, it was time to set up the devices. I configured networking (v4 + v6), added users for myself and my partner in crime, and then set out to install pkg-ng, the FreeBSD package management tool that makes FreeBSD so amazing and easy to use. Sadly, it does not support SPARC64, only the Intel x86 family of systems. The first order of business, then, is making a pkg-ng server for myself. I will cover that later, when I'm able to get that going. For now, it's ports (building from source) only.

Booting into the FreeBSD!

And now, everything is finished. Eject the CD and reset the system. This is going to be wonder----ful?

Luckily, one of my friends is an experienced SPARC sysadmin so she is going to help me fix it. Later today I will post the resolution, but first we have to find it. Until then, happy hacking!

Not having "resolutions" this New Years

Okay, I understand that many people like to make up "resolutions" for New Years. Stop me if you've heard this one before:

• "This is the year I start eating healthier."
• "This is the year I go the gym ALL THE DAYS."
• "This is the year I volunteer to ALL THE CAUSES."
• "This is the year I will change (some basic trait) and instead (do something seen as better by society)!"

Right, that never works. And here's why: the motivations are completely wrong. "New Year's Day" is, by the by, just another day. It's another day that is usually felt hung over, but still it is just another day. This is not a catalyst with which to make a large change, for yourself or for anyone else. What matters is not when you start something, but how and why you start it. Why do you want to eat healthier? Is it because it's the "in" thing to do, or is it because you genuinely want to be healthy? Do you really only want to be healthy because it's a new year?

I realise that many people view a new year as a "new beginning"; whilst a happy sentiment, and one that can sell a lot of greeting cards and party hats, it is nothing more than just that - a sentiment. The boundary of a year is an invented one. If you truly want to be healthy, you should do it because you want to be healthy, not because you want "this year to be different". When you inevitably slip up — don't be ashamed of yourself, we all do at one point or another — you can simply say "I still want to be healthy and I'll keep at it" instead of "oh well, this year isn't being different, I'll try again next year". You'll eventually run out of next years with which to try again.

So what are my goals for this year? I'd like to move to a better locale than I live now and finally finish my schooling. However, these aren't new goals I pulled out of my rear for the new year; I have been working at them for months now and I plan to achieve them this year. Plan ahead and don't stop for the bumps. Be inspired to smooth over the next ones!

I suppose my major point to this is that the best thing you can do this New Years is enjoy it with your friends and family. And if you want to change something besides your wall calendar, you need to have the right reasons for it. You are definitely worth your own investment of time and energy. You just have to see yourself that way, and then it will pay off in dividends.

Now playing: ♫ Skyfall - Adele