Saturday, 28 October 2017

Identity, shame, stigma, and intolerance

I have seen a great number of people in the past few years disavow being a part of a culture or community that they once enjoyed or identified with due to an influx of sexism, nationalism, or other intolerance. I feel like this is a mistake and will only serve to strengthen intolerance amongst the masses, and this is what I'd like to write about and discuss today.

Virtually every person alive on Earth has at least some groups with which they identify. This could be a certain interest or hobby, ranging from music to photography to hiking and beyond. This could be their gender, male, female, or other. This could be a favourite pastime, whether that is sports, video games, or visiting museums. This list could go on for paragraphs and paragraphs. There are the fanatical and obsessive – just search your favourite social media platform for "Game of Thrones" for some decent examples. There are the truly interested and passionate – one of my favourite examples of this is Lazy Game Reviews, a channel on YouTube with enjoyably thorough reviews of old games and computer systems. There are all kinds of people and all kinds of ways to enjoy being part of a group or having an identity that is shared with others. This is typically a very healthy and normal thing for us social creatures.

In the past few years, political discourse has moved towards the more extreme. This has pervaded everyday communication in a way that had not yet been seen in the Millennial generation. The Millennials, in my experience, are generally some of the most open-minded people; however, this leads to a darker side. Just as most Millennials are open-minded towards acceptance of so-called "non-traditional" lifestyles and viewpoints ranging from economics to sex to religion and beyond, some Millennials are open-minded towards violent rhetoric, nationalism, anarchy, and intolerance.

This has sent a great number of the first sort of Millennials running scared from groups and identities that they would otherwise enjoy, because they do not want to be seen as supportive of these views that they feel are regressive. Unfortunately, this may indeed backfire on the ones that want to see the regression stop; when the tolerant leave, the intolerant remain. Let us look at the Southern Poverty Law Center's report on the horrific 2015 church massacre in Charleston, S.C. for an example. The summary: a young man who was raised to respect all people found a series of blogs and Web sites condemning a race, and was so moved by it that he committed a mass murder of that race. How did these blogs and Web sites, written specifically to influence young minds to become hateful and enraged to the point of violence, end up so highly ranked by a search engine?

One answer is that there are so many communities today overrun with people who legitimately believe in the hate and intolerance spread in such writings. This is in part due to the mass exodus of more tolerant people from those communities. As more people put shame and stigma on something as simple as playing video games – the media claims that video games somehow led sick and twisted Internet trolls to threaten rape and murder to women – less people want to admit to being gamers. This causes a vicious cycle, as the ones left stating they are gamers are the ones who are intolerant. This leads to a form of normalisation of the idea of intolerance amongst gamers; it's no longer out of the ordinary to think that anyone who enjoys video games might also enjoy threatening or committing violence towards other groups in real life. Couple this with the fact that teenagers have loved, do love, and will continue to love playing video games. Teenagers also want desperately to fit in with groups, to feel a part of something bigger. If they feel that people who enjoy video games should also hate women, that is what they will begin to do.

This could apply to any number of groups. Many secular people in the United States look down at religious people as being "backwards" or "traditionalist", when the truth of the matter is well over 60% of Catholics and Protestants support gay marriage and homosexuality. Many people view country music as regressive while attitudes, they are a-shifting. The stigma of being a gamer, or religious, or listening to country music comes not from any endemic intolerance, but from the tolerant people from these groups being too ashamed to admit their membership.

The most powerful statement that tolerant people can make in the groups they identify with is the very statement that they are tolerant and identify with said group. Don't erase your group identities to avoid being identified as intolerant. Show your group identity and tolerance; say out loud that you respect all your fellow humans and enjoy what you enjoy. This is the true path towards acceptance and togetherness.

Saturday, 7 October 2017

Fixing the laptop I broke

Sometimes things happen that you don't expect. It can be anything: a power failure during a system upgrade, or maybe a careless chmod 644 /usr/lib/libc.so — in my case, it was the latter (tab completion failure).

Training yourself on the proper way to respond to unexpected failures is the key to recovering them without pain or further data loss. When I had realised my gaffe, the first thing I thought was: How do I chmod it back without the ability to run chmod?

Static-linked rescue binaries are a must-have

The first thing I learned from this experience is that having a set of static-linked rescue binaries somewhere on your system can help in a lot of unexpected situations. We're going to be adding a busybox-static package to Adélie Linux just for such an occasion, and we may put it in the base system depending on community feedback. If I had a static busybox in, say, /var/recovery or such a path, this would have been a ten second fix rather than a few hour fix.

Embrace the system

After a few other attempts, I realised I could drop to assembler. Long ago, I spent my days writing assembler for system-level code. Since assembler is by design writing "below" C, you are not using the C runtime. Theoretically, you should be able to perform the same tasks as any utility on the system as long as there's a matching system call for it. And by luck, there is a single syscall: SYS_chown. Following is x86_64 assembler for Linux to chown /usr/lib/libc.so back to 755 (executable for all users):

_start: mov $90,%rax /* SYS_chown */ mov $str,%rdi /* const char *filename */ mov $493,%rsi /* mode_t mode */ syscall /* do it! */ mov $60,%rax /* SYS_exit */ syscall /* bye */ str: .ascii "/usr/lib/libc.so\0"

Then it was a matter of as -o fixit.o fixit.S; ld -o fixit fixit.o; strip fixit to generate a 440 byte binary file that would solve my issue. The next issue was transferring it to the laptop. I tried to use bash's /dev/tcp; unfortunately however, it does not support binary file transfer without something like `cat` or `dd`. Since I could only use the shell, I did what I had not done in over a decade: echo -n followed by the escape codes. Since a lot of the binary was still padding, I omitted the last 200 or so bytes. The output of the echo command needed to be redirected to a binary that was already executable (otherwise the file created would not have execute permission!), so I chose one I probably wouldn't need urgently: neon-config, a configuration utility for a library I installed for tinkering. The full shell transcript is in my misc Linux directory. This worked! And my laptop ran again...

As I said in the opening of this little musing: I could have made things a lot worse and lost all my open unsaved data by turning off the computer and trying to recover using media. Additionally, that computer is very picky about booting off external media, so that would have wasted even more time. Sometimes all you need is ingenuity and experience, and the only way to acquire either one is by messing about and poking at stuff! Happy hacking.

Thursday, 12 January 2017

Configuring a more secure password hash for OpenLDAP

While working on the Galapagos infrastructure, we ran in to an interesting issue: using passwd(1) as an LDAP user would cause it to add another password instead of modifying it. Setting up the slapo-ppolicy(5) overlay then caused passwd(1) to then fail with:

password change failed: Password policy only allows one password value
passwd: Authentication token manipulation error
passwd: password unchanged

After consulting the #openldap channel on Freenode, the problem turned out to be that although OpenLDAP allows you to set olcPasswordHash on the root cn=config node, it does not work correctly when set there; it must be set under olcDatabase={-1}frontend,cn=config. Note, however, that olcPasswordCryptSaltFormat does belong in cn=config directly.

Sunday, 8 January 2017

Configuring Apache 2.4 to serve GitLab over TLS / HTTPS

As part of my work assisting in the set up of the infrastructure for Galapagos Linux, I volunteered to install and configure GitLab. My colleagues had attempted to use the Debian Omnibus package, but that failed in spectacular ways, including references to directories in the configuration that did not exist after package installation.

The most important piece of advice I can give is that you absolutely must use Bundler v1.10.6 or older[1] to ensure that you do not receive Gemfile.lock errors. You will also need to make a small modification to the Gemfile and Gemfile.lock file to ensure that libv8 is present if you wish to precompile the assets.

Now, for the Apache configuration. Note that I assume you have enabled https in GitLab's config/gitlab.yml and set port: 443. You will need to set a forwarding request header[2] to ensure that GitLab does not throw CSRF authentication errors. Also, if you want to use the recommended Unix sockets of Unicorn, you will need to configure the ProxyPass and ProxyPassReverse to use unix:/path/to/socket|http://HOSTNAME (thanks, Xayto!) - the full VirtualHost for GitLab goes something like this:

<VirtualHost *:443>
        ServerName git.glpgs.io
        ServerAlias code.glpgs.io
        ProxyPass / unix:/home/git/gitlab/tmp/sockets/gitlab.socket|http://git.glpgs.io/
        ProxyPassReverse / unix:/home/git/gitlab/tmp/sockets/gitlab.socket|http://git.glpgs.io/
        SSLEngine on
        SSLCertificateFile /path-to-certificate.crt
        SSLCertificateKeyFile /path-to-key.key
        SSLCertificateChainFile /path-to-ca-chain.crt
        Header always set Strict-Transport-Security "max-age=15768000"
        RequestHeader set X_FORWARDED_PROTO 'https'
</VirtualHost>

<VirtualHost *:80>
        ServerName git.glpgs.io
        Redirect permanent / https://git.glpgs.io/
</VirtualHost>

Additionally, I recommend that you follow Mozilla MozWiki's great TLS advice or use their super handy, easy config generator as a global configuration that applies to all of your VirtualHosts. On Debian, you can pop that in to /etc/apache2/mods-available/ssl.conf, replacing the parameters they already specify.

Happy hacking!

Wednesday, 21 December 2016

Ah, wonderful health hazards

I can't tell what has been overall worse for my health in the past few weeks. The bathroom connected to my home office directly sits over the complex's "laundromat station". This did not used to bother me. In fact, I was quite okay with this, because it means I have the closest walking distance of any of my neighbours to it. However, for the past two or three weeks, I can smell — from the office, mind — a very strong odour of laundry detergent every time someone does a load. Turns out a lot of people do loads in the 18:00 to 21:00 time slot on weekdays, which happens to be when I am at my most productive in my office. I cannot imagine this is at all healthy for me.

But then I remember I've spent every day since Saturday spending multiple hours trying to set up OpenLDAP for new project. I've always just used Active Directory on the server-side, so my only experience thus far with OpenLDAP has been client-side. It's a great client library with easy configuration and a great debug mode that will tell you exactly what is happening and what is going wrong. Unfortunately, the server part, at least on Debian, uses "dynamic configuration" which means everything is in LDAP.

Now, look, LDIF and LDAP are fine and great for phone book-style records. It makes perfect sense. That is what it was designed to do. Storing regexp in ASN.1 BER is pushing it. But the way they do HDB/MDB grouping feels to me like trying to fit in with all those cool kids with their NoSQL and their MapReduce and their terrible terribly-great performance by using "shards" everywhere. And our leader wants replication so that it's fault tolerant. Now I get to convert decades-old documentation about an "enterprise" feature to this "dynamic configuration" thing. I cannot imagine this is at all healthy for me.

Configuring OpenLDAP to authenticate using X.509 client certificates

This is not meant to be a comprehensive guide by any means, but information on the Web for configuring OpenLDAP to authenticate using X.509 client certificates is lacking. And in some cases, over a decade old! It took me hours to find the documentation I needed, but only minutes to see it working once I had the correct "recipe".

You should probably be running your own Certificate Authority for the purpose of generating client certificates, especially since you need one per user. You can lock it up tightly and only use it for the purposes of LDAP if you like. You can also use a certificate vendor like Thawte or GeoTrust or Comodo. Make sure you pick just one, though, because you will configure OpenLDAP to trust only that single CA to sign all the relevant client certificates. (This ensures that nobody can come in with a forged certificate signed by another vendor, or a self-signed one.)

The Ubuntu guide on making a CA is pretty decent, though unfortunately it uses the inferior GnuTLS package. That's okay, because we are only using it for OpenLDAP. And actually, you can't use OpenSSL generated certificates on Debian's OpenLDAP because they patched it in such a way that the certificates cannot be read. (There are conflicting reports on whether this bug was fixed or not upstream.) Note that you definitely want to set a higher expiration_days than the default 365! 10 or even 15 years isn't unheard of, which is 5475 days if you were wondering.

Once you have either created your CA, or decided on a vendor, you may begin configuring OpenLDAP. Replace authority.pem with the file name for your CA's root certificate, and ldap_cert.pem and ldap_key.pem for the server certificate and its private key. Note that the server certificate must have the FQDN of the LDAP server as its only CN. It may have a wildcard as a subjectAltName (or SAN) but the FQDN (normally something like ldap01.myproject.org) must be the CN.

With slapd.conf

TLSCACertificateFile /etc/ssl/certs/authority.pem
TLSCertificateFile /etc/ssl/certs/ldap_cert.pem
TLSCertificateKeyFile /etc/ssl/private/ldap_key.pem
TLSVerifyCert try

With Dynamic Configuration, aka cn=config, aka "OLC"/on-line configuration, aka ...

dn: cn=config
changetype: modify
add: olcTLSCACertificateFile
olcTLSCACertificateFile: /etc/ssl/certs/authority.pem
-
add: olcTLSCertificateFile
olcTLSCertificateFile: /etc/ssl/certs/ldap_cert.pem
-
add: olcTLSCertificateKeyFile
olcTLSCertificateKeyFile: /etc/ssl/private/ldap_key.pem
-
add: olcTLSVerifyCert
olcTLSVerifyCert: try

Note that if you receive an error such as:

ldap_sasl_interactive_bind_s: Unknown authentication method (-6)
 additional info: SASL(-4): no mechanism available: 

then you most likely forgot the olcTLSVerifyCert like I did the first time :) Note that there is nothing printed after "no mechanism available: ". That was the hardest part to debug! Hopefully this can help a few people out.

Also note that for client certificates to work correctly, the DN of the X.509 certificate must exactly match the DN of the LDAP object. If you cannot meet that requirement, you will need to look at authz-regexp: for cn=config, see this mailing list posting, and for standard configuration see the documentation. Note that I was unsuccessful in making this seemingly-useful feature work, but you may have better luck than I did.

References

Wednesday, 14 December 2016

Let's Encrypt and why I still pay for TLS certificates

I am asked with alarming regularity why I am not using Let's Encrypt for my personal Web sites, and for Adélie's site, and for my mother's art gallery site, and so on. "Why do you pay money for something you could have for free? And then you aren't giving money to those evil CAs!"

TLS certificates are still very much "you get what you pay for". Let's Encrypt is free, and on paper it seems to be a great solution with roots in freedom and socialism. However, it has a number of large issues in practice that prevent me from being able to adopt it.

The first, and most evident, is the failure of the community to provide a single ACME client that is well-supported and provides configuration options. As of this writing, there are 49 different client implementations on the official site. The problems with them are as numerous as the offerings; my main complaint is that most of them require themselves to run as the root user to automatically write to sensitive certificate files that are owned by the Web server user and are chmod 400.

The second large issue I've seen is that most of these 'automatic updates' break. This can be due to administrator error - and since there is not one single option, there cannot be a single repository of knowledge. This can also be due to APIs or endpoints changing. I have seen an official Mozilla blog and Void Linux's repository broken in the last week alone, all by botched ACME cron jobs. This solution is sold as "set and forget", but it requires more effort than simply going to a site every year and inputting a CSR and privkey.

Other issues with Let's Encrypt include: Let's Encrypt lacks a "site seal" which is very important on e-commerce sites to foster user trust. Let's Encrypt does not provide OV (let alone EV), which also compromises trust in people who know what to look for.

All in all, I think going forward Let's Encrypt may be suitable for power users and people who run TLS servers off their home servers. It may even be suitable for some personal sites and blogs. But I don't think it is a long-term solution for person who need trust, or those who have a complicated infrastructure (such as a distro, like Adélie).