Using a yubikey in PKCS#11 mode to secure ssh and sudo

Version $Id: yubikey-sudo-ssh.html,v 1.6 2024/02/02 15:55:01 madhatta Exp $

While most of my earlier writeups on yubikeys focussed on HOTP OATH, most of my recent work with yubikeys has been to use them as HSMs for GPG keypairs (strictly speaking, three keypairs per yubikey; an authentication pair, a signing pair, and an encryption pair), then using gpg-agent in ssh-agent-compatibility mode to manage access to these keypairs, using them for controlling both ssh access to remote systems, and sudo access on those systems. For the avoidance of doubt, one single yubikey can do both these things (and moreover TOTP OATH and FIDO as well); you don't need to have one yubikey for OATH purposes and another for ssh keys.

The standard protocol for using GPG keypairs with a yubikey is PKCS No.11, often written "PKCS#11". Because PKCS#11 is a standard, also used by a number of other HSMs, this means that just about everything here is equally applicable to other compliant HSMs, such as Nitrokeys (I have done this with a Nitrokey Start as well).

The steps here are:

  1. Generate the keypair
  2. Enable the keypair for SSH access
  3. Configure sudo to use agent-based authentication

Generate the keypair

There are two basic ways to do this: generate the keypair external to the yubikey and deploy it to the key, or instruct the yubikey to create the keypair internally. The advantage of the latter is that the secret key is never exposed; it's born on the HSM, and will never leave it. The advantages of the former are that key escrow is possible (but if your external repository is compromised, the benefits of an HSM are lost), and that there have in the past been problems with the RNGs on some HSMs, leading to poor keypair generation. It's very hard to validate an RNG, but I'm working on the assumption that major shortcomings in widely-available HSMs would have been noticed by now, and that as long as you're buying reputable products from a reputable vendor you should be OK. Buying HSMs by unknown vendors sold on generic websites is not best security practice, and if you do it, you're an idiot.

This article assumes we generate the keypair on-device. That means arranging your own recovery scenario for loss-of-device; my preferred one is to share sysadmin duties with at least one other person, all of whom are using HSMs to secure ssh and sudo access, and with all of whom I have secure out-of-band communications. If I were to lose my yubikey, I'd simply get another, generate a new key, and have them replace my old privileged public key with a new one across the managed enterprise. Yes, this is a lot of work, and it bloody well should be.

Setting the user and admin PINs

Before we can generate a keypair, we have to set the admin and user PINs on the yubikey.

The user PIN is the one that must be entered each time they key is plugged in, before it will allow itself to be used. If the user enters this wrong three times, the user PIN becomes locked. The number of failures-before-locking is configurable, but we will not cover that here; it is also possible to configure they key to require the user PIN before each security operation, rather than before only the first after each power-on, but again, we will not cover that here.

The admin PIN is the one that must be entered to unlock a locked user PIN, and also to generate a key in the first place. If it's entered wrong three times, the yubikey will lock itself against anything except a factory reset, which erases the onboard keypairs. If, as mine, your recovery scheme involves sharing sysadmin duties with others, you may wish to let them set (and, presumably, securely record) your admin PIN, and vice-versa. It's possible (through poor crypto hygiene) for someone to expose their user PIN as well as losing control of their HSM, but nobody can expose a PIN they don't have. I note also that, at least on my devices, the PIN doesn't need to be numeric, or of set length. I find that a couple of random words are as secure as a six-digit PIN, and a great deal easier to remember.

Insert the yubikey into your computer; we assume that gpg-agent is up and running, as is the scdaemon (smart card daemon). you can confirm both of these by doing gpg --card-edit. If you see

[me@localhost ~]$ gpg --card-edit
gpg: selecting card failed: No such device
gpg: OpenPGP card not available: No such device
They are not, or the Yubikey isn't actually PKCS#11-capable, or something else is wrong. You will need to fix that. If instead you see
[me@localhost ~$ gpg --card-edit

Reader ...........: Yubico YubiKey OTP FIDO CCID 00 00
Application ID ...: D2760001240103040006160890710000
Application type .: OpenPGP
Version ..........: 3.4
Manufacturer .....: Yubico
Serial number ....: 43112911
Name of cardholder: [not set]
Language prefs ...: [not set]
Salutation .......: 
URL of public key : [not set]
Login data .......: [not set]
Signature PIN ....: not forced
Key attributes ...: rsa2048 rsa2048 rsa2048
Max. PIN lengths .: 127 127 127
PIN retry counter : 3 0 3
Signature counter : 0
KDF setting ......: off
Signature key ....: [none]
Encryption key....: [none]
Authentication key: [none]
General key info..: [none]
You're on the right track. First, reset the admin PIN (admin, passwd, 3, the default is "12345678"), then reset the user PIN (from the admin menu which you should still be in, 1, the default is 123456). Some HSMs (eg, Nitrokeys) won't let you change all the PINs until you've generated keys, so if your HSM knocks you back on this step, do the next subsection first, then return here.

Key generation

Then q to quit the admin menu, and generate to generate a new secure keypair. Decline the opportunity to make an off-card backup (see above discussion), specify a lifetime of 0 (key does not expire), and give a real name including "(yubikey)" or similar, since this can be quickly used to direct GPG to your on-card keypair.

Enable the keypair for SSH access

This is a lot simpler, assuming you've got a remote system set up for keypair-based ssh access. You will also need your local system set up to use gpg-agent in SSH compatibility mode for SSH agent purposes; gpg-agent in compatibility mode makes a better SSH agent than ssh-agent itself does, so it's a good thing to do, and you may already be doing it. You can check this with echo $SSH_AUTH_SOCK on your local system. If you get output like /run/user/1000/gnupg/S.gpg-agent.ssh, something that has "gpg-agent" in the name, you're good to go. If it says "ssh-agent", you need to fix that, which is outside the scope of this article; google on "ssh gpg-agent" for tips, I found quite a good guide here, and another here (at time of writing, Jan 2024).

Export your yubikey's keypair's public key, in the appropriate format, with

[me@localhost ~]$ gpg --export-ssh-key yubikey
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOGH6m3QPFStG7/ETTctJ9fYzK8qMU23Swn9dDv67hXwxKQ408N0WyR/Nk0pJjhIV4YEbJcwRhWX/uy+PxTWMFbwShKNkubrjDA9DE3iVEgupnam2d3clkpACA68fiTEHVVMbHjyz7mRLZd0UVMM7mGnO01VKWavBj5JB+ygAJvlnz0xzfF6AfUqcDu/1eCEYL9uAxsAX8Zi/aU2G5L4IbwTttAGclhzcC9nga0cVm5d2nbXY6JUTjjRIhnhmEKLWhS39sartoqMqbipbQRdGNRceVzfCLJYw/lvIGHmBP367uSKbPTHjwSkKbuKkj3/06CxNBdHkxetYByHfQW2mV openpgp:0xE0224427
Note this isn't a magic way of identifying a yubikey, it relies on your having included the string "yubikey" in the key's "real name" field on generation, above. Put that key in the remote system's SSH authorized_keys file (usually ~/.ssh/authorized_keys), making sure there are no extraneous line breaks introduced: the entire key should fit on a single line. It's also worth keeping a local copy of the public key, because if (like me) you have a lot of SSH keys and/or HSMs, the public key is used to tell the local SSH process which key should be used; let's say you save a local copy of the public key in ~/.ssh/my_yubikey.pub.

Insert your yubikey into your local system, and ensure that ssh can see it via the GPG agent with

[me@localhost ~]$ ssh-add -l
[...]
2048 SHA256:pVYQrooH7pjL0d/ax9KBEwMQ52lhyNVCmsed4Angk/E cardno:43 112 911 (RSA)
[...]
Note that the number that follows cardno: above is the same as the (last) one shown after card-no: in the output of gpg --card-edit, and also the same as the number printed in a tiny font on the underside of the yubikey. If you don't see your yubikey listed amongst your other keys, something's not working, and you'll need to fix it before continuing. Assuming you do, ssh into the remote system with ssh -i ~/.ssh/my_yubikey.pub user@remote.example.com. Enter your yubikey's user PIN when prompted (if the yubikey hasn't already been unlocked) and you should SSH right in.

Configure sudo to use agent-based authentication

This bit is not specifically linked to a yubikey per se, it's simply a method of delegating sudo authentication to a forwarded ssh agent connection; sudo access succeeds if, at the instant of request, the user can through the agent prove control of any SSH key specified in a list thereof. But if the only such specified key is the one permanently housed in their yubikey, then sudo access becomes tightly coupled to possession and control of that yubikey.

This is accomplished through PAM. Most distros have a library that needs to be installed (for Red Hat and derivatives, it's pam_ssh_agent_auth, while for Debian and derivatives it's libpam-ssh-agent-auth). Install the relevant package using your system software tool of choice. Then put one of the following lines near the top of your /etc/pam.d/sudo file:

auth	sufficient	pam_ssh_agent_auth.so file=~/.ssh/authorized_keys
or
auth	sufficient	pam_ssh_agent_auth.so file=/etc/ssh/keys/%u
The first uses the same authorized_keys file as the user uses to log in via ssh; this is convenient, because one list of public keys suffices to control all the user's access. The second means keeping key repositories for controlling sudo access separate from those used for ssh access, but has the distinct advantage that the repositories can all be owned by root. That prevents users adding other non-HSM-controlled keys to the list, at least without privilege, and further it's much easier to audit changes to those files using a file-based intrusion-detection system like tripwire.

I have seen several highly-intelligent and well-informed people arguing passionately for the latter, because with the former, if you can persuade a user to add a key to his authorized_keys file, you've not only compromised access to the system, you've compromised privileged access, and it's "game over". So the first formulation is convenient, but excruciatingly risky; the second formulation is inconvenient, but a great deal safer. I nearly always use the second; if you do likewise, you will need to make the /etc/ssh/keys directory, and populate it with one file per controlled user, named as that user's username, and containing the user's yubikey public key, in the format of an ssh authorized_keys file. Each such file should be owned by root, not by the specified user.

sudo very kindly tries hard to maintain security by sanitising the user's environment, including scrubbing unneeded environment variables. Unfortunately, one of those is the environment variable that tells the PAM library where to find the agent. So you will need to include the line

Defaults	env_keep += "SSH_AUTH_SOCK"
somewhere in your sudoers file. Note also that this technique doesn't do anything for actually assigning sudo privileges; it merely changes the authentication step so the user authenticates by proving control of an ssh key rather than control of their login password. You will need to grant sudo privileges to users in the normal way in order for this to be of any use. The user will also need to ensure that their ssh connections are set up to forward their agent, the details of which are outside the scope of this article. If when using sudo the user is prompted for a password in the normal way, check on the remote system that the agent has forwarded correctly with ssh-add -l, and that the yubikey-encapsulated key is one of those listed (again, look for cardno: in the output).

Finally, note that agent forwarding requires the creation of small but important files on the destination system, usually in /tmp. If that partition fills up, and the files can't be created, agent forwarding fails, and this means of authenticating to sudo will not work. In such circumstances, it is useful to have some other method of acquiring privilege, in order that the disc space situation can be remedied; either setting a user password (normal sudo authentication will still work), or having a securely-stored root password for use with su have both been found to be effective.

Back to Technotes index
Back to main page