The Key To Your SSH Strategy

By Blog

If you work with systems that run any variety of Linux or BSD then the probability is high that you have dealt with SSH. Invented in 1995 and established as an internet standard by the IETF in 2006, Secure SHell has become the default mechanism for remote access to servers by individuals and teams everywhere.

SSH Authentication

Authenticating yourself to *nix servers can take a variety of forms, but the most common among them are simple pairings of username and password, or a public and private cryptographic key. Key based authentication is widely regarded as the preferred method due to the greater security that it provides.


Using a password as the means of establishing your identity is subject to a variety of weaknesses. If you monitor the system logs of any server that has an SSH server accessible to the internet, you will see a nearly constant barrage of login attempts from attackers attempting brute force techniques to gain unauthorized access. It is also essentially impossible to ensure that the user who enters a given password is the individual who that credential was intended for. Social engineering, poor password hygiene, and shared logins all increase the probability that your servers will be compromised.

Key Pairs

While it isn't the only alternative to passwords, key-based authentication is by far the most commonly used and widely supported. The primary advantages that it provides over passwords are the ability to add a password as a second factor, and the need for the private key to be present on the device that is logging in.

The first step to using this mechanism is to generate a pair of keys, one that is "public" and can be shared freely without risk of compromise, and one that is "private" which needs to be kept secret and secure. The public key is then placed on the target system(s) in an "authorized_keys" file which is owned by the user account that you would like to log in to. The private key will be placed in a default directory which varies depending on the platform that you are using. Alternatively, you can use the "ssh-agent" process to load and manage your private keys on your behalf.

When generating your key pair there are a few choices to be made which can affect the overall security of your system. The first decision is the type and strength of the key, followed by whether to protect it with a password. Unless the key is going to be used by an automated process, it is a good practice to use a password to protect it from unauthorized use in the event that it is leaked to an attacker.

The default key formats are usually one of RSA or ECDSA, with RSA being the older and more established algorithm. Within those formats are the option to create keys of varying size. In general, a larger key is more difficult to compromise, with the tradeoff being the encrypting and decrypting messages with it will take longer, though on modern hardware that time difference is negligible.

As an example, the following command will generate an RSA key with a size of 2048 bits, that is password protected, and save it to the default location for Linux and OSX computers.

Server Side

On the server side you can enforce which authentication options are supported, and for which users. Assuming that you have decided to rely on key based login, you should disable the option to use passwords on your servers. It is also advisable to disable logging in as the root user. By preventing root logins you are forced to either grant sudo permissions to your users, or configure access to specific commands and directories, both of which ensure that you are considering who should be granted which powers.

Adding the following settings to your `/etc/ssh/sshd_config` file will configure your SSH server process with the desired behaviors.

The challenge that comes with key-based authentication is that as soon as you move beyond a single user and a single server you have to consider how to track, control, and rotate those keys.

Considerations For Your Key Management Strategy

There are a variety of factors that you should be thinking of as you determine how best to design and implement your SSH key management strategy. When using public key authentication as your default (or preferably only) login mechanism, each key that is provisioned on your infrastructure grants access to someone. Access, however, is a rather nuanced concept in this situation


Authenticating a login session on a server requires the user who initiates that connection to have access to the combination of private key and username that are provisioned in advance. At face value this would appear to allow for easy access control and attribution of user actions, but only if each member of your team uses their own set of credentials. An additional concern is how to effectively modify or revoke access when an employee leaves the organization.


Beyond the comparatively simple matter of access and authentication is the process of managing the authorization of a user once they have gained access to a server. What permissions should each user be given? Should you grant superuser (sudo) access to everyone, or do you need more fine-grained control over what commands are allowed?

Beyond the question of what permissions to grant and to whom, is the matter of managing that set of rules in a consistent manner. The more users and systems that are under your purview, the greater the complexity and the need for a scalable solution to the problem.


Once you have determined how to grant access to your servers and manage permissions, you will still need a way to keep a record of who did what and when. Audit trails are an essential requirement of most compliance regulations, in addition to being an important part of proper systems administration and a security best practice.

Audit logs allow you to determine, either retroactively or in real-time, when an unauthorized or destructive action was taken, who performed that action, and the broader context of the user session at the time. By maintaining these records it is easier, in the event of a malicious actor, to identify weaknesses or vulnerabilities in your servers to be addressed. If the damage is a result of inadequate education or guard rails for your employees, then you can review what mistakes were made and update your tooling and documentation to avoid the situation in the future.


The proxy approach to managing SSH access to your systems is a viable one in principle, what makes the difference is how it is executed in practice. The strongDM architecture is also proxy based, with the critical difference being how it manages authentication and authorization.

Rather than needing to integrate your identity management system with OpenSSH, or the PAM system, on each server, you only need to do it once when you set up strongDM. Alternatively, you can use the built in user and role management within the strongDM admin interface. This will give you a single location to manage role-based access control for all of your servers, as well as assigning your users to the appropriate role based on their needs and responsibilities.

To set up your environment to work with strongDM you will still need to provision keys on the destination hosts, but only the one needed by the strongDM gateway. Once that is in place, your users will request a session from the gateway and be granted a temporary credential that is only valid for a single use. Upon connection, all of their activity will be logged within strongDM for your later review and analysis. Because all of the traffic is routed through the strongDM gateway, this audit log also applies when using a bastion or jump host to access additional servers.

By delegating authentication to strongDM and removing it as a concern of your SSH servers, you also gain easy access to integrations with multi-factor authentication (MFA) providers. For further scaling, strongDM natively supports hierarchical deployment patterns so that your users don't have to keep track of which proxy to use for which environment.

Now you have a better idea of the challenges and concerns that are involved in the seemingly simple practice of managing SSH access to your servers. While it can be straightforward at small scales of users and machines, as you scale along one or both of those axes, the challenge and complexity of building and maintaining a secure and easy to use workflow grows exponentially.

In our next post we will explore some of your options for keeping your sanity while you grow your business. If you want to learn more about how strongDM can simplify your SSH strategy, and make it scale with your business, book a time today to talk to one of our experts.

SOC2 certification looming? See how strongDM makes it simple.