What Is the TLS Handshake Process?
A vast majority of web applications and websites today use the HTTPS protocol that enables data encryption between the client and server. It is, therefore, always a good idea to have at least a very basic understanding of how the whole encryption process works—how the encryption begins and how it is established, and that is what the TLS Handshake Process is all about. The whole idea behind the TLS Handshake process is that the client needs to talk to the server in a secure manner and they need to encrypt that communication.
For encryption, you need to have an encryption key that encrypts all the data. On a very high level, when implemented correctly, the TLS Handshake process allows a client (web browser) and a server (Apache HTTP Server, for example) to securely negotiate a session communication key over an insecure connection. Even if the key negotiation messages are intercepted, attackers cannot determine the actual session key negotiated by the client and server. The TLS Handshake process enables the sharing of the “symmetric encryption key” between the client and server so that both parties have the same key (remember here that symmetric encryption is a lot more efficient and faster than asymmetric encryption). This is where the whole TLS Handshake process comes in and it establishes the encrypted communication between the client and server.
Let’s take a look at the whole TLS Handshake process in detail:
1. The client may be your web browser—Google Chrome, Safari, Firefox, Microsoft Edge or any other web browser that you may use. The client wants to establish a secure connection/communication with the server and the very first thing that happens is that the client sends what is called a “Client Hello” message to the server.
Among several things the “Client Hello” message includes is the TLS protocol version number the client supports (for example, TLS version 1.2 or TLS version 1.3). Another important thing in the “Client Hello” message is the Cipher Suite that the client can support. So, let’s say that you use a modern browser that is up to date and can support some sophisticated encryption types or cipher suites—the client lets the server know about all of the different cipher suites that the client can support. The “Client Hello” message also includes a string of random bytes known as the "client random.”
On the server side, the web server maintains a list of the different cipher suites it’s going to support. It’s important to establish the order of precedence of the cipher suites on the server. For example, if the client comes in and says, “Hey server, I would only want this very basic/low-level and may be, not very secured encryption” and if the server wants to be able to support that, that obviously has to be configured on the server. Or, maybe there are certain cipher suites that are very outdated and full of vulnerabilities and the server wouldn’t want to support them—in that case, it needs to be made sure that those are excluded from the server.
Thus, in summary, for the very first step, the client sends the “Client Hello” message to the server. The “Client Hello” message includes, among several other things, the cipher suites that are supported by the client. Once the server receives the “Client Hello” message, it gets to choose the cipher suite that is going to be used during the whole handshake/communication process and the server’s choice of cipher suite is based on the available cipher suites from the client. For a visual of this process, see Figure 1.
Figure 1. The client sends a “Client Hello” message to the server.
2. In reply to the “Client Hello” message, the server sends a “Server Hello” message back to the client. The “Server Hello” message contains the server’s SSL certificate. The certificate includes a lot of different elements and one of the key things that is present in the certificate is the Public Key. This is important because the client can then use the Public Key to encrypt data and send those back to the server and the server will be able to decrypt things (encrypted by the client using the Public Key) using the Private Key that it has.
When the server sends the certificate to the client, the client checks the validity of the server certificate in order to be sure of certain things—that the certificate is not revoked, for example. After a successful validation, the client knows that it’s a good, valid certificate. Technically, this is the “Authentication” part wherein the client verifies the server's SSL certificate with the certificate authority that issued it. This confirms that the server is who it says it is, and that the client is interacting with the actual owner of the domain.
The “Server Hello” message also contains the server's chosen cipher suite, and the "server random," another random string of bytes that are generated by the server.
The server sends a “Hello Done” message to the client. And, with that, the “HELLO” portion of the whole communication process is then done with. For a visual of this process, see Figure 2.
Figure 2. A “Server Hello” message, followed by a "Hello Done” message, from the server to the client concludes the “HELLO” portion of the whole communication process.
3. At this juncture of the process, the client has the Public Key, and sends to the server one more random string of bytes, which is called the "premaster secret.” The client encrypts the “premaster secret” with the Public Key and sends the encrypted “premaster secret” back to the server in what is called a “Key Exchange” message. The server receives the encrypted “premaster secret” and because that was encrypted by the client with the Public Key and the server has the Private Key, the server decrypts the encrypted “premaster secret” using its private key. When the server decrypts the encrypted “premaster secret” it also now has the same “premaster secret” as the client.
Based on the client random, server random and “premaster secret,” both the client and server go through a series of calculations and they generate what can now be called a “Symmetric Encryption Session Key” (the client and server should arrive at the same results) that is going to be used to encrypt everything from that point onwards. Because the Symmetric Encryption Session Key is based on the same “premaster secret” that both the client and server have, the client and server can calculate the exact same Symmetric Encryption Session Key. For a visual of this process, see Figure 3.
Figure 3. The client and server arrive at the exact same Symmetric Encryption Session Key.
4. “Change Cipher Specification” messages are exchanged between the client and server. The “Change Cipher Specification” message is a way of the client and server telling each other that each is now changing from the “Asymmetric Encryption” over to the “Symmetric Encryption.” That way, when the bulk encryption will be done, going forward, the entire encryption process is going to be much more efficient because “Symmetric Encryption” is going to be used for the bulk encryption.
The client sends a “Client Finished” message that is encrypted with the Symmetric Encryption Session Key. The server also sends a "Server Finished" message encrypted with the Symmetric Encryption Session Key. At this point, a secure symmetric encryption has been achieved, the handshake is completed, and communication continues using the Symmetric Encryption Session Keys. For a visual of this process, see Figure 4.
Figure 4. The handshake between the client and server completes when a secure symmetric encryption session has been established.
5. Now, the bulk data encryption can happen using the Symmetric Encryption Session Key that both the Client and Server have generated.
Thus, what we have essentially seen is that the client and server go through asymmetric key exchange in order to ultimately get down to the symmetric bulk data encryption.
The Cipher Suite
If we think about the actual cipher suite that would be offered by the client or server (well, as we have already seen previously that it is ultimately chosen at the server level), it has algorithms like RSA, secure hashing algorithm (SHA) or Diffie Hellman Elliptic Curve (DHEC). RSA and DHEC deal with the whole “key exchange portion” of the TLS Handshake process. A cipher suite will also have cryptography specification standards like AES or AES-GCM. The reason why cipher suites have these different parts is because some parts deal with the key exchange portion (RSA or DHEC), while other parts deal with the symmetric bulk data encryption (AES, for example).
And, so, this is how the client establishes communication with the server in the context of the TLS Handshake Process.