Practical guide to securing gRPC connections with Go and TLS — Part 1

Published in

ITNEXT

8 min readJul 9, 2019

There are different ways to establishing a secure TLS connection with Go and gRPC. Contrary to popular belief, you don’t need to manually provide the Server certificate to your gRPC client in order to encrypt the connection. This post will provide a list of code examples for different scenarios. If you just want to see the code, go to the source-code repository. You need to clone this repository to follow along (Go1.11+).

“Web browsers don’t hold public certificates for TLS, why should my application?” [Not Required: gRPC Client Certs in Go]

This is part 1 of a series of three posts. In part 2 we will cover public certificates with Let’s Encrypt and Automated Certificate Management Environment (ACME), to finally touch on mutual authentication in Part 3.

Intro

As stated in RFC 5246, the primary goal of the Transport Layer Security (TLS) protocol is to provide privacy and data integrity between two communicating applications. TLS is one of the authentication mechanisms that are built-in to gRPC. It has TLS integration and promotes the use of TLS to authenticate the server, and to encrypt all the data exchanged between the client and the server” [gRPC Authentication].

In order to establishing a TLS Connection, the client must send a Client Hello message to the Server to initiate the TLS Handshake. The TLS Handshake Protocol, allows the server and client to authenticate each other and to negotiate an encryption algorithm and cryptographic keys before the application protocol transmits or receives its first byte of data [RFC 5246].

A Client Hello message includes a list of options the Client supports to establish a secure connection; The TLS Version, a Random number, a Session ID, the Cipher Suites, Compression Methods and Extensions as shown in the packet capture below.

The Server replies back with a Server Hello including its preferred TLS version, a Random number, a Session ID, and the Cipher Suite and Compression Method selected (TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 and null in the image below). The Server will also include a signed TLS certificate. The client⁠ — depending on its configuration⁠ — will validate this certificate with a Certificate Authority (CA) to prove the identity of the Server. A CA is a trusted party that issues digital certificates.

The certificate can also come on a separate message, as in the capture below.

After this negotiation, they start the Client Key exchange over an encrypted channel (Symmetric vs. Asymmetric encryption). Next, they start sending encrypted application data. I’m oversimplifying this part a bit, but I think we already have enough context to evaluate the code snippets to follow.

Certificates

Before we jump into code, let’s talk about certificates. The X.509 v3 certificate format is described in detail in RFC 5280. It encodes, among other things, the server’s public key and a digital signature (to validate the certificate’s authenticity).

Certificate  ::=  SEQUENCE  {
    tbsCertificate       TBSCertificate,
    signatureAlgorithm   AlgorithmIdentifier,
    signatureValue       BIT STRING  }

Before you ask, TBS implies To-Be-Signed.

TBSCertificate  ::=  SEQUENCE  {
    version         [0]  EXPLICIT Version DEFAULT v1,
    serialNumber         CertificateSerialNumber,
    signature            AlgorithmIdentifier,
    issuer               Name,
    validity             Validity,
    subject              Name,
    subjectPublicKeyInfo SubjectPublicKeyInfo,
    ...
    }

Some of the most relevant fields of a X.509 certificate are:

subject: Name of the subject the certificate is issued to.
subjectPublicKey: Public Key and algorithm with which the key is used (e.g., RSA, DSA, or Diffie-Hellman). See below.
issuer: Name of the CA that has signed and issued the certificate
signature: algorithm identifier for the algorithm used by the CA to sign the certificate (same as signatureAlgorithm).

SubjectPublicKeyInfo  ::=  SEQUENCE  {
    algorithm            AlgorithmIdentifier,
    subjectPublicKey     BIT STRING  }

You can see this as Go code in the x.509 library .

Creating self-signed certificates

While an SSL Certificate is most reliable when issued by a trusted Certificate Authority (CA), we will be using self-signed certificates for the purpose of this post, meaning we sign them ourselves (we are the CA). In Part 2 we will use Let’s Encrypt certificates instead.

The steps to create these are depicted below, we rely on openssl and a config file (certificate.conf) to prefer a Subject Alternative Name (subjectAltName) over the deprecated Common Name (CN).

In order to reproduce all this in one go, you can run make cert (which is a pre-requisite for all the gRPC examples to follow) after cloning the repository. The step by step is as follows.

Create Root signing Key

openssl genrsa -out ca.key 4096

Generate self-signed Root certificate

You can modify `/C=US/ST=NJ/O=CA, Inc.` to fit your location and imaginary CA name.

openssl req -new -x509 -key ca.key -sha256 -subj "/C=US/ST=NJ/O=CA, Inc." -days 365 -out ca.cert

This will result in the following certificate for our CA.

Create a Key certificate for the Server

openssl genrsa -out service.key 4096

Create a signing CSR

openssl req -new -key service.key -out service.csr -config certificate.conf

Generate a certificate for the Server

openssl x509 -req -in service.csr -CA ca.cert -CAkey ca.key -CAcreateserial -out service.pem -days 365 -sha256 -extfile certificate.conf -extensions req_ext

This will result in the following certificate.

Verify

You can take a look at the certificate with openssl as well.

$ openssl x509 -in service.pem -text -noout
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 12273773735572067708 (0xaa55342eea4ad57c)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=US, ST=NJ, O=CA, Inc.
        Validity
            Not Before: Jun 28 13:56:36 2019 GMT
            Not After : Jun 27 13:56:36 2020 GMT
        Subject: C=US, ST=NJ, O=Test, Inc., CN=localhost
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (4096 bit)
                Modulus:
                    00:c4:02:ab:d6:21:ac:38:58:98:cc:dc:65:b6:9b:
                    df:96:8f:4a:f9:9a:e2:ce:3a:65:78:07:6a:8b:d0:
                    ...

gRPC Service

Now, let’s take a look at how we apply and take advantage of all this with Go and gRPC with a very simple gRPC Service, to retrieve usernames by their ID. We will query for `ID=1`, which should return user Nicolas. The protobuf definition is the following.

The compiled code is already generated in the repository. You can compile again with make proto.

Insecure connections

Let’s check a couple of non-recommended practices.

Connection without encryption

If you do NOT want to encrypt the connection, the Go grpc package offers the DialOption WithInsecure() for the Client. This, plus a Server without any ServerOption will result in an unencrypted connection.

In order to reproduce this, run make run-server-insecure in one tab and run-client-insecure in another.

$ make run-server-insecure
2019/07/05 18:08:03 Creating listener on port: 50051
2019/07/05 18:08:03 Starting gRPC services
2019/07/05 18:08:03 Listening for incoming connections

In another tab

$ make run-client-insecure
User found:  Nicolas

Client does not authenticate the Server

In this case, we do encrypt the connection using the Server’s public key, however the client won’t validate the integrity of the Server’s certificate, so you can’t make sure you are actually talking to the Server and not to a man in the middle (man-in-the-middle attack).

To do this, we provide the public and private key pair on the server side we created previously. The client needs to set the config flag InsecureSkipVerify from the tls package to true.

In order to reproduce this, run make run-server in one tab and run-client in another.

Secure connections

Let’s look at how we can encrypt the communication channel and validate we are talking to who we think we are.

Automatically download the Server certificate and validate it

In order to validate the identity of the Server (authenticate it), the client uses the certification authority (CA) certificate to authenticate the CA signature on the server certificate. You can provide the CA certificate to your client or rely on a set of trusted CA certificates included in your operating system (trusted key store).

Without a CA cert file

In the previous example we didn’t really do anything special on the client side to encrypt the connection, other than setting the InsecureSkipVerify flag to true. In this case we will switch the flag to false to see what happens. The connection won’t be established and the client will log x509: certificate signed by unknown authority.

In order to reproduce this, run make run-server in one tab and run-client-noca in another.

With a Certification Authority (CA) cert file

Let’s manually provide the CA cert file (ca.cert) and keep the InsecureSkipVerify option as false.

In order to reproduce this, run make run-server in one tab and run-client-ca in another.

With CA certificates included in the system (OS/Browser)

An empty tls config (tls.Config{}) will take care of loading your system CA certs. We will validate this scenario in part 2 of this post series (with certificates from Let’s Encrypt for a public domain).

You can alternatively manually load the CA certs from the system with SystemCertPool().

certPool, err := x509.SystemCertPool()

If you have the Server cert and you trust it

This is most common scenario found on Internet tutorials. If you own the server and client, you could pre-share the server’s certificate (service.pem) with the client and use it directly to encrypt the channel.

In order to reproduce this, run make run-server in one tab and run-client-file in another.

Conclusion

There are different ways to go about setting TLS for gRPC. Providing integrity and privacy doesn’t take too much effort, so it’s strongly recommended you stay away of methods like WithInsecure() or setting InsecureSkipVerify flag to true.

Stay tuned for Part 2 and Part 3!.