Using GnuTLS with QTcpSocket

It's been quite a while since I last wrote a blog post, but it's not because I haven't been coding, in fact quite the opposite. The Qt opengov project is finally underway and I've been doing quite a lot of work on the various SSL classes. I'm now an official Qt approver, so as you can see the process of getting non-nokia developers the ability to commit to Qt is working.

In Qt 4.x and currently in Qt5 even though the various QSsl classes provide an abstraction from the underlying SSL implementation, there is only one backend and it uses openssl. I recently made a change that means we can add new backends during the Qt5 life time by separating the concepts of SSL support from the availability of SSL - of course this doesn't change anything if openssl is the only game in town.

Over the last couple of weekends, I've been investigating GnuTLS as a potential implementation that could form the basis of a second backend. I started with the easy part - handling X.509 certificates, and last weekend managed to implement some code that showed that we could implement this part of the Qt API using GnuTLS. I did hit some issues of course, but most were minor. The GnuTLS maintainer Nikos Mavrogiannopoulos has already fixed the minor documentation issues I spotted, and has even implemented a couple of features that I found were missing - definitely a sign that this library is being actively maintained I think you'll agree.

Today I attempted the more challenging task of trying to make GnuTLS work through a QTcpSocket. This integration is essential for using the library in Qt since without it features such as proxy support etc. would not be available - it really is a 'must have'. As you might guess from the fact I'm bothering to blog about it the results were positive, so let's take a look.

What I've done for this prototype is implement a QObject that provides the same basic outline as a QIODevice subclass. In order to keep things simple, I haven't yet tried to make something that is tied to that specific API at this point. I've also not tried to integrate this into QSslSocket, instead this code aims to demonstrate that doing so will be feasible in future.

The API I've implemented is reasonably close to what QSslSocket offers, the header file should make it clear that most of what you'd expect is there:

class SslSocket : public QObject
{
    Q_OBJECT

public:
    SslSocket(QObject *parent=0);
    ~SslSocket();

    QByteArray read(qint64 maxsize);
    
public slots:
    void connectToHost(const QString &hostname, int port);
    void startHandshake();

    qint64 write(const QByteArray &data);

signals:
    void handshakeComplete();
    void readyRead();
    void error();

private slots:
    void dataReceived();

private:
    void setupSession();
    void handshakeInternal();

    ssize_t readInternal(void *buffer, size_t length);
    ssize_t writeInternal(const void *buffer, size_t length);
    ssize_t writeVectorInternal(const giovec_t *iov, int iovcnt);

    static ssize_t read_callback(gnutls_transport_ptr_t transport, void *buffer, size_t length);
    static ssize_t write_callback(gnutls_transport_ptr_t transport, const void *buffer, size_t length);
    static ssize_t write_vector_callback(gnutls_transport_ptr_t transport, const giovec_t *iov, int iovcnt);

    SslSocketPrivate *d;
};

Now you've seen the header, let's take a look at the implementation. The first method we'll look at is the one that performs the initial setup of our SSL session. This happens before we even attempt to send any data to the server since once of the things we're going to do is change the way data is sent to run it via QTcpSocket:

void SslSocket::setupSession()
{
    qDebug() << "Initialise client session";

    // Setup the trust store
    gnutls_certificate_allocate_credentials(&d->x509cred);
    gnutls_certificate_set_x509_trust_file(d->x509cred, "/etc/ssl/ca-bundle.pem", GNUTLS_X509_FMT_PEM);

    // Configure the session
    gnutls_init(&d->session, GNUTLS_CLIENT);
    gnutls_credentials_set(d->session, GNUTLS_CRD_CERTIFICATE, d->x509cred);

    const char *err;
    gnutls_priority_init(&d->priority_cache, "NORMAL", &err);
    gnutls_priority_set(d->session, d->priority_cache);

    // Setup the transport functions to use QTcpSocket
    gnutls_transport_set_ptr(d->session, this);
    gnutls_transport_set_pull_function(d->session, read_callback);
    gnutls_transport_set_push_function(d->session, write_callback);
}

The first thing this method does is setup the trust store, and point it to the location of the CA bundle on my opensuse 12.1 system. In fact, this is a total waste of time, since verifying the certificate isn't something I've implemented yet! The next part is rather more useful however, we initialise a session and tell GnuTLS that we're acting as an SSL client. After this, we set the priority of the various ciphers etc. that are available. This step seemed unimportant, but I discovered that if you fail to do it then rather than using a sane default, GnuTLS dies with an internal error. The last setup step we do is to tell the library that we want to use our own functions to send and receive data rather than using the built-in ones.

For each of the read and write functions, we provide a callback. This is a static method (since a pointer-to-member-function is a no-no for reasons I won't go into). Our static method then calls the member function that implements our callback:

ssize_t SslSocket::read_callback(gnutls_transport_ptr_t transport, void *buffer, size_t length)
{
    SslSocket *self = static_cast<SslSocket *>(transport);
    return self->readInternal(buffer, length);
}

ssize_t SslSocket::readInternal(void *buffer, size_t length)
{
    qDebug() << "readInternal, length" << length << ", available" << d->socket->bytesAvailable();

    if (d->socket->bytesAvailable() < qint64(length)) {
        gnutls_transport_set_errno(d->session, EAGAIN);
        return -1;
    }

    return d->socket->read(static_cast<char *>(buffer), length);
}

As you can see the static function simply casts the user data pointer (which we provided using gnutls_transport_set_ptr earlier) to our class, then calls the appropriate method. The read implementation simply sees if we have enough bytes in our socket's buffer to satisfy the request, and if not tells the library to try again. If we do, then naturally we read the data. The data we need can only become available when the eventloop is running since that's when QTcpSocket performs its data transfers. If we simply retried immediately then we'd see no more data than we did during our first attempt.

The write function is similar, but since QTcpSocket has buffering of its own we can simply perform the write as requested. My real implementation supports a slightly more complex form of the write function that can process requests to write several blocks of data at once, but the basic concept is the same.

In order to allow the retry behaviour I mention above, there's a simple state machine. The SslSocket has the following basic states:

enum State
{
    Disconnected,
    Connecting,
    Handshaking,
    Encrypted
};

If new data is received then we look the state and retry the relevant operation as follows:

void SslSocket::dataReceived()
{
    qDebug() << "dataReceived() state is " << d->state;

    if (d->state == Handshaking) {
        handshakeInternal();
    }
    else if (d->state == Encrypted) {
        emit readyRead();
    }
}

The handshakeInternal() method is the one that actually performs the SSL handshake, and as you can see if we're in the appropriate state then we retry each time more data is received. The actual implementation is pretty simple:

void SslSocket::handshakeInternal()
{
    qDebug() << "start handshake, state is" << d->state;
    int result = gnutls_handshake(d->session);
    qDebug() << "shake result" << gnutls_strerror(result) << result;

    if (result == 0) {
        qDebug() << "handshake completed";
        d->state = Encrypted;
        emit handshakeComplete();
    }
    else if (gnutls_error_is_fatal(result)) {
        qDebug() << "fatal error in handshake";
        emit error();
        d->socket->disconnect();
        d->state = Disconnected;
    }
}

As you can see, once we get a real result from the handshake (ie. one that can't be solved with more data) then we emit either the handshakeComplete() signals or the error() signal and update the state of the SslSocket. If the handshake is completed successfully then we can now send and receive encrypted data.

The read and write methods are similar. In both cases, we need to consider the possibility that we need to retry (indicated by either an interrupted, or again error). The implementations are as follows:

QByteArray SslSocket::read(qint64 maxsize)
{
    QByteArray buffer;

    if (d->state != Encrypted) {
        qFatal("Write before encrypted is not supported yet");
        return buffer;
    }

    buffer.resize(maxsize); // ### This could get pretty big!
    ssize_t result;
    do {
        result = gnutls_record_recv(d->session, buffer.data(), maxsize);
    } while( result == GNUTLS_E_INTERRUPTED || result == GNUTLS_E_AGAIN );

    buffer.truncate(result);

    return buffer;
}

qint64 SslSocket::write(const QByteArray &data)
{
    qDebug() << "write";

    if (d->state != Encrypted) {
        qFatal("Write before encrypted is not supported yet");
        return -1;
    }

    ssize_t result;
    do {
        // ### It's possible that this could fail due to the need to read some data
        // for example if a renegotiation is underway. Not sure.
        result = gnutls_record_send(d->session, data.constData(), data.size());
        qDebug() << "write result" << result;
    } while( result == GNUTLS_E_INTERRUPTED || result == GNUTLS_E_AGAIN );

    return result;
}

In order to test this, I wrote a very simple class that will perform an HTTP GET and read the response. The result is that we can can connect to an HTTPS server and see the root page - not very exciting I guess but a sign that everything is working!

GetSlash::GetSlash(SslSocket *sock)
    : QObject()
{
    this->sock = sock;
    connect(sock, SIGNAL(handshakeComplete()), this, SLOT(start()));
    connect(sock, SIGNAL(readyRead()), this, SLOT(gotData()));
}

void GetSlash::start()
{
    sock->write(QByteArray("GET / HTTP/1.0\n\n"));
}

void GetSlash::gotData()
{
    QByteArray result = sock->read(2000);
    qDebug() << result;
}

Finally there's a basic main() function that ties all this together, and a little bit of code to log what's going on. The end result is that I now feel confident that we could implement a working backend for QSslSocket etc. using GnuTLS without too much trouble.

I'll add the code to my normal qt-examples repository later, but if anyone needs it (or earlier experiment experimenting with the certificate APIs) then just let me know.

EDIT: The code is now available here https://gitorious.org/qt-examples/qt-examples/commit/40e57647242079745c46d84e15d92f2093af300f