Proxying HTTPS

Proxying HTTPS messages sounded like a challenge, but after a bit of research, it was actually quite simple. There are two main things to know:

  1. You have to have a certificate authority on the client’s machine.

This is essential because the proxy has to do the SSL handshake with the client and pretend to be the host that you are proxying to. mkcert is a handy tool that automatically creates a certificate authority, installs it in your browser, and creates certificates for hosts.

To successfully complete the SSL handshake, you need to have a certificate for each host that the client wants to talk to. With mkcert, this is simple and we can shell out to it to create a new certificate if we don’t already have the right one.

(defun certificate (host &key create)
  (let ((cert (cert-file (uiop:strcat host ".pem")))
        (key (cert-file (uiop:strcat host "-key.pem"))))
    (when create
      (unless (and (uiop:file-exists-p cert) (uiop:file-exists-p key))
        (uiop:with-current-directory ((cert-file ""))
          (uiop:run-program (uiop:strcat "mkcert " host)))))
    (values cert key)))

Now in the proxy, we can call (certificate "www.google.com" :create t) and get the certificate and key files, creating new ones if we didn’t already have them.

  1. You need to handle HTTP and HTTPS on the same port.

Since a proper intercept proxy needs to handle both protocols, we need a way to do it on the same port. To do this, we can use RFC 2817.

In our connection handler, we need to make a special case if the request method is CONNECT, and then upgrade the protocol to HTTPS. It is really quite easy, all we have to do is write a 200 response to the socket, and then pass it to another connection handler that will do the SSL handshake.

Here is a simplified example of what my server looks like:

(defun write-ssl-accept (stream)
  "Send a 200 response to accept the connection."
  (http:write-response stream (make-instance 'http:response)))

(defun ssl-connection-handler (conn host)
  "Handle connections upgraded to SSL."
  (with-ssl-server-stream (stream conn (first (str:split ":" host)))
    (let* ((req (http:read-request stream :host host))
           (resp (send-request-ssl req :raw t :host host)))
      (http:write-response stream resp))))

(defun connection-handler (conn)
  "Handle incoming connections from the client."
  (let* ((stream (us:socket-stream conn))
         (req (http:read-request stream)))
    (if (string-equal :connect (http:request-method req))
        (let ((host (puri:render-uri (http:request-uri req) nil)))
          (write-ssl-accept stream)
          (ssl-connection-handler conn host))
        (let ((resp (http:send-request req :raw t)))
          (http:write-response stream resp)))))

Conclusion

It’s really not that hard once you find the right RFCs and tools (2817 and mkcert), and now we are able to intercept and read HTTPS messages as if they were plain HTTP.

2024-03-16