Introduction¶

From https://developers.google.com/protocol-buffers/

Protocol buffers are Google’s language-neutral, platform-neutral, extensible mechanism for serializing structured data

Below is a simple .proto Protocol Buffer (protobuf or pb) definition:

syntax = "proto3";
package com.example.addressbook;

message Person {
    string name = 1;
    int32 id = 2;  // Unique ID number for this person.
    string email = 3;

    enum PhoneType {
        MOBILE = 0;
        HOME = 1;
        WORK = 2;
    }

    message PhoneNumber {
        string number = 1;
        PhoneType type = 2;
    }

    repeated PhoneNumber phones = 4;
}

message AddressBook {
    repeated Person people = 1;
}

Most of the protobuf ecosystem works by compiling the language neutral .proto definition into some language specific bindings, and protojure is no different. In the case of Protojure, ‘Language specific’ translates to Clojure maps and records. As such, you will find a map-oriented approach to working with protocol buffers with Protojure.

Getting Started¶

Given the above contents in a file, addressbook.proto, in our current directory, we may use the protojure protoc plugin to generate our Clojure bindings:

mkdir src/
protoc --clojure_out=src/ addressbook.proto

After running the above, we will find a nested directory structure corresponding to the package above (com.example):

$ cat src/com/example/addressbook.cljc
;;;----------------------------------------------------------------------------------
;;; Generated by protoc-gen-clojure.  DO NOT EDIT
;;;
;;; Message Implementation of package com.example.addressbook
;;;----------------------------------------------------------------------------------
...
..
. <File contents continues -- run the `cat` command above to see your generated output in full>

We can create a project.clj file alongside our .proto file:

(defproject protojure-tutorial "0.0.1-SNAPSHOT"
  :description "FIXME: write description"
  :url "http://example.com/FIXME"
  :license {:name "Apache License 2.0"
            :url "https://www.apache.org/licenses/LICENSE-2.0"
            :year 2022
            :key "apache-2.0"}
  :dependencies [[org.clojure/clojure "1.10.3"]

                 ;; -- PROTOC-GEN-CLOJURE --
                 [io.github.protojure/core "2.0.1"]
                 [io.github.protojure/google.protobuf "2.0.0"]])

Now, running lein repl:

$ lein repl
nREPL server started on port 35997 on host 127.0.0.1 - nrepl://127.0.0.1:35997
WARNING: cat already refers to: #'clojure.core/cat in namespace: net.cgrand.regex, being replaced by: #'net.cgrand.regex/cat
REPL-y 0.3.7, nREPL 0.2.12
Clojure 1.10.0
OpenJDK 64-Bit Server VM 1.8.0_222-8u222-b10-1ubuntu1~18.04.1-b10
    Docs: (doc function-name-here)
          (find-doc "part-of-name-here")
  Source: (source function-name-here)
 Javadoc: (javadoc java-object-or-class-here)
    Exit: Control+D or (exit) or (quit)
 Results: Stored in vars *1, *2, *3, an exception in *e

user=> (use 'com.example.addressbook)
nil

We create clojure representations of protobuf messages by requiring the appropriate ns generated by the protojure protoc plugin, as above, and then exercising the new-<Pb-message-name-here like so:

user=> (new-Person {:name "Jane Doe"})
#com.example.addressbook.Person{:name "Jane Doe", :id 0, :email "", :phones []}

We can then transform this in-memory clj representation to the protocol buffer bytes representation by requiring the protojure.protobuf namespace from the protojure lib protojure lib:

user=> (require '[protojure.protobuf :as protojure])
nil
user=> (protojure/->pb (new-Person {:name "Jane Doe"}))
#object["[B" 0x11398205 "[B@11398205"]

For illustration purposes, here is the native java byte array in a clojure vector:

user=> (into [] (protojure/->pb (new-Person {:name "Jane Doe"})))
[10 8 74 97 110 101 32 68 111 101]

In order to deserialize an array of bytes, we use the pb-><Message name here> form from the generated code (here, our ns com.example.addressbook):

user=> (pb->Person (byte-array [10 8 74 97 110 101 32 68 111 101]))
#com.example.addressbook.Person{:name "Jane Doe", :id 0, :email "", :phones []}

You’ve now round tripped a Protocol Buffer message from definition, to .clj language bindings, to in-memory clj representation to bytes and back to the in-memory form!

Advanced Topics¶

This section details how some of the more esoteric aspects of the protobuf ecosystem are supported under Protojure.

Well-known Types¶

Google supports a set of standard well-known types throughout their APIs, and they have been adopted by the broader Protobuf community in a few different ways. As such, Protojure offers support for them too. This library packages up the well-known types into Protojure pre-compiled artifacts so that they may be naturally consumed. This is standard practive for most language bindings in the ecosystem.

Any types¶

Protobufs support the notion of an Any type. This section details how to use the ‘Any’ facilities from within Protojure.

• coming soon *

Oneof Types¶

• coming soon *

Timestamp Types¶

• coming soon *