Photo by Austrian National Library on Unsplash

Protobuf vs JSON for Android

Data communication protocol efficiency and robustness

Introduction

As Android Developers we are often consuming data via web services. JSON is a king, it is widely spread and it is very well supported everywhere.
But slowly I see Protobuf gaining momentum, in this article you will learn some what is Protobuf and the key advantages using Protobuf over JSON. Let’s get started.

What is Protobuf

Protobuf is short for Protocol Buffer. It is a language and platform agnostic mechanism to serialize structured data. It is used to communicate between heterogenous systems, such as client and server.
Protobuf is designed to be compact and efficient. This makes it ideal for applications where bandwidth or performance is a critical concerns.
Protobuf enforce strong typing, developers need to clearly define the object structure and type.
In addition data is encoded as binary and not string. The binary format allow a compact footprint.

What is JSON

JSON (JavaScript Object Notation in it’s long form, is a lightweight, human-readable data interchange format. It’s widely used to transmit data between different systems, especially in web applications. JSON data is structured as key-value pairs, similar to objects in JavaScript. This makes it easy to parse and manipulate by programming languages. JSON is often used for APIs, web services, and data storage (No SQL databases).

Structure comparison

JSON vs Protobuf for the same object, I use this tool for this example.

JSON Representation

{
 "firstname": "John",
 "lastname": "Doe",
 "dateOfBirth": "1990–01–01",
 "city": "New York"
}

Protobuf Representation

Data definition

message Person {
 string firstname = 1;
 string lastname = 2;
 string dateOfBirth = 3;
 string city = 4;
}

Actual data

bare in mind that this data is transmitted as binary, hence its actual length is shorter.

Base64: CgRKb2huEgNEb2UaDjE5OTDigJMwMeKAkzAxIghOZXcgWW9yaw==
Hex: 0a044a6f686e1203446f651a0e31393930e280933031e28093303122084e657720596f726b

Is Protobuf an improvement over JSON

Memory foot print

In short, YES. Protobuf is arguably more memory efficient than JSON. Protobuf stores data in binary format while JSON store everything as string.

Let’s take an example with encoding the number 12345.

When serialized, Protobuf uses a variable-length encoding scheme called varint. For small integers like 12345, varint can encode the value in a single byte.
In this example, Protobuf would likely use only one byte to represent the integer 12345, while JSON would use 5 bytes, one for each character as JSON encode everything as string, that’s 5x more! In addition JSON will need to add all the structure (Curly brackets, double quotes, comma…) and each additional character will require one more byte.

Protobuf is also more efficient for string as it apply the same principle converting character to integer and using varint.

CPU and processing

Protobuf is generally faster to process than JSON. This is primarily due to its more compact binary (less footprint to read) format and the efficient decoding algorithms used by Protobuf libraries. Additionally, Protobuf’s schema-based approach allows for efficient validation and type checking, which can improve performance.

Ease of use

JSON is easier to use than Protobuf, mostly because Protobuf requires a set of tools like a protobuf compiler. In addition Protobuf structure need to be define strictly where as JSON is very flexible. JSON allows faster iteration on schema. That said it is often required to constraint decoded JSON to object, Java, Kotlin or other languages, hence while the format is flexible, usage is not much more flexible than Protobuf.
Protobuf requires to share the pb file with all participants (server, clients…), so the protocol is clear with all parties.

Strict typing

Protobuf format is strictly typed. This mean your data structure can be defined with primitive types such as integer, string… This offer more robustness and is less prompt to errors. JSON on the other end as no strict data format, it can often lead to parsing errors.

So what should you use for your next app?

If efficiency is important and you have a lot of users, Protobuf is often a better option. While it takes slightly longer to setup, it guarantees better performance and robustness over time.
If you are prototyping your app and you need to iterate fast on the data models, or you haven’t many users, you might just be happy with JSON.

Can I use Protobuf with Retrofit or Ktor?

Yes, Retrofit offers a converter for protobuf (just like for JSON), it works out of the box. You need to setup Retrofit with the protobuf convert as follow:

Retrofit retrofit = new Retrofit.Builder()
 .baseUrl(API_URL_STRING)
 .addConverterFactory(ProtoConverterFactory.create())
 .build();

Ktor also supports protobuf with its ktor-serialization-kotlinx-protobuf plugin.

You need to initialize Ktor with protobuf as below:

install(ContentNegotiation) {
 protobuf()
}

Then you can use your POJO Kotlin or Java object as usual.

Conclusion

Choosing a communication protocol is a big stake as you may use it for years. Both JSON and Protobuf can do this job, Protobuf offer undeniable advantages over JSON at the cost of slightly more complicated setup as the protocol needs to be shared with different participants.

Given both Ktor and Retrofit support Protobuf and JSON with minimal change, you will be able to migrate from one to the other.

If efficiency is important have a chat with your team and consider Protobuf. Happy coding!

Thanks for reading this far. You can show you appreciation by following and clapping, thanks again!