Introducing Internet Object

The Internet Object (IO) is a well-conceived data serialization format that has been designed specifically for transmitting textual data over the internet. At the core of IO is an object structure that serves as an ordered collection of values, similar to CSV, but with additional capabilities such as support for nested objects, arrays, inline keys, and more. These features make IO a powerful and flexible tool for researchers seeking to optimize data transmission over the internet.

A plain IO object

John Doe, 25, T, {Bond Street, New York, NY}, [red, blue]

JSON

{
  "0": "John Doe",
  "1": 25,
  "2": true,
  "3": {
    "0": "Bond Street",
    "1": "New York",
    "2": "NY"
  },
  "colors": ["red", "blue"]
}

The Internet Object is a document-oriented format that puts emphasis on schema-first design. When a schema is included in the header of an Internet Object document, it appears as shown in the following example.

An IO document with a separate header and data section

name, age, active, address: {street, city, state}, colors: []
---
John Doe, 25, T, {Bond Street, New York, NY}, [red, blue]

JSON

{
  "name": "John Doe",
  "age": 25,
  "isActive": true,
  "address": {
    "street": "Bond Street",
    "city": "New York",
    "state": "NY"
  },
  "colors": ["red", "blue"]
}

The schema-first approach of the Internet Object format ensures that data received is always in a consistent and predictable format. In addition, the document-oriented approach separates the data from the definitions, ensuring that the two do not become intertwined. Internet Object documents can contain a collection of data records, and this collection enforces record isolation, supports data streaming, and enhances readability.

An IO document with a collection of records and schema

name, age, active, address: {street, city, state}, colors: []
---
~ John Doe, 25, T, {Bond Street, New York, NY}, [red, blue]
t~ Jane Done, 20, T, {Bond Street, New York, NY}, [green, purple]

JSON

[
  {
    "name": "John Doe",
    "age": 25,
    "isActive": true,
    "address": {
      "street": "Bond Street",
      "city": "New York",
      "state": "NY"
    },
    "colors": ["red", "blue"]
  },
  {
    "name": "Jane Doe",
    "age": 20,
    "isActive": true,
    "address": {
      "street": "Bond Street",
      "city": "New York",
      "state": "NY"
    },
    "colors": ["green", "purple"]
  }
]

In addition to schema, the header section of an Internet Object document can contain other metadata such as keys and values, variables, and multiple schemas, among other definitions. It is worth noting that the schema in the header section is not mandatory, and documents can be sent without it if the receiver already has a schema.

For example, the following IO code does not include a schema as it is assumed that the recipient of the document already has access to the schema.

An IO document with a collection of records & definitions (api response) in header

~ recordCount: 22 
~ page: 3
~ prevPage: "/api/v1/people/page/2"
~ nextPage: N
---
~ John Doe, 25, T, {Bond Street, New York, NY}, [red, blue]
~ Jane Done, 20, T, {Bond Street, New York, NY}, [green, purple]

JSON

{
  "recordCount": 22,
  "page": 3,
  "prev": "/api/v1/people/page/2",
  "next": null,
  "data": [
      {
        "name": "John Doe",
        "age": 25,
        "isActive": true,
        "address": {
          "street": "Bond Street",
          "city": "New York",
          "state": "NY"
        },
        "colors": ["red", "blue"]
      },
      {
        "name": "Jane Doe",
        "age": 20,
        "isActive": true,
        "address": {
          "street": "Bond Street",
          "city": "New York",
          "state": "NY"
        },
        "colors": ["green", "purple"]
      }
    ]
  }
}

Definitions are key-value pairs that can be used to define metadata, additional header details, schemas, variables, and other elements. Definitions provide a convenient way to encapsulate and manage such metadata within an Internet Object document.

For further information on how definitions can be used in the context of the Internet Object format, please the Definition section.

In ideal scenarios, it is not recommended to include the schema within an Internet Object document. Instead, the schema is typically compiled and stored at the endpoints where the documents are serialized and deserialized. This approach can provide certain benefits, such as allowing for easier versioning of schemas and reducing the size of the document by eliminating redundant schema information. However, it is important to note that there may be cases where it is necessary to include the schema within the document itself, such as when the recipient does not have access to the schema or when the document needs to be self-contained. As with any design decision, the choice of whether to include the schema within the document or store it at the endpoints should be made based on the specific requirements of the system or application in question.

Person Schema

name: string,  # The person name
age: {int, min:20, max:35},  # The person age!
isActive?: {bool, deafult:False},
address?*: { # Person's address (optional) (Nullable)
  street: string, 
  city: {string, choices:[New York, San Fransisco, Washington], 
  state: {string: maxLen:2, choices:[NY, CA, WA]}
}, 
colors?: [string] # Color array in the form of string array

The following example will produce the same results, regardless of whether it is parsed using the independently compiled Person schema (as mentioned earlier) or not. This illustrates the flexibility of the Internet Object format, as it allows for the schema to be included within the document itself or stored separately at the endpoints. This capability makes the format suitable for a wide range of applications and scenarios, as it can be adapted to suit different requirements and constraints. Overall, the Internet Object format's ability to work with independently compiled schemas underscores its versatility and robustness as a data serialization format.

An IO document with a collection of records and the API response in the header

~ recordCount: 22 
~ page: 3
~ prevPage: "/api/v1/people/page/2"
~ nextPage: N
---
~ John Doe, 25, T, {Bond Street, New York, NY}, [red, blue]
~ Jane Done, 20, T, {Bond Street, New York, NY}, [green, purple]

JSON

{
  "recordCount": 22,
  "page": 3,
  "prev": "/api/v1/people/page/2",
  "next": null,
  "data": [
      {
        "name": "John Doe",
        "age": 25,
        "isActive": true,
        "address": {
          "street": "Bond Street",
          "city": "New York",
          "state": "NY"
        },
        "colors": ["red", "blue"]
      },
      {
        "name": "Jane Doe",
        "age": 20,
        "isActive": true,
        "address": {
          "street": "Bond Street",
          "city": "New York",
          "state": "NY"
        },
        "colors": ["green", "purple"]
      }
    ]
  }
}

At many places in this document, JSON code snippets are included alongside their equivalent IO code. The aim is not to compare IO and JSON but rather to provide readers, especially those familiar with JSON, with an accessible understanding of the IO format.

It should be noted that in certain instances, the JSON object may differ from the associated IO version. This is because IO includes concepts such as header and data separations, built-in schema support, and variables, which cannot be directly mapped with the JSON object. Despite these differences, the inclusion of JSON examples serves as a useful tool for conveying the key features and capabilities of the IO format.