This document aims to provide the Internet Object 1.0 specification and showcase various aspects of the subject.

The Internet Object format is a document-oriented format that emphasizes the separation of header and data. This structure is similar to that of HTML, and MIME, where the header is kept separate from the data or body.

In an Internet Object document, the header is optional but can be used to define schemas and definitions. The data section always starts with the --- separator. This separator is the first element of the data section and is mandatory to distinguish it from the header.

[ Internet Object Document Structure Diagram ]

Internet Object Document Examples

Full Document

If an Internet Object document includes both a header and a data section you can call it a full document.

# Header
name, age:int, address: {street, city, state}, active # Header

# Data Section
---
John Doe, 25, {Bond Street, New York, NY}, T

Data-only Document

When an Internet Object document contains only a data section, it is okay to omit the --- separator. Such documents are sent to the server without any header because the schema is either not required or already known to the recipient.

With Separator:

---
John Doe, 25, {Bond Street, New York, NY}, T # Data section

Without Separator:

~ John Doe, 25, {Bond Street, New York, NY}, T
~ Jane Done, 48, {Malibu Point 10880, Malibu, CA}

Header-only Document

In many cases, a query-generating document may not yield any results. In such cases, you can use the header with result metadata to send the query and the results. However, it is important to include the --- separator to mark the end of the header and the start of the data section.

# Header Record Metadata
~ reocordCount: 0
~ pageSize: 10
~ currentPage: 1
~ nexPage: N
~ prevPage: N

# Empty Data Section
---

Document with Multiple Data Sections

Internet Object document can contain multiple data sections. This facility allows user to provide multiple types of data collection to be embedded in the single document.

# Schema section
~ $address: {street, city, state, zip} # Adddress Schema
~ $person: {firstName, lastName, age, gender }

# The collection of person
--- $person
~ John, Doe, 25, M
~ Jane, Doe, 22, F

# The address
--- $address
~ Bond Street, New York, NY, 500001
~ Georeg Street, New York, NY, 500002

Internet Object document structure is designed to be simple and flexible. The next section will discuss the Header and Data section in detail.

The poem serves as a unique and memorable medium to communicate the foundational principles that form the basis of the Internet Object format. By creatively capturing these concepts in verse, the poem enables readers to appreciate and remember the essence of the data interchange system more effectively. The Internet Object format is designed to be efficient, clear, and versatile in facilitating data interchange, and the poem highlights these attributes by artistically expressing the philosophy and goals that drive its development. The poem, therefore, not only adds an engaging element to the specification but also reinforces the core values of the Internet Object format.

Poem

Size holds weight, in bytes confined, Small prevails, large left behind.

Simplicity shines over complexity's shroud, Readability echoes, accurate and loud.

Reusability births productivity's rise, Verbosity's burden efficiency defies.

Data, definitions, separate ways, Together they clutter, apart they amaze.

Headers and data, distinctions drawn, Confusion dissolves, clarity's dawn.

Errors and statuses, data's divide, Their entanglement brings chaos inside.

Two lone records, states unswayed, No interference, connections unmade.

Trust not the sender, vigilance displayed, Expect the unanticipated, foundations laid.

Surprises, enchanting, yet beware, Not all of them good, handle with care.

Internet Object is a data interchange format designed for modern network communication. This specification introduces Internet Object as a text-based, schema-first, document-oriented, and streamable format that prioritizes human readability and language independence. Internet Object aims to optimize the serialization of structured data for efficient transmission between servers and clients across the internet.

Internet Object (IO) is a document-oriented data serialization format designed to optimize data transmission over networks. This specification introduces IO as an alternative to existing formats such as JSON, offering a structured approach to data representation and exchange.

Core Structure

The fundamental structure of IO is an ordered collection of values, analogous to CSV (Comma-Separated Values) but with extended capabilities. These capabilities include support for nested objects, arrays, and inline keys, providing enhanced expressiveness and flexibility.

Key Features

• Document-Oriented Design: In contrast to value-oriented formats, IO adopts a document-centric approach, facilitating the separation of data from definitions to enhance clarity and maintainability.

• Ordered Collection with Extended Functionality: IO's core structure maintains an ordered collection of values while supporting complex data structures such as nested objects and arrays.

• Schema-First Approach: IO emphasizes schema-first design to ensure data consistency and predictability. While schemas are optional, their inclusion significantly enhances data integrity and validation.

• Concise Syntax: The syntax of IO is optimized for readability and efficiency, minimizing data size without compromising clarity.

• Metadata Integration: IO documents can incorporate metadata, variables, and multiple schemas within the header section, providing comprehensive context for the data.

Illustrative Examples

Basic IO Document Structure

The following example demonstrates a basic IO document structure:

This structure illustrates IO's concise syntax and inherent schema support. For comparison, an equivalent JSON representation would be:

IO Document with Collections and Data Types

IO supports collections and various data types, as demonstrated in the following example:

This example illustrates several key features:

• Explicit data type definitions in the schema (string, int, bool)

• Nested object structures (address)

• Collection of objects denoted by the tilde (~) prefix

• Correspondence between the order of values and the schema definition

The equivalent JSON representation would be:

This comparison demonstrates IO's capacity to represent structured data collections efficiently, offering a compact and readable format while maintaining an ordered structure.

Advanced Examples

Separate Schema and Document with Collection

In many scenarios, it's beneficial to define schemas separately from the data. This approach allows for schema reuse, versioning, and easier maintenance. Here's an example of a separate schema followed by a document using that schema:

1. Separate Schema (person.io)

1. Document with Collection and Metadata

In this example:

• The schema is defined separately, potentially in a file named "person.io".

• The document references the schema URL in its metadata.

• The document includes additional metadata such as record count and pagination information.

• The collection contains multiple records, each prefixed with ~.

• Each record follows the structure defined in the schema, including an array of skills.

This structure allows for efficient data transmission, as the schema only needs to be sent once and can be cached by the receiving system. It also facilitates easy updates to the schema without necessarily changing the data format.

Conclusion

Internet Object represents a significant advancement in data serialization technology. By combining the simplicity of ordered collections with the robustness of schema-based validation, Internet Object offers a powerful yet accessible solution for modern data exchange needs. Its key strengths include:

1. Efficiency in data transmission and storage

2. Clarity through its schema-first approach and document-oriented design

3. Flexibility in handling various data structures and types

4. Compatibility with existing JSON-based systems

These attributes make Internet Object suitable for a wide range of applications, from web-based and networked environments to data storage and interchange in diverse domains such as IoT, cloud computing, and enterprise systems.

The subsequent sections of this specification provide comprehensive details on Internet Object's syntax, schema definition language, supported data types, and advanced features. This information will enable developers, system architects, and data engineers to fully leverage the capabilities of Internet Object in their projects and applications.

As data exchange continues to play a crucial role in our interconnected world, Internet Object stands poised to address current challenges and anticipate future needs in data serialization and transmission.

The inception of the Internet Object began as a side project aimed at addressing the limitations observed in the JSON format. Over time, it evolved into an independent research endeavor, focusing on effectively tackling data-transfer challenges such as size, schema validation, data streaming, header, and metadata support, among others. The design of the Internet Object format revolves around the following key objectives:

Uninfluenced Development

To optimize the format for internet wire transfer, the Internet Object must be conceived and developed without being excessively influenced by existing mechanisms. However, it may draw inspiration from other formats as needed.

Human Friendly

Internet Object documents must be text-based, human-friendly, and easy to work with. Developers should be able to write these documents using plain text IDEs without needing any frameworks, libraries, or utilities.

Minimal Footprint

To ensure a small footprint, the Internet Object format should separate data and schema, allowing data to be sent alone over the network.

Schema First

To uphold data integrity during wire transfer, the Internet Object format should prioritize a schema-first approach.

Document Oriented

Embracing a comprehensive document-oriented approach, the Internet Object format should facilitate the bundling of all essential components - including records, data, definitions, schemas, and comments - within a single document. This approach ensures that all related information is conveniently stored together, promoting the efficient organization and streamlined management of data and its associated elements. Moreover, it enhances maintainability and simplifies collaboration among team members, as they can easily access and understand the complete context within a single, unified document.

Complex Data Types

The Internet Object must support complex data types so that any kind of data whether large number or complex data structure can be easily serialized and deserialized for the wire.

Streaming Friendly

The Internet Object format should support the streaming of independent records, allowing for efficient and continuous data transfer. With this feature, the failure of a single record will not affect the processing of other records, ensuring more resilient data transmission.

Platform and Language independence

The Internet Object format should be designed to work seamlessly across different platforms, operating systems, and programming languages. This universal compatibility ensures broad adoption and versatility, enabling developers to easily integrate the format into their projects without limitations.

Comments

By providing support for inline comments, the Internet Object format allows users to document schemas and definitions directly within the data itself. This feature enhances readability and maintainability, making it easier for users to understand and manage complex data structures.

Reusability

To increase the format's adaptability, the Internet Object should promote reusability through concepts like references and variables. This dynamic feature allows for the customization of data structures and enables users to manipulate data more effectively, catering to various needs and scenarios.

The header of an Internet Object document is positioned at the beginning of the document and serves a crucial role in defining the schema or associated definitions for the data it contains. This section includes essential metadata, context, variables, and schema references for the document's content. It plays an important role in ensuring that the data is presented in a consistent format and provides the necessary information for accurate interpretation and processing.

[ Header Image Placeholder ]

Default Schema

name, age:int, address, isActive?, remark
---

In this schema example, five keys are defined with additional details:

1. name: Represents a standard key, expected to contain a value such as a string.

2. age:int: Specifies that the age key should contain an integer value, indicating the data type explicitly.

3. address: Another standard key, which could hold a more complex value like a string or an object, depending on the context.

4. isActive?: The question mark (?) signifies that the isActive key is optional, meaning it may or may not be present in the data.

5. remark: Represents a standard key, expected to contain a value, likely a string, which could hold additional comments or notes.

This schema not only defines the structure but also includes type annotations and optionality, enhancing the clarity and robustness of the data model. By using this schema, the document can ensure consistent and accurate data representation, making it easier to process and interpret across different systems.

Definitions

Definitions, at their core, are collections of key-value pairs used to declare metadata, variables, complex schemas, and other key-value pairs within the header of an Internet Object document.

~ pageSize: 1
~ currentPage: 1
~ recordCount: 4
~ $address: {street, city, state}
~ $schema: {name, age, $address}
---

In this example, the header contains response metadata and schema details presented as Definitions, rather than using a Default Schema as seen in the previous example. The Definitions provide metadata that specify the page size (pageSize), the current page number (currentPage), and the total record count (recordCount). Additionally, more complex structures are defined, such as an address schema ($address) with nested keys (street, city, state) and a higher-level schema ($schema) that references both simple and complex data types. The $schema is a reserved key used to define the default schema for the document.

Special characters are used in conjunction with structural characters and literals to provide additional functionality or context within an Internet Object document. The following are the special characters used in the Internet Object format.

The Internet Object format includes several structural charactersm, literals and other special characters that are used to structure and delimit data within a document. These characters are used in conjunction with objects, strings, arrays, numbers, and whitespace to create complex and flexible data structures.

In the Internet Object format, whitespace refers to any character with a Unicode code point less than or equal to U+0020 (i.e., characters in the range U+0000 to U+0020). This range includes both non-printable control characters and common whitespace characters such as the horizontal tab (U+0009), newline (U+000A), vertical tab (U+000B), form feed (U+000C), carriage return (U+000D), and space (U+0020).

In addition to the characters in the range U+0000 to U+0020, the Internet Object format also includes characters in the Unicode whitespace category as whitespace. This includes characters such as the non-breaking space (U+00A0), em space (U+2003), and en space (U+2002), among others. Including Unicode whitespace characters can make it easier to work with text in languages that use non-Latin scripts, such as Arabic, Chinese, or Japanese.

It's also worth noting that the Internet Object format recognizes the zero-width non-breaking space (U+FEFF) as whitespace. This character is often used as a byte order mark (BOM) in Unicode-encoded documents. Incorporating a more comprehensive range of whitespace characters in Internet Object offers several advantages that can make the format easier to work with, more readable, and more compatible with different systems and programming languages.

The following table lists the valid whitespace characters:

Notes

• Case Sensitivity: All whitespace characters are recognized based on their Unicode code points. Ensure that the correct character is used to avoid parsing issues.

• Whitespace Sensitivity: Internet Object is not whitespace sensitive, meaning that the parser ignores the whitespaces surrounding the values and structural elements. However, any whitespace characters found within the values or strings themselves are preserved.

• Reserved Characters: All listed whitespace characters are reserved and should not be used as part of identifiers or keys to prevent conflicts and parsing errors.

• Best Practices:

  • Enhance Readability: Use whitespace characters like spaces and tabs to format your document for better readability.

  • Avoid Unnecessary Whitespace: While whitespace can improve readability, excessive or unnecessary whitespace can clutter the document.

  • Consistent Formatting: Maintain a consistent use of whitespace throughout the document to ensure uniformity and ease of maintenance.

  • Be Mindful of Invisible Characters: Some whitespace characters, like zero-width spaces, are invisible but can affect the parsing and rendering of the document. Use them only when necessary.

Structural characters define the structure of data within an Internet Object document. Below are the structural characters used in the Internet Object format:

The Data Section in an Internet Object Document is where the actual data resides. An internet Object document can have one or more Data Section. It consists of one or more sections, each defined by a separator line (---) and optionally accompanied by a section name and schema. The data itself can be represented as either a single object or a collection of objects, allowing for a flexible and structured approach to data representation. Following diagram shows the structure of the Data Section.

Structure Overview

Section Separator Line

Each Data Section begins with a separator line (---), which organizes the document into distinct sections. The separator line can include optional elements:

• Section Name: Identifies the section and its purpose.

• Schema Name: Defines the structure or constraints of the data, prefixed with $.

ℹ️ The separator line must end with a newline character (\n) or EoF (End of File).

The separator line can take on various forms for different levels of detail, each ending with a newline character (\n) or EoF (End of File):

• Without Name and Schema: The simplest form, just the separator (---).

• With Section Name: The separator followed by a section name (--- employee).

• With Section Name and Schema: The separator followed by a section name and a schema name, separated by a colon (--- employee : $employee).

• With Only Schema: The separator followed by just the schema name (--- $employee).

Rules for Section Names and Schemas

• Omitting Section Name: In a multi-section document, the section name can be omitted only once. When omitted, the section name will be derived from the associated schema (e.g., --- $employee implies that the section name is employee).

• Default Section Name and Schema: If both the section name and schema are omitted, the section name will default to data, and a default schema will be used.

• Unique Section Names: Each section in an Internet Object Document must have a unique section name. Duplicate section names are not allowed.

Examples of Section Separator

Separator Line without Name and Schema

It is the simplest form of the separator line. It will use the default section name (data) and the default schema set for the document.

Separator Line with Section Name (employee)

Here the section name is employee. The schema will be the default schema set for the document.

Separator Line with Section Name and Applicable Schema

Here the section name and schema are both are explicitly mentioned as employee and $employee respectively.

Separator Line with Only Schema

Here only the schema is mentioned. The section name will be derived from the schema name (employee). However, if the document the section name is already used, then it will be an error.

Data

After the separator line, the data within a section is introduced. This data can either be a single object or a collection of objects. The flexibility in data representation allows the Internet Object Document format to handle various types of information efficiently.

Objects

Objects are structured data entities composed of key-value pairs. Each object is defined within curly braces {} and can contain nested objects or other data types, forming a hierarchical structure.

Collections

Collections represent a list of objects, making it possible to include multiple records within a single Data Section. Each object within a collection is defined in the same way as a standalone object but is part of a broader collection context.

Examples of Data

Single Object

A single object can be represented after the separator.

It is not necessary to have a section separator for a single section document if there is no header or schema. Hence, the above example can be written as:

Collection of Objects

A collection is represented by listing objects, each prefixed with ~ on separate lines:

Empty Data Section

You can have an empty data section. An empty data section can be represented by just the separator line without any data.

It is not necessary to have a section separator for an entirely empty document.

Multi-Section Document Example

An Internet Object Document can include multiple sections, each with its own data:

The Data Section, organized by separators and structured using objects and collections, offers a robust and flexible method for handling data within Internet Object Documents. This structure ensures that the documents are clear, consistent, and effective for a wide range of applications.

Literals are specific values that can be used within an Internet Object document. They represent special values and basic data indicators. Below are the literals used in the Internet Object format:

Usage Examples

Notes

• Case Sensitivity: All literals are case-sensitive. For example, True or FALSE are not recognized as valid literals.

• Short vs. Long Forms: The short forms (T, F, N) are convenient for brevity, while the long forms (true, false, null) enhance readability and compatibility with JSON.

An array is represented by a pair of square brackets, which may contain zero or more values. It begins with an open square bracket ([ U+005B) and ends with a close square bracket (] U+005D). Each value is separated by commas (, U+002C). Essentially, an array is expressed as a sequence of values separated by commas enclosed in square brackets.

Syntax

Array Structural Characters

Characteristics

Arrays can contain values of various types, including objects, other arrays, strings, numbers, boolean, and null.

Basic Arrays

A simple array of strings:

An array of objects:

An array with mixed values:

Multi-dimensional Arrays

Arrays can be nested to create multi-dimensional data structures.

Two-dimensional Arrays

Two-dimensional arrays represent rows and columns:

Three-dimensional Arrays

Three-dimensional arrays represent collections of two-dimensional arrays:

Empty Arrays and Empty Values

An empty array is represented by a pair of square brackets with no values:

Empty values between array elements are not permitted. To include a missing value, you must explicitly specify a valid value such as null. However, since the Internet Object specification neither assumes null by default nor supports undefined, any omission is strictly forbidden. Following are some examples of invalid array structures:

Internet Object values must adhere to a specific set of data types, including Objects, Arrays, Strings, Numbers, Boolean values, and Null. The values can also be represented using specific literals such as T, true, F, false, Inf, -Inf, NaN, N, and null. By adhering to this set of data types, Internet Object promotes consistency and interoperability across different implementations and use cases.

Like many programming languages, strings in Internet Object are a sequence of Unicode codepoints. They may be enclosed in quotation marks (" U+0022 or ' U+0027) or remain free without being enclosed. One noticeable difference with Internet Object strings is they all preserve the whitespace found within the boundary!

String Formats

The Internet Object strings can be written in three different formats (a) Open Strings (b) Regular Strings (c) Raw Strings. All of these formats have different ways of representing strings and handling escapes.

Objects are a fundamental element in Internet Object documents, providing a clear and intuitive way to represent structured data.

An object is expressed as a sequence of values (and key/value pairs) separated by commas (, U+002C). For simplicity, clarity, and ease of reading, Internet Object supports two modes for objects: Open and Closed. The Open mode does not require enclosing values in curly brackets and is only supported for top-level objects.

Syntax

Open Object

Closed Object

Object Structural Characters

Characteristics

Objects can contain values of various types, including other objects, arrays, strings, numbers, boolean, and null. Keys can also be attached to all or some of the values to provide more information and make the objects more self-explanatory. The keys are valid Internet Object string values, and any format of string (Open, Regular, Raw) can be used to represent them.

Basic object

An object is essentially an ordered collection of values similar to CSV records.

John Doe, 25, T

Objects with child objects and arrays.

John Doe, 25, T, {Bond Street, New York, NY}, [extrovert]

An object is not required to be wrapped inside the curly braces unless it is a child object. However, putting them in between the braces will not make it invalid.

{John Doe, 25, T, {Bond Street, New York, NY}, [extrovert]}

Object with Inline Keys

Object structure also supports unique inline keys. In the following example isActive, address, and personalities have associated keys.

{
  John Doe,
  25,
  isActive: T,
  address: {Bond Street, New York, NY},
  personalities: [extrovert]
}

Inline keys can be attached to all the values or some of them. When an object contains both types of values (key value and non-key value), the key-value pair must be placed after the non-key values (sequential values).

{
  name: John Doe,
  age: 25,
  isActive: T,
  address: {Bond Street, New York, NY},
  personalities: [extrovert]
}

{
  name: John Doe,                       # Open string key
  address: {Bond Street, New York, NY},
  peamlrsonalities: [extrovert],
  "age": 25,                             # A key in regular string!
  'isActive': T                          # A key wrapped in single quotes
}

Empty objects and empty trailing values

An empty object must be enclosed by curly braces.

{} # An empty object

An object can contain empty values. The following object contains two empty values, after the name John Doe, and the second before the address.

John Doe,,true,, {Bond Street, New York, NY}

Trailing empty commas are ignored.

John Doe,,,,,

Providing accurate numerical representation for various applications, Internet Object numbers offers a system that efficiently handles tasks ranging from simple counting to complex financial calculations. It supports three numeric data types—Number, BigInt, and Decimal—designed specifically to meet different numerical requirements in modern applications.

[Diagram: The Number Values]

• Number (64-bit floating-point): Ideal for fractional or general floating-point values.

• BigInt: Suited for very large integers that exceed the 64-bit limit.

• Decimal: A fixed-precision type that stores exact numeric values with a set number of decimal places, making it critical for high-precision applications like financial calculations.

Internet Object supports a variety of number formats, allowing:

• Integers to be expressed in decimal (base 10), binary, octal, or hexadecimal.

• Floating-point numbers to be written using scientific notation.

• IEEE 754 special values such as NaN and Infinity to be represented.

This is the simplest and the most basic type of format is the Open string format. As the name suggests, open strings are not enclosed within any sort of enclosures or quotation marks. This free and open type of string starts with any non-whitespace codepoint. They end when any structural character(s) is encountered or when the end of the document is reached.

Open strings can not start or end with the whitespace character. However, whitespace characters within the strings are preserved. The quotation characters ( " U+0022 or ' U+0027) do not require to be escaped.

An Open String Walkthrough

A simple open string. Notice it is not enclosed in any sort of enclosures.

John Doe

Quotes in the open strings don't cause termination.

Peter D'mello 

The following object contains three open string Unicode values.

जॉन डो, Wow Great, 😃

To create a multiline string, you don't need any escaping mechanism. In the following case, a string is spread over the five lines.

Lorem ipsum dolor sit amet consetetur sadipscing elitr sed 
diam nonumy eirmod. 

Tempor invidunt ut labore et dolore magna aliquyam erat 
sed diam voluptua

Escaping

In order to keep things simple, the open string format does not support character escaping. If the text does not fit into open string format, other formats such as regular string can be used.

In some cases (such as representing regular expressions) open string or regular string is not quite an efficient way to represent the string. Raw strings encapsulate the series of Unicode characters inside a single quote (' U+0027) character.

Use Cases

The raw string is used for representing text with complex characters. The following string represents an application path in the windows directory, it uses both characters i.e colon and backslash which makes it difficult to be represented using open strings and regular strings format.

'C:\program files\example\app.exe'

Raw strings format simplifies the complex string representation and enhances the readability by preventing escapes. The following regular expression is much more readable if represented using raw string format.

'^(19|20)\d\d([- /.])(0[1-9]|1[012])\2(0[1-9]|[12][0-9]|3[01])$'

As with other format of strings, raw strings can also deligently handle Unicode characters.

'जॉन डो', 'Can contain Ucharacters 😃'

Raw string preserves structural characters, newline characters, and whitespace while parsing

'A Unicode string (😃) which does not force you to escape
characters like \, \n or anything except a single quote 
char ''.'

Escaping

The Raw string does not support the character escaping with a reverse solidus. Every character inside the raw string is treated as exactly the same including reverse solidus (\ U+005C).

Only the single quote within the Raw string must be escaped to avoid string termination. To avoid the string termination, single quotes within the raw strings must be escaped using double single quotes.

'Jonas D''costa'

Open strings are a simpler and beautiful way to represent textual data. In many cases, open strings do not fit. One such issue is, they can't start with whitespace, contain structural characters such as colon (:), comma (,), or supports complex escaping. Regular strings solve this problem by letting the user enclose the string in the quotation marks (" U+0022). This makes a regular string look similar to the strings found in most programming languages.

A Regular String Walkthrough

Some of the examples of regular strings are...

"John Doe"

A string with leading and trailing white spaces

"   John Doe   "

A single quote within the regular strings doesn’t cause termination.

"Peter D'mello " 

Regular string represented using the Unicode characters.

"जॉन डो", "Can contain unicode characters 😃"

The new-line characters such as line-feed and carriage-return in the string are preserved while parsing.

"Lorem ipsum dolor sit amet consetetur sadipscing 
elitr sed diam nonumy eirmod. 

Tempor invidunt ut labore et dolore magna aliquyam 
erat sed diam voluptua"

Escaping

Any character may be escaped using a regular string. The Regular String format support escaping any characters. The control characters and the double quotes inside the string must be escaped.

If the character is among the special code point

All the characters in the special code point shown in the table below may be escaped by placing Reverse solidus (\ U+005C) before that character. If the character is among the special code points and it is a double quotation mark (" U+0022) then it must be escaped.

Escaping special code point characters

Characters that are not included in the above table will not be escaped by the parser by placing Reverse solidus. In the following example, character a will not be escaped as it is not a special code point character. Similarly, character u will not be escaped as it is not followed by a four-digit hexadecimal number.

"\bmax"  # b will be escaped
"\fmax"  # f will be escaped
"\rmax"  # r will be escaped
"\amax"  # a will not be escaped 
"\umax"  # u will not be escaped

"Some special chars such as \n \t can be escaped"

In the above example, \n and \t will be escaped as it is among the special code point characters.

"She said, \"I Love it\""

A double quotation mark within the string must be escaped as shown in the above example because it will cause the string termination.

A backslash before any non-escapable character will not have any effect! In the following example, both values represent "John Doe".

"\John Do\e", "John doe"

If the character is in the range from (U+0000) to (U+00FF)

If the character is in the Basic Latin Unicode character range i.e from (U+0000) to (U+00FF) then it may be escaped by representing it as the reverse solidus followed by the lower case letter "x", followed by two digits hexadecimal number.

"\x3A"
"\x7F"

If the character is in the range from (U+0000) to (U+FFFF)

If the character is in the range i.e from (U+0000) to (U+FFFF), then it may be escaped by representing it as a reverse solidus followed by a lower case letter "u", followed by four hexadecimal digits. For example, "\u00AF".

"\uA45E" # valid
"\u00AF" # valid

If the character is out of the range from (U+0000) to (U+00FF)

String Comparison

The software implementation that parses the Internet Object text requires checking the string values in the objects and their members for equality. In order to achieve interoperability, the software implementations must ensure that the Unicode character units of the transformed textual data are compared code by code numerically with the other string. And at the same time, it must agree on all the cases of equality and inequality of two strings.

In order to compare two strings, the software implementations must first evaluate the strings and convert the escape characters into respective Unicode points and then compare them.

• "cafe\u0301" is the same as "café"

• "\u000A" is the same as "\n"

• "\x0A" is the same as "\n"

• "\ud83d\ude00" is the same as "😀"

• "\uD83D\uDE00" is the same as "😀"

• "\ud83d\udcaf" is the same as "💯"

• "\uD83D\uDCAF" is the same as "💯"

The Decimal or a base 10 number is the most common format that we use today. It contains an integer component, that may be prefixed with a minus sign (- U+002D) or plus sign (+ U+002B) and maybe followed by a fraction part and/or exponent part.

Some valid examples of decimal numbers are.

Overview

The BigInt type in Internet Object represents arbitrary-precision integers that can handle numeric values exceeding the limitations of standard 64-bit number representations. This makes BigInt essential for applications that need to process extremely large whole numbers with perfect precision, such as cryptographic operations, large-scale counting, or mathematical computations requiring unbounded integer arithmetic.

Unlike the regular Number type, which is limited to safe integers within approximately ±9 quadrillion (±2^53-1), BigInt can represent integers of arbitrary length, ensuring that large numerical operations remain exact regardless of magnitude.

Representation and Syntax

A BigInt value in Internet Object is represented as an integer with the n suffix:

Basic BigInt Literals

Alternative Numeric Bases

BigInt values can be expressed in different numeric bases:

Key Concepts

Arbitrary Precision

BigInt values are not subject to the precision limitations that affect standard floating-point numbers:

Integer-Only Operations

BigInt values represent whole numbers only and do not support fractional components:

Type Compatibility

BigInt is a distinct data type and cannot be implicitly mixed with regular numbers:

Schema Definition and Validation

When used with Internet Object schemas, BigInt types can be defined and validated:

The BigInt type supports these validation properties:

• min/max: Validates value range

• choices: Limits valid values to a predefined set

• optional: Specifies if the field is optional

• null: Determines if null values are allowed

Operations and Behavior

Arithmetic Operations

BigInt supports these arithmetic operations:

Bitwise Operations

BigInt supports standard bitwise operations:

Comparisons

BigInt values can be compared as expected:

Use Cases and Examples

BigInt types are particularly valuable in:

1. Cryptography - key generation, hash calculations

2. Large-scale counting - web analytics, statistics

3. Financial ledgers - tracking very large monetary amounts

4. Mathematical computations - number theory, combinatorial calculations

5. IDs and timestamps - high-precision time tracking, unique identifiers

Cryptographic Example

Analytics Counter Example

Best Practices

1. Use for Whole Numbers Only: BigInt is designed for integer operations and does not support fractional values.

2. Consider Performance Implications: BigInt operations may be slower than standard number operations, especially for very large values.

3. Explicit Type Conversions: When interfacing with systems that don't support BigInt, explicitly convert values to ensure compatibility.

4. Range Constraints: Even though BigInt can represent arbitrarily large values, consider setting practical min/max limits in schemas to prevent excessive resource usage.

5. Avoid Mixing with Regular Numbers: Maintain type consistency by not mixing BigInt with standard numbers in operations.

Technical Considerations

When implementing or working with BigInt values, keep these points in mind:

1. Memory Usage: BigInt values can consume significantly more memory than standard numbers, especially for very large values.

2. Serialization: When serializing to formats that don't natively support BigInt (like standard JSON), values must be represented as strings or custom formats.

3. Integer Division: Division with BigInt always produces integer results (truncated toward zero), which may require special handling for fractional calculations.

4. No Decimal Point: BigInt does not support decimal points or fractional values; for such needs, use the Number or Decimal types.

5. Implementation Model: Many BigInt implementations use a variable-length sequence of bits to represent integers of arbitrary size, providing theoretical support for numbers limited only by available memory.

Overview

The Decimal type in Internet Object provides a fixed-precision decimal representation designed for applications that require exact numeric values, especially when dealing with financial calculations or other scenarios where floating-point precision issues could lead to significant errors.

Unlike standard floating-point numbers (which may suffer from approximation issues), Decimal types store exact numeric values with a defined precision and scale, ensuring accurate and predictable arithmetic operations.

Representation and Syntax

A Decimal value in Internet Object is represented as a number with the m suffix:

123.45m
-98765.4321m
0.000123m

Basic Decimal Literals

// Simple decimal values
123m         // Integer decimal
123.45m      // Fractional decimal
0.001m       // Leading zeros
-789.01m     // Negative decimal

Scientific Notation

Decimal values also support scientific notation:

1.23e2m      // Equivalent to 123m
1.23e-2m     // Equivalent to 0.0123m
5e3m         // Equivalent to 5000m

Key Concepts

Precision and Scale

Each Decimal value is defined by two key properties:

• Precision (M): The total number of significant digits in the number

• Scale (D): The number of digits after the decimal point

For example, in 123.45m:

• The precision is 5 (total digits: 1,2,3,4,5)

• The scale is 2 (decimal digits: 4,5)

Fixed-Precision Arithmetic

Decimal values maintain their precision throughout operations, making them ideal for financial calculations where exact values are required:

// With regular floating-point:
0.1 + 0.2 ≈ 0.30000000000000004 // Approximation error

// With decimal type:
0.1m + 0.2m = 0.3m // Exact representation

Rounding Behavior

When precision or scale constraints require rounding:

1. If a value has more decimal places than the specified scale, it's rounded using the "half up" rounding method (≥ 5 rounds up)

2. If the rounded value would exceed the precision, a validation error is raised

Schema Definition and Validation

When used with Internet Object schemas, decimal types can be precisely defined and validated:

{
  amount: {
    type: decimal,
    precision: 15,    // Total digits
    scale: 4,         // Decimal places
    min: 0,           // Minimum value constraint
    max: 10000.0000   // Maximum value constraint
  }
}

The decimal type supports various validation properties:

• precision: Restricts the total number of significant digits

• scale: Controls the number of decimal places

• min/max: Validates value range

• choices: Limits valid values to a predefined set

• optional: Specifies if the field is optional

• null: Determines if null values are allowed

Operations and Behavior

Type Conversion

Internet Object provides mechanisms for converting between different numeric types:

// Converting to decimal from other formats
decimal("123.45")    // From string
decimal(123.45)      // From number

Decimal Conversion and Compatibility

Decimal values with different precision/scale configurations can be converted to ensure compatibility during operations:

// Converting between decimal configurations
123.45m.convert(8, 4)    // Converts to precision=8, scale=4 → 123.4500m
123.4567m.convert(5, 2)  // Converts to precision=5, scale=2 → 123.46m (rounds)

The convert method allows:

• Increasing scale: Adds zeros to the right (e.g., 123.45 → 123.4500)

• Decreasing scale: Truncates with rounding (e.g., 123.456 → 123.46)

• Adjusting precision: Ensures the new value fits within specified constraints

When working with decimals of different formats:

1. Two decimals must have the same precision and scale for direct comparison.

2. Decimals with different configurations must be converted to a common format.

3. Conversion may fail if the target precision cannot accommodate the value.

It's important to note that both the integer and fractional parts of a decimal must fit within the precision minus scale. For example:

// This would fail because 12345 requires 5 digits, but precision(4)-scale(2)=2 only allows 2 integer digits
12345.67m.convert(4, 2)  // Error: Value exceeds precision

// This works because the value fits within the precision constraints
123.45m.convert(5, 2)    // 123.45m (no change needed)

Example of decimal comparison with conversion:

let price = 19.95m;        // precision=4, scale=2
let newPrice = 19.95000m;  // precision=7, scale=5

// Cannot compare directly (different precision/scale)
// Must convert to a common format first:
price.convert(7, 5) == newPrice;  // true, both as precision=7, scale=5

Comparisons

Decimal values can be compared with other decimal values as expected:

12.34m > 12.33m    // true
12.34m == 12.34m   // true
12.34m < 12.35m    // true

Use Cases and Examples

Decimal types are particularly valuable in:

1. Financial applications - currency calculations, banking, accounting

2. Scientific computing - when exact decimal representation matters

3. Regulatory compliance - when calculations must be exactly reproducible

4. Monetary APIs - for consistent data exchange with financial systems

Financial Transaction Example

{
  transactionId: "TX12345",
  amount: 1299.99m,
  currency: "USD",
  tax: 78.00m,
  total: 1377.99m
}

Scientific Measurement Example

{
  experiment: "E-201",
  measurement: 0.00000123m,
  uncertainty: 0.00000002m,
  unit: "meters"
}

Best Practices

1. Explicitly Define Precision/Scale: Always specify the precision and scale for decimal types in schemas to ensure data consistency.

2. Use for Financial Data: Prefer decimal type over floating-point for monetary values to avoid rounding errors.

3. Consider Storage Implications: Decimal types require more storage than standard numbers due to their exact representation.

4. Range Validation: Use min/max constraints to ensure decimal values stay within expected business bounds.

5. Manage Precision/Scale When Combining Decimals: When performing operations across decimals with different precision/scale, explicitly convert them to a common format or ensure your target format can accommodate the result.

Technical Considerations

When implementing or working with decimal values, keep the following points in mind:

1. Precision: Decimal types preserve exact values without rounding errors, unlike floating-point numbers. This makes them ideal for financial and high-precision applications.

2. Interoperability: Decimal types are compatible with database systems and financial applications that require exact decimal arithmetic, ensuring seamless data exchange and integration.

3. Performance: While operations on decimal values may be slower than native floating-point operations, they provide guaranteed precision, which is crucial for applications where accuracy is paramount.

4. Consistency: Decimal calculations produce the same results regardless of the platform or implementation, ensuring reliable and predictable outcomes across different environments.

5. Implementation Model: The underlying implementation of the Decimal type uses a coefficient-exponent model, similar to database systems like SQL Server and Oracle. This provides a strong basis for interoperability with enterprise data systems and ensures consistent behavior.

Specs coming up soon!

Internet Object supports binary non-fractional integer values. The hexadecimal integer may be prefixed with optional plus or minus signs and must start with "0b" or "0B" literal characters. It is then followed by one or more 0 or 1 digits.

Some examples of Binary Integer values...

0B01100010, 0b010010010, +0B1010101010, -0b0111111

Internet Object supports octal non-fractional integer values. The octal integer may be prefixed with optional plus or minus signs and must start with 0c or 0C literal characters. It is then followed by one or more octal digits.

Some of the octal integer values are...

Internet Object supports NaN (Not a Number) and Infinity values. Not a number and the Infinity must be represented as literal characters NaN and Inf. The Infinity value may be prefixed with an optional plus or minus sign.

Valid Values

Internet Object supports hexadecimal non-fractional integer values. The hexadecimal integer may be prefixed with optional plus or minus signs and must start with "0X" or "0x" literal characters. It is then followed by one or more hexadecimal digits. The hexadecimal number from A through F can be lower or upper case.

Some examples of Hexadecimal Integer values...

0XFF00FF, 0xff00ff, +0XAA21FF, -0X010408

Internet Object supports single-line comments. Internet Object comments start with a hash sign (# U+0023) and end when the line terminates. You can place comments anywhere inside the document, and any code written after the hash sign on the same line is ignored by the parser.

# Internet Object Document: Personnel Records

# Address schema definition
~ $address: {street:string, zip:{string, maxLength:5}, city:string}

# Person schema definition
~ $schema: {
    name:string,               # Individual's full name
    age:int,                   # Age in years
    homeAddress?: $address,    # Optional home address
    officeAddress?: $address   # Optional office address
}

---
# Personnel Records
~ John Doe, 25, {Queens, 50010, NewYork}, {Bond Street, 50001, NewYork}
~ Jane Doe, 20, {Queens, 50010, NewYork}, {Bond Street, 50001, NewYork}

Usage and Benefits

Comments in Internet Object serve multiple purposes:

1. Document Structure Elucidation: Providing context for different sections of the document.

2. Schema and Field Description: Offering explanations for data structures and individual fields.

3. Metadata Provision: Including information about the document itself, such as version or purpose.

4. Code Segmentation: Improving readability by logically separating different parts of the document.

Best Practices:

• Be Clear and Concise: Use comments to clarify complex sections, but avoid stating the obvious.

• Keep Comments Updated: Ensure that comments reflect any changes in the code to prevent misinformation.

• Avoid Overusing: Excessive comments can clutter the document. Use them judiciously to highlight important information.

• Place Comments Appropriately: Position comments near the relevant code or data structure to maintain context.

Effective use of comments enhances document comprehensibility and maintainability by providing contextual information and explanations for data structures and values.

Boolean values are represented using T and F. true and false. The true values can be represented using T or true keywords. The false values can be represented using F or false keywords.

In the following example, isActive is of boolean type and will accept only boolean values i.e T, F, true , false.

name: string, age: number, isActive: bool
---
~ John Doe, 30, T
~ Jane Doe, 25, F,
~ Peter Peterson, 28, false
~ Jack Jackson, 30, true

The null values are represented using N and null keywords.

In the following example, the null value can be passed for age.

The Internet Object format uses UTF-8 as the default and mandatory encoding. This ensures that all implementations can reliably read and write text consistently. While you can use other encodings like UTF-16, UTF-32, or ASCII, keep in mind that not all systems might support them.

Byte Order Mark (BOM)

Adding a Byte Order Mark (U+FEFF) at the start of your Internet Object text won’t cause issues—the parser will simply treat it as a space. However, it's generally a good idea to omit the BOM unless you have a specific reason to include it.

A Collection is a record aggregator that allows sending multiple records over the internet without repetitively defining a key-value pair. The Collection embeds more than one independent record in the IO document.

Benefits

Collection reduces the complexity of defining key-value pairs every time a record is sent over the internet. Thus, simplify application development by offering data parallelism and operational simplicity.

The collection permits an internet object document to have multiple records with different types and structures independent of each other.

Collection Structure

The Collection must be represented with the tidal sign (~ U+007E) followed by the object and separated by the whitespace as shown in the Collection structure.

The tidal sign enables the parser to identify the next record. Here is a code snippet that shows how to represent a collection.

# Representing a collection in the Internet Object document
~ Id, empName, age, department, Address  
---
# Following records represents Collection
~ 101, Thomas, 25, HR, {Bond Street, New York, NY} 
~ 102, George, 30, Sales, {Duke Street, New York, NY}

A collection may be created with or without explicitly defining schema definition for the records. However, it is always recommended to define a schema for the collections of records.

Simple Collection

A Simple Collection can be created in the data section of the Internet object document by prefixing each record with a tidal sign (~ U+007E). It enables the parser to identify the next record when multiple records are sent.

In the Simple Collection as the schema is not defined the type and the structure of collection records can differ.

# Creating a simple Collection
---
~ Ironman, 20, Male, {Bond Street, New York, NY}
~ Spiderman, 25, Male, {Duke Street, New York, NY}, cool

Explicit Collection

An Explicit Collection is created by explicitly defining the schema for the collection of records. Prefixing schema with the tidal sign (~ U+007E) enables the parser to understand the multiple records that may be sent according to a particular schema definition.

# Creating an Explicit Collection
~ $address: {street, city, state}
~ $schema: { 
            name: string, 
            age: {int, min:28}, 
            gender: {string, choices: [Male, Female]}, 
            $address
            }
---
~ Ironman, 20, Male, {Bond Street, New York, NY}
~ Spiderman, 25, Male, {Duke Street, New York, NY}
~ Wonderwoman, 25, Female, {Z street, San Francisco, California}

Here in the code snippet, multiple records are passed in the data section of the document using Collections.

For frequently passing object data between the system over the internet there is a need to stream objects over a single connection.

As the collection enables embedding more than one independent record in the document because of its nature it allows streaming real-time data changes. So that the application can react immediately to the changing events in real-time.

~ $address: {street, city, state}
~ $schema: {name, $address, isActive, * } 

Multiple records can be sent in batches after validating with the schema as,

~ John Doe, {Red Street, Phoenix, US}, T              # valid 
~ Alex, {Carnival Street, San Fransisco, US}, T, Male # valid
~ Bob, {Green Street, Los Angeles, US}, T, Cool       # valid

# First batch of the record 

After you have received the first batch of records, the collection allows you to receive more records for the same collection separately. The Internet Object processor should take care of merging the stream of data into the same collection.

~ Oliver, {Alpha Street, Denver, Colorado}, T, "77858"# valid 
~ Max, {Hill Street, Jacksonville, Florida}, T, 789445 # valid
~ James, { Elm Street, Los Angeles, California}, T, 24 # valid

# Second batch of the record 

Collection Without schema

If the schema is not defined, the records in the collection can have a different structure from each other across the document. Here is the code snippet,

# Record in the Collection with different structure
---
~ John, 24, {X Street, New York, NY}        # valid
~ true, false                               # valid
~ a, b, c                                   # valid
~ 100, 56, abc                              # valid
~ test, one ,457                            # valid
~ XYZ, 99, female, {X Street, New York, NY} # valid

# Parsing a collection when the schema is not defined
---
~ John Doe, 20, female
~ true, false
~ marketing, 123, {Z street, Los Angeles, LA}

In the above example, the schema is not defined for the collection records so it will be parsed as,

# Parsing records in the collection to respective indices
{
"0": "John Doe";
"1": 20;
"2": "Female";
}
{
"0": true;
"1": false;
}
{
"0": "marketing";
"1": 123;
"2": {
      "0": "Z street";
      "1": "Los Angeles";
      "2": "LA";
     }
}

Empty Record in Collection

Sending an empty record is valid only if all the variables defined in the schema are either set to null or optional or both.

# Sending Empty record in the collection
~ A*, B*: {number, default:1}, C?
---
~ James, 36, Mumbai # valid
~ Viki              # valid
~                   # valid

Here in the above code snippet A and B is null and C is optional thus sending an empty record is valid. Because just sending a "~" means an empty object { }.

# Sending an empty record in the collection
~ $address: {street, city, state}
~ $schema: {
    A*, B?*: {
              number,
              default: 1,
              optional: true,
              null: true,
              max:100
              },
    C?: $address,
  }
---
~ James, 20, {X Street, New York, NY}# valid
~                                    # empty but valid
~ 100                                # valid
~ ABC                                # valid
~                                    # empty but valid
~ {Duck street, California, CA}      # valid

In the above example, A is null, B is null and optional and C is optional. So all the keys are either optional or null or both thus sending an empty record is valid. Because just sending a "~" means an empty object { }.

# Sending empty records
~ $schema: {
             name: string,
             age?*: {
                      type: int,
                      default: 1,
                      optional: true,
                      null: true,
                      max: 25
                     }
           }
---
~ John 25   #valid
~ William   #valid
~ Ronald    #valid
~           #invalid as name is not optional or null
~ George 20 #valid

In the above example, the invalid record fails while parsing as the name variable is not optional or null. On the other hand, the age variable is optional as well as null so it is valid to not pass any value for the age variable.

Handling Errors

The Collection enables the parser to parse the rest of the document even if the previous record fails to execute.

If the record fails while parsing, that record state becomes invalid and it does not stop parsing the rest of the document.

# Error Handaling in Collections
~ $address: {street, city, state}
~ $schema: {
             name: string,
             age: {int, max:25},
             $address,
           }
---
# collection enables paser to parse next record even if the
# previous record fails to execute
~ James, 20, {X Street, New York, NY}                # valid
~ Alex, 30, {Z Street, Los Angeles, California}      # invalid
~ Bob, 20, {Melrose Street, San Fransisco, California} # valid

The Internet Object document promotes reusability through variables. It allows defining variables that can be applied to simplify schema and definitions, obfuscate values, or reduce the data size. Every key defined in the definition section can be used as a variable.

Types of Variables

Internet Object variables can be categorized into two groups.

1. Value Variables

2. Schema Variables

Value Variable

The Value variables are used to directly access and reuse values.

# Defining value variables 
~ record: 123
~ y: yes
~ n: no
~ rgb: [red, green, blue]
---

In the above snippet records, y, n, and rgb are the value variables.

Schema Variable

The schema variables start with $sign and it is used to directly access and reuse schema.

# Defining schema variables
~ $address: {
    street: string, 
    zip: {string, maxLength: 5}, 
    city: string
  }
~ $person: {
    name: string,
    age: int,
    homeAddress?: $address,
    officeAddress?: $address
  }
 ---

In the above code snippet, the schema variable address is reused in another schema variable named person .

Advantages and Use Cases

Reuse Definitions

The value variables and schema variables enable the reuse of definition.

# Reusing value variable and schema variable
~ y: yes
~ n: no
~ rgb: [red, green blue]
~ $address: { 
            street: string, 
            zip: {string, maxLength: 5}, 
            city: string }
~ $person: {
    name: string,
    age: int,
    seniorCitizen: {choices: [$y, $n]}
    color: {string, choices: $rgb},
    homeAddress?: $address,
    officeAddress?: $address
       }
---
Spiderman, 25, $n, red, {Queens, 50010, New York}, {Bond Street, 50001, New York}

# Reusing value variable and schema variable
~ $person: {
    name: string,
    age: int,
    seniorCitizen: {choices: [yes, no]}
    color: {string, choices: [red, green, blue},
    homeAddress?: {
      street: string, 
      zip: {string, maxLength: 5}, 
      city: string
    },
    officeAddress?: {
      street: string, 
      zip: {string, maxLength: 5}, 
      city: string
    }
  }
---
Spiderman, 25, no, red, {Queens, 50010, New York}, {Bond Street, 50001, New York}

Obfuscate Data

Variables are also used for hiding critical information with modified content to enforce data protection and security.

The following example demonstrates how one can pass critical information over the internet using variables.

# Key saved on the client side
~ s: ghhhj456nhghhhy11569bbbgtxxcv123654897
~ a: jk889456llkhynnnk12364lkkkhuk4125336nk

The above code snippet represents secrectKey as s and activationKey as a saved on the client-side. This information is securely passed over the internet using variables as shown below,

# Information passed on the server
~ y: yes
~ n: no
~ rgb: [red, green blue]
~ $address: {city, zipCode, state} 
~ $person: {
             name, 
             age, 
             currentAddress: $address, 
             permanentAddress: $address
            }
~ $accountDetails: {
                    name,
                    phoneNo, 
                    currentAddress: $address, 
                    secretKey, activationKey
                    }
---
Spiderman, 7855423656,  {Queens, 50010, New York}, $s, $a

The receiver will receive the following information without compromising on data security.

# Information Received by the receiver
{
  name: "Spiderman",
  phoneNo: 7855423656,
  currentAddress:{
                   city: "Queens",
                   zipCode: 50010,
                   state: "New York"
                 }
secretKey: "ghhhj456nhghhhy11569bbbgtxxcv123654897",
activationKey: "jk889456llkhynnnk12364lkkkhuk4125336nk"      
}

Reduce data size

The use of variables helps to reduce the code size as it enables definition reuse that ultimately reduces bandwidth utilization.

#  Reduce the code length by facilating code reuse
~ rgb: [red, green blue]
~ $address: {
             street: string, 
             zip: { string, maxLength:5 }, 
             city: string 
             }
~ $accountDetails: {branchName:string, accountNo, IFSCcode}   
~ $person: {
             name: string,
             age: int,
             bankaccountInfo: $accountDetails,
             color: { string, choices: $rgb },
             homeAddress?: $address,
             officeAddress?: $address
            }
---
{ Spiderman, 25, red, 
   {Queens, 50010, New York}, 
     {Bond Street, 50001, New York}
     }

In the above code snippet, the schema variable address and accountDetails are used in the person schema definition. So, rather than creating a similar schema multiple times for address it can be created once and reused multiple times in the document.

Improves schema readability

Variables improve schema readability by grouping similar and reusable codes and limiting line length.

Apart from the schema, the IO document header can have definitions. The definitions allow you to define schema, variables, metadata, and much more. In essence, the definitions are the collection of key-value pairs, with the following structure.

The definition must start with a tidal symbol (~ U+007E) followed by a key-value pair. The key-value pair must be separated by a colon (: U+003A).

A definition walkthrough

Simple definitions

Simple definitions such as meta-data declaration can be written as shown in the code snippet below.

# The result meta-data
~ pageSize: 1
~ currentPage: 1
~ totalPages: 1
~ recordCount: 2
~ success: T
---
# The data
~ John Done, 25, {Bond Street, New York, NY}
~ Jane Doe, 20, {Bond Street, New York, NY}

Value and Schema definitions

Any value defined in the definition section can be used as a variable. The dollar $ prefix can be used to declare schema and/or consume the variable value. If the key starts with $ a sign the variable will be treated as a schema and handled likewise.

# Variables and schema definitions
~ y: yes # value var
~ n: no  # value var
~ $address: {street, city, state} # schema var
~ $schema: {name, age, $address, ready:{string, choices:[$y, $n]} # schema var
---
~ John Done, 25, {Bond Street, New York, NY}, $y
~ Jane Doe, 20, {Bond Street, New York, NY}, $y

Here in the code snippet, y: yes and n: no are used as value definition similarly keys in the schema prefixed with $ sign represents schema definitions.

The header section of the internet object document can have single or multiple schema definitions

In the above example, the schema definitions are created for reuse to improve the readability of a schema. The schema definition created for address is reused in the person schema definition.

Internet Object is a schema first format! The Internet Object schema defines the shape and structure of Internet Object documents and helps the developers and designers to create the object definitions in a simple, concise, clear, and human-readable way.

The schema asserts the shape of the IO objects and ensures the validity of the data during the serialization and deserialization process.

The schema may be attached and placed in the document header or kept separately. Internet Object schema provides a simple way to define the object structure!

In its simplest form, an object schema is just an object with a list of required members. The following example represents the schema with five keys i.e name, age, address, isActive, remark which are separated by ",".

#Schema with the valid representation
name, age, address, isActive, remark 

A schema can be embedded in the IO document header or defined independently. The following example shows the schema embedded into the document itself. The upper section declares the schema while the lower section contains the object.

# Schema with the valid representation
name, age, address, isActive, remark 
---
John Doe, 20, {Bond Street, New York}, T, Nothing

The schema in the previous example lacks the datatypes. Since the keys are not associated any data type, the default datatype is any. That means the value for the name field could be John Doe, T or 999 or anything.

We can attach types and sub-schema to the keys to add more constraints and clarity.

# Schema with the members associated with types
name: string,
age: int, 
address: {
  street: string, 
  city: string, 
  state: string
}, 
isActive: bool, 
remark: string

Here name, age, address, isActive, and remark are defined with the type string, int, object, bool and string respectively.

A string type can be defined with the members such as type, default, choices, pattern, minLen, maxLen, len, optional, and null . Schema of the string TypeDef should be written as,

TypeDef Schema

The TypeDef schema ensures the validity of string MemberDefs.

type

The first member of the typedef is type. The string can define with a type string or its derived types i.e email, url, datetime, date, time. Here the next snippet shows, how the string type and its derived types can be defined.

default

The second member in the string typedef is default . Here is how the default values can be defined for a string.

Rules for default:

1. The default value is applicable only if no other value is provided for the key.

2. If for a key, null is set to true then it must be replaced by its default value.

3. The default value when set must match with the data type of a key.

Choices

The next member in the string typedef is choices . If set, the choices must be an array of strings. Here the snippet shows how the choices can be added to member variables in a string so that it is restricted to the fixed set of available choices.

Pattern

The value of the pattern must be a String. The string value passed should be a valid Regular Expression. The data will be then validated according to the Regular Expression and passed accordingly. Regular Expression can be defined in the schema by using pattern in the schema of a string.

Different versions of schema can be created and executed for patterns in the programming environment. But to remain compatible with the host environment, it is better to stick to the constraints described below.

• A single Unicode character, other than the special characters specified below matches itself.

• (. U+002E): Matches any character except newline character (U+000A).

• (^ U+005E): Matches only at the start of the string.

• ($ U+0024): Matches only at the end of the string.

• (...): Assembles the sequence of regular expressions into a single regular expression.

• (| U+007C): Matches the regular expression either preceding or following with the "|" symbol.

• [abc]: Matches any of the characters enclosed by the square brackets.

• [a-z]: Matches the range of characters enclosed by the square bracket.

• [^abc]: Matches any character not in the list.

• [^a-z]: Matches any character out of the given range.

• (+ U+002B): repeats the preceding regular expression one or more times and is greedy as they match as many items as possible.

• (* U+002A): repeats the preceding regular expression zero or more times and greedy as they match as many items as possible.

• (? U+003F): makes the preceding regular expression optional. Greedy, matches zero or one preceding regular expression.

• +?, *?, ??: The *, +, and ? qualifiers are used to match as much text as possible which is not always desired.

• (?!x), (?=x): Negative and positive lookahead.

• {x}: Match exactly x occurrences of the preceding regular expression.

• {x,y}: Match at least x and at most y occurrences of the preceding regular expression.

• {x,}: Match x or more occurrences of the preceding regular expression.

• {x}?, {x,y}?, {x,}?: Lazy versions of the above expressions.

maxLen

The value of maxLen must be a non-negative integer. The string instance is valid only if the number of characters in the string will be less than or equal to the value of maxLen. Here is the snippet showing how to assign maxLen.

minLen

The value of minLen must be a non-negative integer. The string instance is valid only if the number of characters in the string will be greater than or equal to the value of minLen. Here is the snippet showing how to assign minLen.

len

The value of length represented as len must be a non-negative integer. The string instance is valid only if the number of characters in the string will be equal to the value of len. The code snippet shows how to assign len.

optional

The member of a string type can be set to optional. Here is the code snippet that demonstrates how a string can be set to optional.

null

A string when set to null: true will accept null values. The snippet below shows how to set a nullable string.

Examples

Here are some of the examples that demonstrate how to define string member definition.

In the above snippet, name can be kept optional and null. When no value is passed for the name then, its default value is set to anonymous. The name should be a string containing characters from a to z (upper or lower case) with a minimum length of 5 and a maximum length of 50.

Here the code snippet shows that the users can only select the department provided in choices i.e input is restricted to the set of available departments.

In the above code snippet, users can select the location provided in choices i.e the input is restricted to the set of available locations ( locations are enclosed in double-quotes to pass numeric data as string ).

Any data type is used to assign any type of value to the variables. It is useful in the case where either the actual type is not known or types are needed to be dynamically assigned. Thus for undefined type, the default type will be always set to any .

Any type can be defined with the members such as type, default, choices, anyOf, optional, and null . Schema of any TypeDef should be written as,

TypeDef Schema

type?     : {string, default: any, choices: [any]},
default?  : any,
choices?  : [any],
anyOf?    : [$memberDef],
null?     : {bool, default: F},
optional? : {bool, default: F}
---

The TypeDef schema ensures the validity of any MemberDefs.

type

As with most of the types in Internet Object, the first member of typedef is type. The next snippet shows different ways to define the members a, b, and c as any.

# Type set to any
a, b: any, c: {type: any}
---

default

The second member in the any typedef is default . Here is how the default values can be defined.

# Set default value for a and b
a?: {any, Monday}, b?: {any, default: N}
---

Here, the default value for a is Monday and default value for b is null .

choices

The choices restricts the member to be strictly bound with the unique constant values. If set, the choices must be an array of any type of value. The code snippet shows how choices can be defined for the any type.

# Defining choices for member
a: {any, choices: [1, One, 2, Two, 3, Three]}
---

anyOf

In some cases, a member must accept different kinds of values. Such as, a number could be a multiple of 3 or a multiple of 5; they could be a string or number but not that of other types; two different formats of the schema. TheanyOf allows schema designers to define members that can accept different kinds of constrained values. It accepts an array of MemberDef and/or schema and types.

member: {any, anyOf?: [$memberDef, type, schema]}
---

This snippet explains how a , b and c can accept various kinds of values.

# Can accept multipole of 5 or multiple of 3
a: {any, anyOf: [{int, multipleOf: 5}, {int, multipleOf: 3}]},

# Can accept string or number values
b: {any, anyOf: [string, number]

# Can accept any of two type of address!
c: {any, anyOf: [{city, state}, {street, city, state}]}
---

null

When set null to true, a member can accept null values. Here are some of the ways through which a member of any type can accept null values.

# Set default value of a, b, c, d to null
a*, b*: any, c: {any, null: T}, d: {any, null: true}
---

optional

When set optioanl to true, a member can be marked as optional. Here are some of the ways through which a member of any type can be made optional.

# Set a, b, c, and d to optional
a?, b?: any, c: {any, optional: T}, d: {any, optional: true}
---

Examples

Some of the valid examples of members with any are...

# The members a, b, and are by default assigned the any type.
a, b, c

# The name is any and default value is null
name: {any, N}

# The numWord can accept any string or number value.
numWord: {any, anyOf: [string, number]} 

In its simplest form, an object schema can be just a set of keys separated by a comma ",". Such schema ensures that the object has values for all the required keys. Schema can be defined in the header section of the Internet Object document or maintained separately. The following example shows how easy to create a basic schema-first Internet Object document.

id, department, employeeId, address # The schema
---
02, Marketing, 255, { X Street, New York, NY } # The object

Schema with Types

name: string, 
age: int, 
address: {
  street: string, 
  city: string, 
  state: string,
  zipCode: int
}, 
isActive: bool, 
remark: string

The MemberDefs

Defining members with simple types are good in some cases, however, they do not provide a mechanism to create complex constraints. In such cases, the MemberDefs (The member's definitions) are used to create complex type definitions. The memberDefs are objects designed to impose complex constraints on the associated member. For example, the following schema allows the member phrases to accept string values only if the length is between 20 and 200 characters. It will invalidate the value when the constraint is not match.

phrase: {string, minLen: 20, maxLen: 200}
---
~ A quick brown fox jumps over the lazy dog # Valid value
~ Hello world! # Invalid, len < 20

Meberdef is generally represented by enclosing data types or schema followed by optional comma-separated-values, and conditions: value pairs all inside the curly braces as shown below.

The following example represents a schema with MemberDef.

name: {string,  min: 20, max: 100},
age: {int, max: 20},           
gender: {string, choices: [Male, Female, NotDisclosed]}

In the above example, the schema has a complex type definition which is created using MemberDef. The variable name has type, min and max as its members. The variable age has type, and max as its members. Similarly, gender has type and choices as its members.

The above representations are valid and will successfully validate the following object.

Jane Doe, 20, Male

Schema with nested objects and arrays

An object schema can be defined with the child objects and arrays schema. The child object defined in the schema definition must be enclosed inside the curly braces.

 { 
    name: string, 
    age: int, 
    address: {
               street: string, 
               city : string,
               state: string, 
               zip: int
               },        
     gender: string,
     skills: {
              default: Null, 
              array: [strings]
              }
  }

In the above example, the key variable address has a child object having four keys i.e street, city, state and zip. Similarly, the key skills is defined with an array schema having a default value null and array of type strings.

The above representations are valid and will successfully validate the following object.

{
  Jane Doe, 20, { X Street, New York, NY, 10005 }, 
  [Leadership, Problem-Solving]
}

Multiple Schemas and Schema Reusability

Internet object documents may have multiple schemas. The document containing multiple schemas can be reused and/or nested to reduce code length in such a case the schema must be defined before use.

The schema must be prefixed with a dollar sign ($ U+0024) before reusing the schema in the multiple schemas definitions. So that the parser will identify the schema variable and will map values to the respective keys according to the Schema Definition.

~ $address: {street, city, state}
~ $personalDetails: {
             name: string, 
             age: int, 
             $address, 
             isActive: bool, 
             remark: string
            }

In the above example, the address schema is represented using a dollar sign ($ U+0024), so that it can be reused inside another schema personalDetails.

~ $address: {city, zipCode, state} 
~ $person:{
            name, 
            age, 
            currentAddress: $address, 
            permanentAddress: $address
          }
~ $accountDetails: { 
                     personDetails: $person, 
                     AccountNo:int
                   }
---

In the above example, the header section has three schemas i.e address, person and accountDetails . The address schema is used inside the person schema and person schema is used inside the accountDetails schema making the schema representation nested.

The above representations are valid and will successfully validate the following object.

{
  Jane Doe, 20, { X Street, New York, NY, 10005 },
  { Hills Street, Lake Peekskill, NY, 10537 }, 12344564567
}

Default Schema

To map values from the data section to the variables in the header, the parser will first check the header section for the default schema definition that starts with the word "schema". Once the parser finds out the default schema, all values will be checked by matching them with the variables. If they are matched then they will be mapped to the schema variables.

If the values are not matched with the variables the parser will check for the other schema definition.

~ $address: {street, city, state}
~ $Schema: {
             name: string, 
             age: int, 
             $address, 
             isActive: bool, 
            }
 ---
 John Doe, 25, {X-street, California, US}, T

In the above example, the parser will first check the header part for the schema definition with the word "schema". Once it is found it will start validating all the variables in the schema i.e name, age, address, and isActive to the values in the data section if it matches then it will be mapped successfully.

If the header section of the document does not contain schema definition and only contain the value definition and/or meta-data as shown in the example below. The values will be mapped to the positioned variables after getting the values from the value variable.

~ pageSize: 1
~ currentPage: 1
~ totalPages: 1
~ y: Yes
~ n: No
 ---
~ John Doe, 25, $y, {X-street, California, US}
~ Bob Redwood, 36, $n, {blue-street, New Jersey, US}