Marshalling (computer science)

In computer science, marshalling or marshaling (US spelling) is the process of transforming the memory representation of an object into a data format suitable for storage or transmission, especially between different runtimes.^{[ citation needed ]} It is typically used when data must be moved between different parts of a computer program or from one program to another.

Marshalling simplifies complex communications, because it allows using composite objects instead of being restricted to primitive objects .

Comparison with serialization

Marshalling is similar to or synonymous with serialization, although technically serialization is one step in the process of marshalling an object.

• Marshalling is describing the overall intent or process to transfer some live object from a client to a server (with client and server taken as abstract, mirrored concepts mapping to any matching ends of an arbitrary communication link ie. sockets). The point with marshalling an object is to have that object that is present in one running program be present in another running program; that is, an object on the client should be transferred to the server, which is a form of reification allowing the object’s structure, data and state to transit from a runtime to another, leveraging an intermediate, serialized, "dry" representation (which is of second importance) circulating onto the communication socket.
• Serialization does not necessarily have this same intent, since it is only concerned about transforming data to generate that intermediate, "dry" representation of the object (for example, into a stream of bytes) which could then be either reified in a different runtime, or simply stored in a database, a file or in memory.

Marshalling and serialization might thus be done differently, although some form of serialization is usually used to do marshalling. ^[1]

The term deserialization is somewhat similar to un-marshalling a dry object "on the server side", i.e., demarshalling (or unmarshalling) to get a live object back: the serialized object is transformed into an internal data structure, i.e., a live object within the target runtime. It usually corresponds to the exact inverse process of marshalling, although sometimes both ends of the process trigger specific business logic.

The accurate definition of marshalling differs across programming languages such as Python, Java, and .NET, and in some contexts, is used interchangeably with serialization.

Marshalling in different programming languages

To "serialize" an object means to convert its state into a byte stream in such a way that the byte stream may be converted back into a copy of the object, which is unmarshalling in essence. Different programming languages either make or don’t make the distinction between the two concepts. A few examples:

In Python, the term "marshal" is used for a specific type of "serialization" in the Python standard library ^[2] – storing internal python objects:

The marshal module exists mainly to support reading and writing the “pseudo-compiled” code for Python modules of .pyc files.

…

If you’re serializing and de-serializing Python objects, use the pickle module instead

— The Python Standard Library ^[3]

In the Java-related RFC   2713, marshalling is used when serializing objects for remote invocation. An object that is marshalled records the state of the original object and it contains the codebase (codebase here refers to a list of URLs where the object code can be loaded from, and not source code). Hence, in order to convert the object state and codebase(s), unmarshalling must be done. The unmarshaller interface automatically converts the marshalled data containing codebase(s) into an executable Java object in JAXB. Any object that can be deserialized can be unmarshalled. However, the converse need not be true.

To "marshal" an object means to record its state and codebase(s) in such a way that when the marshalled object is "unmarshalled," a copy of the original object is obtained, possibly by automatically loading the class definitions of the object. You can marshal any object that is serializable or remote (that is, implements the java.rmi.Remote interface). Marshalling is like serialization, except marshalling also records codebases. Marshalling is different from serialization in that marshalling treats remote objects specially.

…

Any object whose methods can be invoked [on an object in another Java virtual machine] must implement the java.rmi.Remote interface. When such an object is invoked, its arguments are marshalled and sent from the local virtual machine to the remote one, where the arguments are unmarshalled and used.

— Schema for Representing Java(tm) Objects in an LDAP Directory (RFC 2713) ^[4]

In .NET, marshalling is also used to refer to serialization when using remote calls:

When you marshal an object by value, a copy of the object is created and serialized to the server. Any method calls made on that object are done on the server

— How To Marshal an Object to a Remote Server by Value by Using Visual Basic .NET (Q301116) ^[5]

Usage and examples

Marshalling is used within implementations of different remote procedure call (RPC) mechanisms, where it is necessary to transport data between processes and/or between threads.

In Microsoft's Component Object Model (COM), interface pointers must be marshalled when crossing COM apartment boundaries. ^{[6] [7]} In the .NET Framework, the conversion between an unmanaged type and a CLR type, as in the P/Invoke process, is also an example of an action that requires marshalling to take place. ^[8]

Additionally, marshalling is used extensively within scripts and applications that use the XPCOM technologies provided within the Mozilla application framework. The Mozilla Firefox browser is a popular application built with this framework, that additionally allows scripting languages to use XPCOM through XPConnect (Cross-Platform Connect).

Example

In the Microsoft Windows family of operating systems the entire set of device drivers for Direct3D are kernel-mode drivers. The user-mode portion of the API is handled by the DirectX runtime provided by Microsoft.

This is an issue because calling kernel-mode operations from user-mode requires performing a system call, and this inevitably forces the CPU to switch to "kernel mode". This is a slow operation, taking on the order of microseconds to complete. ^[9] During this time, the CPU is unable to perform any operations. As such, minimizing the number of times this switching operation must be performed would optimize performance to a substantive degree.

Linux OpenGL drivers are split in two: a kernel-driver and a user-space driver. The user-space driver does all the translation of OpenGL commands into machine code to be submitted to the GPU. To reduce the number of system calls, the user-space driver implements marshalling. If the GPU's command buffer is full of rendering data, the API could simply store the requested rendering call in a temporary buffer and, when the command buffer is close to being empty, it can perform a switch to kernel-mode and add a number of stored commands all at once.

Formats

Marshalling data requires some kind of data transfer, which leverages a specific data format to be chosen as the serialization target.

XML vs JSON vs…

XML is one such format and means of transferring data between systems. Microsoft, for example, uses it as the basis of the file formats of the various components (Word, Excel, Access, PowerPoint, etc.) of the Microsoft Office suite (see Office Open XML).

While this typically results in a verbose wire format, XML's fully-bracketed "start-tag", "end-tag" syntax allows provision of more accurate diagnostics and easier recovery from transmission or disk errors. In addition, because the tags occur repeatedly, one can use standard compression methods to shrink the content—all the Office file formats are created by zipping the raw XML. ^[10] Alternative formats such as JSON (JavaScript Object Notation) are more concise, but correspondingly less robust for error recovery.

Once the data is transferred to a program or an application, it needs to be converted back to an object for usage. Hence, unmarshalling is generally used in the receiver end of the implementations of Remote Method Invocation (RMI) and Remote procedure call (RPC) mechanisms to unmarshal transmitted objects in an executable form.

JAXB

JAXB or Java Architecture for XML Binding is the most common framework used by developers to marshal and unmarshal Java objects. JAXB provides for the interconversion between fundamental data types supported by Java and standard XML schema data types. ^[11]

XmlSerializer

XmlSerializer is the framework used by C# developers to marshal and unmarshal C# objects. One of the advantages of C# over Java is that C# natively supports marshalling due to the inclusion of XmlSerializer class. Java, on the other hand requires a non-native glue code in the form of JAXB to support marshalling. ^[12]

From XML to an executable representation

An example of unmarshalling is the conversion of an XML representation of an object to the default representation of the object in any programming language. Consider the following class:

publicclassStudent{privatecharname[150];privateintID;publicStringgetName(){returnthis.name;}publicintgetID(){returnthis.ID;}voidsetName(Stringname){this.name=name;}voidsetID(intID){this.ID=ID;}}

• XML representation of a specific Student object:

<!-- Code Snippet 1 --><?xml version="1.0" encoding="UTF-8"?><studentid="11235813"><name>Jayaraman</name></student><studentid="21345589"><name>Shyam</name></student>

• Executable representation of that Student object:

// Code Snippet 2Students1=newStudent();s1.setID(11235813);s1.setName("Jayaraman");Students2=newStudent();s2.setID(21345589);s2.setName("Shyam");

Unmarshalling is the process of converting the XML representation of Code Snippet 1 to the default executable Java representation of Code Snippet 2, and running that very code to get a consistent, live object back. Had a different format been chosen, the unmarshalling process would have been different, but the end result in the target runtime would be the same.

Unmarshalling in Java

Unmarshaller in JAXB

The process of unmarshalling XML data into an executable Java object is taken care of by the in-built Unmarshaller class. The unmarshal methods defined in the Unmarshaller class are overloaded to accept XML from different types of input such as a File, FileInputStream, or URL. ^[13] For example:

JAXBContextjcon=JAXBContext.newInstance("com.acme.foo");Unmarshallerumar=jcon.createUnmarshaller();Objectobj=umar.unmarshal(newFile("input.xml"));

Unmarshalling XML Data

Unmarshal methods can deserialize an entire XML document or a small part of it. When the XML root element is globally declared, these methods utilize the JAXBContext's mapping of XML root elements to JAXB mapped classes to initiate the unmarshalling. If the mappings are not sufficient and the root elements are declared locally, the unmarshal methods use declaredType methods for the unmarshalling process. These two approaches can be understood below. ^[13]

Unmarshal a global XML root element

The unmarshal method uses JAXBContext to unmarshal the XML data, when the root element is globally declared. The JAXBContext object always maintains a mapping of the globally declared XML element and its name to a JAXB mapped class. If the XML element name or its @xsi:type attribute matches the JAXB mapped class, the unmarshal method transforms the XML data using the appropriate JAXB mapped class. However, if the XML element name has no match, the unmarshal process will abort and throw an UnmarshalException. This can be avoided by using the unmarshal by declaredType methods. ^[14]

Unmarshal a local XML root element

When the root element is not declared globally, the application assists the unmarshaller by application-provided mapping using declaredType parameters. By an order of precedence, even if the root name has a mapping to an appropriate JAXB class, the declaredType overrides the mapping. However, if the @xsi:type attribute of the XML data has a mapping to an appropriate JAXB class, then this takes precedence over declaredType parameter. The unmarshal methods by declaredType parameters always return a JAXBElement<declaredType> instance. The properties of this JAXBElement instance are set as follows: ^[15]

JAXBElement Property	Value
name	xml element name
value	instanceof declaredType
declaredType	unmarshal method declaredType parameter
scope	null (actual size is not known)

See also

• Free and open-source graphics device driver#Software architecture
• Component Object Model
• CORBA
• Pickle (Python)
• Protocol Buffers
• Java Architecture for XML Binding
• Calling convention

References

1. Jeffrey Hantin. "What is the difference between Serialization and Marshaling?". Stack Exchange Network. Retrieved 23 July 2021.
2. "marshal — Internal Python object serialization". Python Software Foundation. Retrieved 4 November 2016.
3. "marshal — Internal Python object serialization". Python Software Foundation. Retrieved 9 October 2019.
4. "Schema for Representing Java(tm) Objects in an LDAP Directory". IETF. October 1999. Retrieved 4 November 2016.
5. "How To Marshal an Object to a Remote Server by Value by Using Visual Basic .NET". Microsoft. July 2004. Archived from the original on 2004-11-15. Retrieved 4 November 2016.
6. "Apartments and COM Threading Models". Archived from the original on 2015-09-23. Retrieved 2009-06-19.
7. "CoInitializeEx function (COM)". Windows Desktop App Development. Retrieved 2013-02-22.
8. Interop Marshaling Overview
9. Code Quality: The Open Source Perspective.
10. What is a DOCX file? https://docs.fileformat.com/word-processing/docx/ Accessed Oct 13, 2020.
11. "Binding XML Schemas - The Java EE 5 Tutorial". docs.oracle.com. Retrieved 2016-09-14.
12. "Using the XmlSerializer Class". msdn.microsoft.com. Retrieved 2016-09-23.
13. "Unmarshaller (JAXB 2.2.3)". jaxb.java.net. Retrieved 2016-09-14.
14. "JAXBContext (JAXB 2.2.3)". jaxb.java.net. Retrieved 2016-09-23.
15. "JAXBElement (JAXB 2.2.3)". jaxb.java.net. Retrieved 2016-09-23.