Xgrid

Last updated

Xgrid
Developer(s) Apple Inc.
Initial releaseJanuary 6, 2004 [1]
Final release
10.7.5 / October 4, 2012;12 years ago (2012-10-04)
Operating system Mac OS X
Platform Independent
Type Distributed computing
License Proprietary EULA [2]
Website Homepage at the Wayback Machine (archived January 2, 2006)

Xgrid is a proprietary grid computing program and protocol developed by the Advanced Computation Group subdivision of Apple Inc. [3]

Contents

It provides network administrators a method of creating a computing cluster, which allows them to exploit previously unused computational power for calculations that can be divided easily into smaller operations, such as Mandelbrot maps. The setup of an Xgrid cluster can be achieved at next to no cost, as Xgrid client is pre-installed on all computers running Mac OS X 10.4 to Mac OS X 10.7. The Xgrid client was not included in Mac OS X 10.8. The Xgrid controller, the job scheduler of the Xgrid operation, is also included within Mac OS X Server and as a free download from Apple. Apple has kept the command-line job control mechanism minimalist while providing an API to develop more sophisticated tools built around it.

The program employs its own communication protocol layered on top of a schema to communicate to other nodes. This communication protocol interfaces with the BEEP infrastructure, a network application protocol framework. Computers discovered by the Xgrid system, that is computers with Mac OS X's Xgrid service enabled, are automatically added to the list of available computers to use for processing tasks.

When the initiating computer sends the complete instructions, or job, for processing to the controller, the controller splits the task up into these small instruction packets, known as tasks. The design of the Xgrid system consists of these small packets being transferred to all the Xgrid-enabled computers on the network. These computers, or nodes, execute the instructions provided by the controller and then return the results. The controller assembles the individual task results into the whole job results and returns them to the initiating computer.

Apple modeled the design of Xgrid on the Zilla program, distributed with NeXT's OpenStep operating system application programming interface (API), which Apple owned the rights to. The company also opted to provide the client version of Mac OS X with only command-line functions and little flexibility, while giving the Mac OS X Server version of Xgrid a GUI control panel and a full set of features.

History

Zilla Zilla.gif
Zilla

Xgrid's origins can be traced back to NeXT's Zilla application created by scientist Richard Crandall in the late 1980s. Zilla was the first distributed computing program released on an end-user operating system and which used the idle screen-saver motif, a design feature since found in widely used projects such as Seti@Home. [1] [4] Zilla won the national Computerworld Smithsonian Award (Science Category) in 1991 for ease of use and good design. [5] Apple acquired Zilla along with the rest of NeXT in 1997, [6] and used Zilla as inspiration for Xgrid. [1] Xgrid Technology Preview 1 was released in January 2004, [7] [3] [8] followed by Technology Preview 2 in November 2004. Xgrid 1.0 was released as part of Mac OS X 10.4 Tiger in April 2005. [8]

Several organizations have adopted Xgrid in large international computing networks. One example of an Xgrid cluster is MacResearch's OpenMacGrid, where scientists can request access to large amounts of processing power to run tasks related to their research. [9] [10] Another was the now defunct Xgrid@Stanford project, which used a range of computers on the Stanford University campus and around the world [11] to perform biochemical research. [12] [13] [14]

In a report covering the announcement, Macworld cited Xgrid among the Unix features in "10 Things to Know about Tiger", calling it "handy if you work with huge amounts of experimental data or render complex animations". [15] After Xgrid's introduction in 2004, InfoWorld noted that it was a "'preview' grade technology" which would directly benefit from the Xserve G5's launch later that year. [16] InfoWorld commentator Ephraim Schwartz also predicted that Xgrid was an opening move in Apple's entry into the enterprise computing market. [17] [8]

Apple discontinued Xgrid with OS X 10.8 Mountain Lion, [18] along with dependent services such as Podcast Producer. [19] [20]

Protocol

Xgrid Protocol Xgridprotocol.svg
Xgrid Protocol

The Xgrid protocol uses the BEEP network framework to communicate with nodes on the network. The system's infrastructure includes three types of computers which communicate over the protocol:

A computer can act as one or all three of these components at the same time. The Xgrid protocol provides the basic infrastructure for computers to communicate, but is not involved in the processing of the specified calculation. [22] Xgrid is targeted towards time-consuming computations that can be easily segregated into smaller tasks, sometimes called embarrassingly parallel tasks. [23] This includes Monte Carlo calculations, 3D rendering and Mandelbrot maps. [22]

Within the Xgrid protocol, three types of messages can be passed to other computers on the same cluster: requests, notifications and replies. Requests must be responded to by the recipient with a reply, notifications do not require a reply, and replies are responses to sent messages. They are identified by their name, type (request/notification/reply) and contents. Each message is encapsulated in a BEEP message (BEEP MSG) and is acknowledged on receipt by an empty reply (RPY). [7] Xgrid does not leverage BEEPs message/reply infrastructure. Any received message which requires a response merely generates an independent BEEP message containing the reply. The Xgrid messages are encoded as dictionaries of key/value pairs which are converted to XML before being sent across the BEEP network.

Architecture

The architecture of the Xgrid system is designed around a job based system; the controller sends agents jobs, and the agents return the responses. The actual computation that the controller executes in an Xgrid system is known as a job. The job contains all the files required to complete the task successfully, such as the input parameters, data files, directories, executables and/or shell scripts, the files included in an Xgrid job must be able to be executed either simultaneously or asynchronously, or any benefits of running such a job on an Xgrid is lost. Once the job completes, the controller can be set to notify the client of the task's completion or failure, for example by email. The client can leave the network while the tasks are running. It can also monitor the job status on demand by querying the controller, although it cannot track the ongoing progress of individual tasks. [24] [8]

The controller is central to the correct function of an Xgrid, as this node is responsible for the distribution, supervision and coordination of tasks on agents. The program running on the controller can assign and reassign tasks to handle individual agent failures on demand. The number of tasks assigned to an agent depend on two factors: the number of agents on an Xgrid and the number of processors in each node. The number of agents on an Xgrid determines how the controller will assign tasks. The tasks may be assigned simultaneously for a large number of agents, or queued for a small number of agents. When a node with more than one processor is detected on an Xgrid, the controller may assign one task per processor; this only occurs if the number of agents on the network is lower than the number of tasks the controller has to complete. [24]

Xgrid is layered upon the Blocks Extensible Exchange Protocol (BEEP), an IETF standard comparable to HTTP, but with a focus on two-way multiplexed communication, such as that found in peer-to-peer networks. BEEP, in turn, uses XML to define profiles for communicating between multiple agents over a single network or internet connection. [25]

Xgrid agent software was available for Unix and Linux systems. [26]

Interface

Xgrid administration tool XgridAdminTool.jpg
Xgrid administration tool

While it is possible to access Xgrid from the command line, the Xgrid graphical user interface, a program bundled with Mac OS X Server and, as of March 2009, available online, is a much more efficient way of administering an Xgrid system. Originally, the Xgrid agent was included in all Mac OS X 10.4 Tiger installations but the GUI was reserved for users of Mac OS X Server. This decision limited the efforts of the computer community to embrace the platform. Eventually, Apple released the Mac OS X Server Administration Tools to the public, which included the Xgrid administration application bundled with Mac OS X Server. [27]

Despite the lack of a graphical controller interface in the standard (non-server) Mac OS X distribution, it is possible to set up an Xgrid controller via the command line tools xgridctl and xgrid. Once the Xgrid controller daemon is running, administration of the grid with Apple's Xgrid Admin tool is possible. [27] Some applications, such as VisualHub, provided Xgrid controller capability through their user interfaces. [28] [29]

See also

Notes

  1. 1 2 3 "Mac OS X Manual Page For xgrid(1)". Apple Inc. June 27, 2007. Archived from the original on July 25, 2008. Retrieved July 25, 2008.
  2. ↑ "Mac OS X 10.5 SLA" (PDF). Apple Inc. March 9, 2005. Archived from the original (PDF) on May 12, 2008. Retrieved June 12, 2008.
  3. 1 2 Cohen, Peter (March 29, 2004). "Apple posts Xgrid intro". Macworld. Archived from the original on October 6, 2022. Retrieved May 14, 2023.
  4. ↑ Norbury-Glaser, Mary (2006). "Build Your Own Supercomputer From Your Macs Laying Around: How to employ Tiger's Xgrid to build clusters and to contribute to grid/cluster projects". MacTech . Vol. 22, no. 1. Archived from the original on May 14, 2023. Retrieved May 14, 2023.
  5. ↑ "Zilla - Computerworld Honours Program Archives". Archived from the original on July 16, 2011. Retrieved February 27, 2009.
  6. ↑ Schlender, Brent; Tetzeli, Rick (2015). Becoming Steve Jobs: The Evolution of a Reckless Upstart into a Visionary Leader. Crown Business. pp. 190–197. ISBN   978-0-7710-7914-6.
  7. 1 2 Côté, Daniel (June 21, 2004). "XGrid agent for Unix architectures". Simple. Archived from the original on March 10, 2008. Retrieved July 18, 2008.
  8. 1 2 3 4 5 Hughes, Baden (2006). Building computational grids with Apple's Xgrid Middleware. Fourth Australasian Symposium on Grid Computing and e-Research (AusGrid 2006) and the Fourth Australasian Information Security Workshop (AISW 2006). doi:10.1145/1151828.1151835 (inactive November 1, 2024).{{cite conference}}: CS1 maint: DOI inactive as of November 2024 (link)
  9. ↑ Martellaro, John (February 13, 2007). "OpenMacGrid Computing Grid for Mac Users Announced". The Mac Observer. Archived from the original on August 30, 2008. Retrieved March 1, 2009.
  10. ↑ Saunders, Grover (February 12, 2007). "OpenMacGrid creates distributed computing love-fest". Ars Technica . Archived from the original on August 24, 2009. Retrieved March 1, 2009.
  11. ↑ Parnot, Charles (December 3, 2007). "Xgrid@Stanford - Home". Stanford University. Archived from the original on October 10, 2014. Retrieved July 25, 2008.
  12. ↑ Parnot, Charles (December 3, 2007). "Xgrid@Stanford - Projects". Stanford University. Archived from the original on September 26, 2011. Retrieved July 25, 2008.
  13. ↑ Zhuge, Hai; Fox, Geoffrey C. (November 24, 2005). Grid and Cooperative Computing - GCC 2005: 4th International Conference, Beijing, China, November 30 -- December 3, 2005, Proceedings. Springer Science & Business Media. p. 408. ISBN   978-3-540-30510-1.
  14. ↑ Rizzo, John (September 23, 2011). Mac OS X Lion Server For Dummies. John Wiley & Sons. ISBN   978-1-118-17748-8.
  15. ↑ Michaels, Philip; Griffiths, Rob; Snell, Jason; Seff, Jonathan (September 2004). "10 Things to Know about Tiger". Macworld . Vol. 21, no. 9. pp. 50–55. ISSN   0741-8647 . Retrieved October 16, 2008. (Subscription required)
  16. ↑ Connolly, P.J. (June 21, 2004). "Apple's Xserve Hardware Makes Waves". InfoWorld . Vol. 26, no. 25. pp. 29–30. ProQuest   194358336. (Subscription required)
  17. ↑ Schwartz, Ephraim (January 26, 2004). "Doth Apple Protest Too Much?". InfoWorld . Vol. 26, no. 4. p. 12. ProQuest   194347451. (Subscription required)
  18. ↑ Pogue, David (July 25, 2012). OS X Mountain Lion: The Missing Manual. O'Reilly Media. p. 2. ISBN   978-1-4493-4367-5.
  19. ↑ Rizzo, John (August 31, 2012). OS X Mountain Lion Server For Dummies. John Wiley & Sons. pp. 76–77. ISBN   978-1-118-41781-2. Archived from the original on November 8, 2021. Retrieved May 16, 2023.
  20. ↑ Rizzo, John (September 23, 2011). "What's in Podcast Producer?". Mac OS X Lion Server For Dummies. John Wiley & Sons. ISBN   978-1-118-17748-8.
  21. ↑ Zhuge, Hai; Fox, Geoffrey C. (November 24, 2005). Grid and Cooperative Computing - GCC 2005: 4th International Conference, Beijing, China, November 30 -- December 3, 2005, Proceedings. Springer Science & Business Media. ISBN   978-3-540-30510-1.
  22. 1 2 Côté, Daniel (January 7, 2004). "XGrid". Simple. Archived from the original on April 12, 2008. Retrieved July 18, 2008.
  23. ↑ Gulker, Chris (January 15, 2004). "Xgrid: Grid computing for the rest of us?". Linux.com. Archived from the original on November 20, 2008. Retrieved July 26, 2008.
  24. 1 2 "Xgrid Programming Guide: How It Works". Apple Inc. October 31, 2007. Retrieved June 12, 2008.
  25. ↑ "Mac OS X Server: Xgrid" (PDF). Technology Brief. Apple Inc. November 1, 2007. p. 2. Archived (PDF) from the original on March 19, 2009. Retrieved March 1, 2009.
  26. ↑ Jepson, Brian; Rothman, Ernest E. (June 1, 2005). Mac OS X Tiger for Unix Geeks. O'Reilly Media. p. 211. ISBN   978-0-596-55274-9.
  27. 1 2 Muir, Dylan (June 23, 2005). "10.4: Run an Xgrid with Tiger client". Mac OS X Hints. Archived from the original on August 24, 2007. Retrieved July 26, 2008.
  28. ↑ "VisualHub offers Xgrid support". MacNN. June 7, 2006. Archived from the original on December 9, 2017. Retrieved July 26, 2008.
  29. ↑ Chartier, David (June 7, 2006). "VisualHub - The universal video converter". Engadget. Retrieved May 14, 2023.

Further reading

Related Research Articles

AppleTalk is a discontinued proprietary suite of networking protocols developed by Apple Computer for their Macintosh computers. AppleTalk includes a number of features that allow local area networks to be connected with no prior setup or the need for a centralized router or server of any sort. Connected AppleTalk-equipped systems automatically assign addresses, update the distributed namespace, and configure any required inter-networking routing.

<span class="mw-page-title-main">Client–server model</span> Distributed application structure in computing

The client–server model is a distributed application structure that partitions tasks or workloads between the providers of a resource or service, called servers, and service requesters, called clients. Often clients and servers communicate over a computer network on separate hardware, but both client and server may reside in the same system. A server host runs one or more server programs, which share their resources with clients. A client usually does not share any of its resources, but it requests content or service from a server. Clients, therefore, initiate communication sessions with servers, which await incoming requests. Examples of computer applications that use the client–server model are email, network printing, and the World Wide Web.

<span class="mw-page-title-main">Mac OS X Server</span> Server software for macOS

Mac OS X Server is a series of discontinued Unix-like server operating systems developed by Apple Inc. based on macOS. It provided server functionality and system administration tools, and tools to manage both macOS-based computers and iOS-based devices, network services such as a mail transfer agent, AFP and SMB servers, an LDAP server, and a domain name server, as well as server applications including a Web server, database, and calendar server.

<span class="mw-page-title-main">Beowulf cluster</span> Type of computing cluster

A Beowulf cluster is a computer cluster of what are normally identical, commodity-grade computers networked into a small local area network with libraries and programs installed which allow processing to be shared among them. The result is a high-performance parallel computing cluster from inexpensive personal computer hardware.

<span class="mw-page-title-main">Yellow Dog Linux</span> Linux distribution

Yellow Dog Linux (YDL) is a discontinued free and open-source operating system for high-performance computing on multi-core processor computer architectures, focusing on GPU systems and computers using the POWER7 processor. The original developer was Terra Soft Solutions, which was acquired by Fixstars in October 2008. Yellow Dog Linux was first released in the spring of 1999 for Apple Macintosh PowerPC-based computers. The last version, Yellow Dog Linux 7, was released on August 6, 2012. Yellow Dog Linux lent its name to the popular YUM Linux software updater, derived from YDL's YUP and thus called Yellowdog Updater, Modified.

<span class="mw-page-title-main">Xserve</span> Apple rack-mounted server

The Xserve is a discontinued series of rack-mounted servers that was manufactured by Apple Inc. between 2002 and 2011. It was Apple's first rack-mounted server, and could function as a file server, web server or run high-performance computing applications in clusters – a dedicated cluster Xserve, the Xserve Cluster Node, without a video card and optical drives was also available. The first Xserve had a PowerPC G4 processor, replaced by a PowerPC G5 in 2004, and by Intel Xeon processors in 2006; each was available in single-processor and dual-processor configurations. The Xserve was discontinued in 2011, and replaced with the Mac Pro Server and the Mac Mini Server.

Zero-configuration networking (zeroconf) is a set of technologies that automatically creates a usable computer network based on the Internet Protocol Suite (TCP/IP) when computers or network peripherals are interconnected. It does not require manual operator intervention or special configuration servers. Without zeroconf, a network administrator must set up network services, such as Dynamic Host Configuration Protocol (DHCP) and Domain Name System (DNS), or configure each computer's network settings manually.

<span class="mw-page-title-main">Server Message Block</span> Network communication protocol for providing shared access to resources

Server Message Block (SMB) is a communication protocol used to share files, printers, serial ports, and miscellaneous communications between nodes on a network. On Microsoft Windows, the SMB implementation consists of two vaguely named Windows services: "Server" and "Workstation". It uses NTLM or Kerberos protocols for user authentication. It also provides an authenticated inter-process communication (IPC) mechanism.

<span class="mw-page-title-main">VNC</span> Graphical desktop-sharing system

VNC is a graphical desktop-sharing system that uses the Remote Frame Buffer protocol (RFB) to remotely control another computer. It transmits the keyboard and mouse input from one computer to another, relaying the graphical-screen updates, over a network. Popular uses for this technology include remote technical support and accessing files on one's work computer from one's home computer, or vice versa.

<span class="mw-page-title-main">Network-attached storage</span> Computer data storage server

Network-attached storage (NAS) is a file-level computer data storage server connected to a computer network providing data access to a heterogeneous group of clients. The term "NAS" can refer to both the technology and systems involved, or a specialized device built for such functionality.

Microsoft Exchange Server is a mail server and calendaring server developed by Microsoft. It runs exclusively on Windows Server operating systems.

The Apple Filing Protocol (AFP), formerly AppleTalk Filing Protocol, is a proprietary network protocol, and part of the Apple File Service (AFS), that offers file services for macOS, classic Mac OS, and Apple II computers. In OS X 10.8 Mountain Lion and earlier, AFP was the primary protocol for file services. Starting with OS X 10.9 Mavericks, Server Message Block (SMB) was made the primary file sharing protocol, with the ability to run an AFP server removed later in macOS 11 Big Sur. AFP supports Unicode file names, POSIX and access-control list permissions, resource forks, named extended attributes, and advanced file locking.

<span class="mw-page-title-main">Xsan</span> Storage area network by Apple

Xsan is Apple Inc.'s storage area network (SAN) or clustered file system for macOS. Xsan enables multiple Mac desktop and Xserve systems to access shared block storage over a Fibre Channel network. With the Xsan file system installed, these computers can read and write to the same storage volume at the same time. Xsan is a complete SAN solution that includes the metadata controller software, the file system client software, and integrated setup, management and monitoring tools.

Grid MP is a commercial distributed computing software package developed and sold by Univa, a privately held company based primarily in Austin, Texas. It was formerly known as the MetaProcessor prior to the release of version 4.0, however the letters MP in Grid MP do not officially stand for anything.

The NetWare Core Protocol (NCP) is a network protocol used in some products from Novell, Inc. It is usually associated with the client-server operating system Novell NetWare which originally supported primarily MS-DOS client stations, but later support for other platforms such as Microsoft Windows, the classic Mac OS, Linux, Windows NT, Mac OS X, and various flavors of Unix was added.

Network booting, shortened netboot, is the process of booting a computer from a network rather than a local drive. This method of booting can be used by routers, diskless workstations and centrally managed computers such as public computers at libraries and schools.

<span class="mw-page-title-main">Volunteer computing</span> System where users donate computer resources to contribute to research

Volunteer computing is a type of distributed computing in which people donate their computers' unused resources to a research-oriented project, and sometimes in exchange for credit points. The fundamental idea behind it is that a modern desktop computer is sufficiently powerful to perform billions of operations a second, but for most users only between 10–15% of its capacity is used. Common tasks such as word processing or web browsing leave the computer mostly idle.

A clustered file system (CFS) is a file system which is shared by being simultaneously mounted on multiple servers. There are several approaches to clustering, most of which do not employ a clustered file system. Clustered file systems can provide features like location-independent addressing and redundancy which improve reliability or reduce the complexity of the other parts of the cluster. Parallel file systems are a type of clustered file system that spread data across multiple storage nodes, usually for redundancy or performance.

Distributed networking is a distributed computing network system where components of the program and data depend on multiple sources.

<span class="mw-page-title-main">Computer cluster</span> Set of computers configured in a distributed computing system

A computer cluster is a set of computers that work together so that they can be viewed as a single system. Unlike grid computers, computer clusters have each node set to perform the same task, controlled and scheduled by software. The newest manifestation of cluster computing is cloud computing.

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.