Synnefo

Last updated
Synnefo
Developer(s) GRNET
Stable release
0.19.1 / June 23, 2017;5 years ago (2017-06-23) [1]
Repository github.com/grnet/synnefo
Written in Python
Type Cloud computing
License GNU General Public License
Website www.synnefo.org

Synnefo is a complete open-source cloud stack written in Python that provides Compute, Network, Image, Volume and Storage services, similar to the ones offered by AWS. Synnefo manages multiple Google Ganeti clusters at the backend that handle low-level VM operations and uses Archipelago to unify cloud storage. To boost 3rd-party compatibility, Synnefo exposes the OpenStack APIs to users.

Contents

Synnefo is being developed by GRNET (Greek Research and Technology Network), and is powering two of its public cloud services, the ~okeanos service, which is aimed towards the Greek academic community, and the ~okeanos global service, which is open for all members of the GÉANT network.

History

In November 2006, in an effort to provide advanced cloud services for the Greek academic and research community, GRNET decides to launch a cloud storage service, similar to Amazon's S3, called Pithos. The project is outsourced and opens for public beta to the members of the Greek academic and research community in May 2009. [2]

In June 2010, GRNET decides the next step in this course; to create a complete, AWS-like cloud service (Compute/Network/Volume/Image/Storage). This service, called ~okeanos, aims to provide the Greek academic and research community with access to a virtualized infrastructure that various projects can take advantage of, e.g. experiments, simulations and labs. Given the non-ephemeral nature of the resources that the service provides, the need arises for persistent cloud servers. In search for a solution, in October 2010 GRNET decides to base the service on Google Ganeti and to design and implement all missing parts in-house.

In May 2011, the older Pithos service is rewritten from scratch in-house, with the intention of being integrated to ~okeanos as its storage service. Moreover, the new Pithos adds support for Dropbox-like syncing.

In July 2011, ~okeanos reaches its public alpha stage. This version (v0.5.2.1) includes the Identity, Compute, Network and a primitive Image service. The alpha release of the new, rewritten Pithos follows shortly after, in November 2011. It is marketed as Pithos+ and the old Pithos remains as a separate service. The new Pithos+, though not integrated to ~okeanos yet, provides syncing and sharing capabilities for files, as well as native syncing clients for Mac OS X, iPhone, iPad and Windows.

In March 2012, ~okeanos enters the public alpha2 phase. This version (v0.9) includes a complete integration of the new Pithos as part of ~okeanos and now acts as the unified store for Images and Files. Around this point, in April 2012, the ~okeanos team decides to refer to the whole software stack as Synnefo and starts writing the first version of the Synnefo documentation.

In December 2012, due to interest from other parties to the Synnefo stack, GRNET decides to conceptually separate the ~okeanos and Synnefo projects. Synnefo starts to become a branding-neutral, IaaS cloud computing software, while ~okeanos becomes its real-world application, an IaaS for the Greek academic and research community.

In April 2013, a new Synnefo version (v.013) gets released after a huge cleanup and code refactoring. All separate components are merged under the single Synnefo repository. This is the first release as a unified project, containing all parts (Compute/Network/Volume/Image/Storage).

In Jun 2013, Synnefo v0.14 gets released. Since this version, Synnefo is branding neutral (all remaining ~okeanos references are removed). It also gets a branding mechanism and the corresponding documentation, so that others can adapt it to their branding identity.

Overview

Synnefo has been designed to be deployed in any environment

Components

Synnefo is modular in nature and consists of the following components:

Astakos (Identity/Account services)

Astakos is the Identity management component which provides a common user base to the rest of Synnefo. Astakos handles user creation, user groups, resource accounting, quotas, projects, and issues authentication tokens used across the infrastructure. It supports multiple authentication methods:

Pithos (File/Object Storage services)

Pithos is the Object/File Storage component of Synnefo. Users upload files on Pithos using either the Web UI, the command-line client, or native syncing clients. It is a thin layer mapping user-files to content-addressable blocks which are then stored on a storage backend. Files are split in blocks of fixed size, which are hashed independently to create a unique identifier for each block, so each file is represented by a sequence of block names (a hashmap). This way, Pithos provides deduplication of file data; blocks shared among files are only stored once.

The current implementation uses 4MB blocks hashed with SHA256. Content-based addressing also enables efficient two-way file syncing that can be used by all Pithos clients (e.g. the kamaki command-line client or the native Windows/Mac OS clients). Whenever someone wishes to upload an updated version of a file, the client hashes all blocks of the file and then requests the server to create a new version for this block sequence. The server will return an error reply with a list of the missing blocks. The client may then upload each block one by one, and retry file creation. Similarly, whenever a file has been changed on the server, the client can ask for its list of blocks and only download the modified ones.

Pithos runs at the cloud layer and exposes the OpenStack Object Storage API to the outside world, with custom extensions for syncing. Any client speaking to OpenStack Swift can also be used to store objects in a Pithos deployment. The process of mapping user files to hashed objects is independent from the actual storage backend, which is selectable by the administrator using pluggable drivers. Currently, Pithos has drivers for two storage backends:

Whatever the storage backend, it is responsible for storing objects reliably, without any connection to the cloud APIs or to the hashing operations.

Cyclades (Compute/Network/Image/Volume services)

Cyclades is the Synnefo component that implements the Compute, Network, Image and Volume services. It exposes the associated OpenStack REST APIs: OpenStack Compute, Network, Glance and soon also Cinder. Cyclades is the part which manages multiple Ganeti clusters at the backend. Cyclades issues commands to a Ganeti cluster using Ganeti's Remote API (RAPI). The administrator can expand the infrastructure dynamically by adding new Ganeti clusters to reach datacenter scale. Cyclades knows nothing about low-level VM management operations, e.g., handling of VM creations, migrations among physical nodes, and handling of node downtimes; the design and implementation of the end-user API is orthogonal to VM handling at the backend.

There are two distinct, asynchronous paths in the interaction between Synnefo and Ganeti. The effect path is activated in response to a user request; Cyclades issues VM control commands to Ganeti over RAPI. The update path is triggered whenever the state of a VM changes, due to Synnefo- or administrator-initiated actions happening at the Ganeti level. In the update path, we monitor Ganeti's job queue to produce notifications to the rest of the Synnefo infrastructure over a message queue.

Users have full control over their VMs: they can create new ones, start them, shutdown, reboot, and destroy them. For the configuration of their VMs they can select number of CPUs, size of RAM and system disk, and operating system from pre-defined Images including popular Linux distros (Debian, Ubuntu, CentOS, Fedora, Gentoo, Archlinux, OpenSuse), MS-Windows Server 2008 R2 and 2012 as well as FreeBSD.

The REST API for VM management, being OpenStack compatible, can interoperate with 3rd party tools and client libraries.

The Cyclades UI is written in Javascript/jQuery and runs entirely on the client side for maximum responsiveness. It is just another API client; all UI operations happen with asynchronous calls over the API.

The networking functionality includes dual IPv4/IPv6 connectivity for each VM, easy, platform-provided firewalling either through an array of pre-configured firewall profiles, or through a roll-your-own firewall inside the VM. Users may create multiple private, virtual L2 networks, so that they construct arbitrary network topologie, e.g., to deploy VMs in multi-tier configurations. The networking functionality is exported all the way from the backend to the API and the UI.

See also

Related Research Articles

In telecommunication, provisioning involves the process of preparing and equipping a network to allow it to provide new services to its users. In National Security/Emergency Preparedness telecommunications services, "provisioning" equates to "initiation" and includes altering the state of an existing priority service or capability.

NetApp, Inc. is an American hybrid cloud data services and data management company headquartered in San Jose, California. It has ranked in the Fortune 500 from 2012–2021. Founded in 1992 with an IPO in 1995, NetApp offers cloud data services for management of applications and data both online and physically.

<span class="mw-page-title-main">Greek Research and Technology Network</span>

The Greek Research and Technology Network or GRNET is the national research and education network of Greece. GRNET S.A. gives internet connectivity, e-infrastructure and other services to the Greek Educational, Academic and Research community. Additionally, GRNET makes digital applications. It also provides services to the following sectors: Education, Research, Health, Culture. GRNET supports all Universities, Technological Education Institutes, Research Centers and over 9,500 schools.

In computing, a solution stack or software stack is a set of software subsystems or components needed to create a complete platform such that no additional software is needed to support applications. Applications are said to "run on" or "run on top of" the resulting platform.

The first major provider of infrastructure as a service (IaaS) was Amazon in 2008. IaaS is a cloud computing service model by means of which computing resources are supplied by a cloud services provider. The IaaS vendor provides the storage, network, servers and virtualization. This service enable users to free themselves from maintaining an on-premise data center. The IaaS provider is hosting these resources in either a public cloud, private cloud, or hybrid cloud.

<span class="mw-page-title-main">Ceph (software)</span> Open-source storage platform

Ceph is an open-source software-defined storage platform that implements object storage on a single distributed computer cluster and provides 3-in-1 interfaces for object-, block- and file-level storage. Ceph aims primarily for completely distributed operation without a single point of failure, scalability to the exabyte level, and to be freely available. Since version 12, Ceph does not rely on other filesystems and can directly manage HDDs and SSDs with its own storage backend BlueStore and can completely self reliantly expose a POSIX filesystem.

oVirt Free, open-source virtualization management platform

oVirt is a free, open-source virtualization management platform. It was founded by Red Hat as a community project on which Red Hat Virtualization is based. It allows centralized management of virtual machines, compute, storage and networking resources, from an easy-to-use web-based front-end with platform independent access. KVM on x86-64, PowerPC64 and s390x architecture are the only hypervisors supported, but there is an ongoing effort to support ARM architecture in a future releases.

<span class="mw-page-title-main">Rackspace Cloud</span> Cloud computing platform

The Rackspace Cloud is a set of cloud computing products and services billed on a utility computing basis from the US-based company Rackspace. Offerings include Cloud Storage, virtual private server, load balancers, databases, backup, and monitoring.

<span class="mw-page-title-main">Cloud computing</span> Form of shared Internet-based computing

Cloud computing is the on-demand availability of computer system resources, especially data storage and computing power, without direct active management by the user. Large clouds often have functions distributed over multiple locations, each of which is a data center. Cloud computing relies on sharing of resources to achieve coherence and typically uses a "pay as you go" model, which can help in reducing capital expenses but may also lead to unexpected operating expenses for users.

Eucalyptus is a paid and open-source computer software for building Amazon Web Services (AWS)-compatible private and hybrid cloud computing environments, originally developed by the company Eucalyptus Systems. Eucalyptus is an acronym for Elastic Utility Computing Architecture for Linking Your Programs To Useful Systems. Eucalyptus enables pooling compute, storage, and network resources that can be dynamically scaled up or down as application workloads change. Mårten Mickos was the CEO of Eucalyptus. In September 2014, Eucalyptus was acquired by Hewlett-Packard and then maintained by DXC Technology. After DXC stopped developing the product in late 2017, AppScale Systems forked the code and started supporting Eucalyptus customers.

<span class="mw-page-title-main">OpenStack</span> Cloud computing software

OpenStack is a free, open standard cloud computing platform. It is mostly deployed as infrastructure-as-a-service (IaaS) in both public and private clouds where virtual servers and other resources are made available to users. The software platform consists of interrelated components that control diverse, multi-vendor hardware pools of processing, storage, and networking resources throughout a data center. Users manage it either through a web-based dashboard, through command-line tools, or through RESTful web services.

<span class="mw-page-title-main">OpenNebula</span> Cloud computing platform for managing heterogeneous distributed data center infrastructures

OpenNebula is a hyper-converged infrastructure platform for managing heterogeneous distributed data center infrastructures. The OpenNebula platform manages a data center's virtual infrastructure to build private, public and hybrid implementations of Infrastructure as a Service. The two primary uses of the OpenNebula platform are data center virtualization and cloud deployments based on the KVM hypervisor, LXD/LXC system containers, and AWS Firecracker microVMs. The platform is also capable of offering the cloud infrastructure necessary to operate a cloud on top of existing VMware infrastructure. In early June 2020, OpenNebula announced the release of a new Enterprise Edition for corporate users, along with a Community Edition. OpenNebula CE is free and open-source software, released under the Apache License version 2. OpenNebula CE comes with free access to maintenance releases but with upgrades to new minor/major versions only available for users with non-commercial deployments or with significant contributions to the OpenNebula Community. OpenNebula EE is distributed under a closed-source license and requires a commercial Subscription.

<span class="mw-page-title-main">Ganeti</span>

Ganeti is a virtual machine cluster management tool originally developed by Google. The solution stack uses either Xen, KVM, or LXC as the virtualization platform, LVM for disk management, and optionally DRBD for disk replication across physical hosts or shared storage for external replication. Since 2007 Ganeti is developed and released as free and open-source software. Originally subject to the requirements of the GNU General Public License (GPL) version 2, the license was changed to the 2-clause BSD license in version 2.11.6, released September 2014.

Google Compute Engine (GCE) is the Infrastructure as a Service (IaaS) component of Google Cloud Platform which is built on the global infrastructure that runs Google's search engine, Gmail, YouTube and other services. Google Compute Engine enables users to launch virtual machines (VMs) on demand. VMs can be launched from the standard images or custom images created by users. GCE users must authenticate based on OAuth 2.0 before launching the VMs. Google Compute Engine can be accessed via the Developer Console, RESTful API or command-line interface (CLI).

Mobile backend as a service (MBaaS), also known as "backend as a service", is a model for providing web app and mobile app developers with a way to link their applications to backend cloud storage and APIs exposed by back end applications while also providing features such as user management, push notifications, and integration with social networking services. These services are provided via the use of custom software development kits (SDKs) and application programming interfaces (APIs). BaaS is a relatively recent development in cloud computing, with most BaaS startups dating from 2011 or later.

Elliptics is a distributed key–value data storage with open source code. By default it is a classic distributed hash table (DHT) with multiple replicas put in different groups. Elliptics was created to meet requirements of multi-datacenter and physically distributed storage locations when storing huge amount of medium and large files.

Kubernetes is an open-source container orchestration system for automating software deployment, scaling, and management. Originally designed by Google, the project is now maintained by the Cloud Native Computing Foundation.

ObjectiveFS is a distributed file system developed by Objective Security Corp. It is a POSIX-compliant file system built with an object store backend. It was initially released with AWS S3 backend, and has later implemented support for Google Cloud Storage and object store devices. It was released for beta in early 2013, and the first version was officially released on August 11, 2013.

A cloudlet is a mobility-enhanced small-scale cloud datacenter that is located at the edge of the Internet. The main purpose of the cloudlet is supporting resource-intensive and interactive mobile applications by providing powerful computing resources to mobile devices with lower latency. It is a new architectural element that extends today’s cloud computing infrastructure. It represents the middle tier of a 3-tier hierarchy: mobile device - cloudlet - cloud. A cloudlet can be viewed as a data center in a box whose goal is to bring the cloud closer. The cloudlet term was first coined by M. Satyanarayanan, Victor Bahl, Ramón Cáceres, and Nigel Davies, and a prototype implementation is developed by Carnegie Mellon University as a research project. The concept of cloudlet is also known as follow me cloud, and mobile micro-cloud.

<span class="mw-page-title-main">Rclone</span> Cloud storage management software

Rclone is an open source, multi threaded, command line computer program to manage or migrate content on cloud and other high latency storage. Its capabilities include sync, transfer, crypt, cache, union, compress and mount. The rclone website lists supported backends including S3, and Google Drive.

References

  1. ↑ "Synnefo v0.19.1 Released" . Retrieved 13 July 2017.
  2. ↑ "Synnefo timeline". synnefo.org. Retrieved 4 April 2014.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.