Synnefo

Last updated
Synnefo
Developer(s) GRNET
Stable release
0.19.1 / June 23, 2017;7 years ago (2017-06-23) [1]
Repository github.com/grnet/synnefo
Written in Python
Type Cloud computing
License GNU General Public License
Website www.synnefo.org

Synnefo is a complete open-source cloud stack written in Python that provides Compute, Network, Image, Volume and Storage services, similar to the ones offered by AWS. Synnefo manages multiple Google Ganeti clusters at the backend that handle low-level VM operations and uses Archipelago to unify cloud storage. To boost 3rd-party compatibility, Synnefo exposes the OpenStack APIs to users.

Contents

Synnefo is being developed by GRNET (Greek Research and Technology Network), and is powering two of its public cloud services, the ~okeanos service, which is aimed towards the Greek academic community, and the ~okeanos global service, which is open for all members of the GÉANT network.

History

In November 2006, in an effort to provide advanced cloud services for the Greek academic and research community, GRNET decides to launch a cloud storage service, similar to Amazon's S3, called Pithos. The project is outsourced and opens for public beta to the members of the Greek academic and research community in May 2009. [2]

In June 2010, GRNET decides the next step in this course; to create a complete, AWS-like cloud service (Compute/Network/Volume/Image/Storage). This service, called ~okeanos, aims to provide the Greek academic and research community with access to a virtualized infrastructure that various projects can take advantage of, e.g. experiments, simulations and labs. Given the non-ephemeral nature of the resources that the service provides, the need arises for persistent cloud servers. In search for a solution, in October 2010 GRNET decides to base the service on Google Ganeti and to design and implement all missing parts in-house.

In May 2011, the older Pithos service is rewritten from scratch in-house, with the intention of being integrated to ~okeanos as its storage service. Moreover, the new Pithos adds support for Dropbox-like syncing.

In July 2011, ~okeanos reaches its public alpha stage. This version (v0.5.2.1) includes the Identity, Compute, Network and a primitive Image service. The alpha release of the new, rewritten Pithos follows shortly after, in November 2011. It is marketed as Pithos+ and the old Pithos remains as a separate service. The new Pithos+, though not integrated to ~okeanos yet, provides syncing and sharing capabilities for files, as well as native syncing clients for Mac OS X, iPhone, iPad and Windows.

In March 2012, ~okeanos enters the public alpha2 phase. This version (v0.9) includes a complete integration of the new Pithos as part of ~okeanos and now acts as the unified store for Images and Files. Around this point, in April 2012, the ~okeanos team decides to refer to the whole software stack as Synnefo and starts writing the first version of the Synnefo documentation.

In December 2012, due to interest from other parties to the Synnefo stack, GRNET decides to conceptually separate the ~okeanos and Synnefo projects. Synnefo starts to become a branding-neutral, IaaS cloud computing software, while ~okeanos becomes its real-world application, an IaaS for the Greek academic and research community.

In April 2013, a new Synnefo version (v.013) gets released after a huge cleanup and code refactoring. All separate components are merged under the single Synnefo repository. This is the first release as a unified project, containing all parts (Compute/Network/Volume/Image/Storage).

In Jun 2013, Synnefo v0.14 gets released. Since this version, Synnefo is branding neutral (all remaining ~okeanos references are removed). It also gets a branding mechanism and the corresponding documentation, so that others can adapt it to their branding identity.

Overview

Synnefo has been designed to be deployed in any environment

Components

Synnefo is modular in nature and consists of the following components:

Astakos (Identity/Account services)

Astakos is the Identity management component which provides a common user base to the rest of Synnefo. Astakos handles user creation, user groups, resource accounting, quotas, projects, and issues authentication tokens used across the infrastructure. It supports multiple authentication methods:

Pithos (File/Object Storage services)

Pithos is the Object/File Storage component of Synnefo. Users upload files on Pithos using either the Web UI, the command-line client, or native syncing clients. It is a thin layer mapping user-files to content-addressable blocks which are then stored on a storage backend. Files are split in blocks of fixed size, which are hashed independently to create a unique identifier for each block, so each file is represented by a sequence of block names (a hashmap). This way, Pithos provides deduplication of file data; blocks shared among files are only stored once.

The current implementation uses 4MB blocks hashed with SHA256. Content-based addressing also enables efficient two-way file syncing that can be used by all Pithos clients (e.g. the kamaki command-line client or the native Windows/Mac OS clients). Whenever someone wishes to upload an updated version of a file, the client hashes all blocks of the file and then requests the server to create a new version for this block sequence. The server will return an error reply with a list of the missing blocks. The client may then upload each block one by one, and retry file creation. Similarly, whenever a file has been changed on the server, the client can ask for its list of blocks and only download the modified ones.

Pithos runs at the cloud layer and exposes the OpenStack Object Storage API to the outside world, with custom extensions for syncing. Any client speaking to OpenStack Swift can also be used to store objects in a Pithos deployment. The process of mapping user files to hashed objects is independent from the actual storage backend, which is selectable by the administrator using pluggable drivers. Currently, Pithos has drivers for two storage backends:

Whatever the storage backend, it is responsible for storing objects reliably, without any connection to the cloud APIs or to the hashing operations.

Cyclades (Compute/Network/Image/Volume services)

Cyclades is the Synnefo component that implements the Compute, Network, Image and Volume services. It exposes the associated OpenStack REST APIs: OpenStack Compute, Network, Glance and soon also Cinder. Cyclades is the part which manages multiple Ganeti clusters at the backend. Cyclades issues commands to a Ganeti cluster using Ganeti's Remote API (RAPI). The administrator can expand the infrastructure dynamically by adding new Ganeti clusters to reach datacenter scale. Cyclades knows nothing about low-level VM management operations, e.g., handling of VM creations, migrations among physical nodes, and handling of node downtimes; the design and implementation of the end-user API is orthogonal to VM handling at the backend.

There are two distinct, asynchronous paths in the interaction between Synnefo and Ganeti. The effect path is activated in response to a user request; Cyclades issues VM control commands to Ganeti over RAPI. The update path is triggered whenever the state of a VM changes, due to Synnefo- or administrator-initiated actions happening at the Ganeti level. In the update path, we monitor Ganeti's job queue to produce notifications to the rest of the Synnefo infrastructure over a message queue.

Users have full control over their VMs: they can create new ones, start them, shutdown, reboot, and destroy them. For the configuration of their VMs they can select number of CPUs, size of RAM and system disk, and operating system from pre-defined Images including popular Linux distros (Debian, Ubuntu, CentOS, Fedora, Gentoo, Archlinux, OpenSuse), MS-Windows Server 2008 R2 and 2012 as well as FreeBSD.

The REST API for VM management, being OpenStack compatible, can interoperate with 3rd party tools and client libraries.

The Cyclades UI is written in Javascript/jQuery and runs entirely on the client side for maximum responsiveness. It is just another API client; all UI operations happen with asynchronous calls over the API.

The networking functionality includes dual IPv4/IPv6 connectivity for each VM, easy, platform-provided firewalling either through an array of pre-configured firewall profiles, or through a roll-your-own firewall inside the VM. Users may create multiple private, virtual L2 networks, so that they construct arbitrary network topologie, e.g., to deploy VMs in multi-tier configurations. The networking functionality is exported all the way from the backend to the API and the UI.

See also

Related Research Articles

In telecommunications, provisioning involves the process of preparing and equipping a network to allow it to provide new services to its users. In National Security/Emergency Preparedness telecommunications services, "provisioning" equates to "initiation" and includes altering the state of an existing priority service or capability.

<span class="mw-page-title-main">Greek Research and Technology Network</span> National research and education network of Greece

The National Infrastructure for Research and Technology Network or GRNET is the national research and education network - NREN of Greece and was formerly named Greek Research and Educational Network. GRNET S.A. provides internet connectivity, e-infrastructure, and other services to the Greek Educational, Academic, and Research community. Additionally, GRNET makes digital applications. It also provides services to the following sectors: Education, Research, Health, Culture. GRNET supports all Universities, Technological Education Institutes, Research Centers, and over 9,500 schools.

In computing, a solution stack or software stack is a set of software subsystems or components needed to create a complete platform such that no additional software is needed to support applications. Applications are said to "run on" or "run on top of" the resulting platform.

Infrastructure as a service (IaaS) is a cloud computing service model where a cloud services vendor provides computing resources such as storage, network, servers, and virtualization. This service frees users from maintaining their own data center, but they must install and maintain the operating system and application software. Iaas provides users high-level APIs to control details of underlying network infrastructure such as backup, data partitioning, scaling, security and physical computing resources. Services can be scaled on-demand by the user. According to the Internet Engineering Task Force (IETF), such infrastructure is the most basic cloud-service model. IaaS can be hosted in a public cloud, a private cloud, or a hybrid cloud.

Ceph is a free and open-source software-defined storage platform that provides object storage, block storage, and file storage built on a common distributed cluster foundation. Ceph provides distributed operation without a single point of failure and scalability to the exabyte level. Since version 12 (Luminous), Ceph does not rely on any other conventional filesystem and directly manages HDDs and SSDs with its own storage backend BlueStore and can expose a POSIX filesystem.

oVirt Free, open-source virtualization management platform

oVirt is a free, open-source virtualization management platform. It was founded by Red Hat as a community project on which Red Hat Virtualization is based. It allows centralized management of virtual machines, compute, storage and networking resources, from an easy-to-use web-based front-end with platform independent access. KVM on x86-64, PowerPC64 and s390x architecture are the only hypervisors supported, but there is an ongoing effort to support ARM architecture in a future releases.

<span class="mw-page-title-main">Rackspace Cloud</span> Cloud computing platform

The Rackspace Cloud is a set of cloud computing products and services billed on a utility computing basis from the US-based company Rackspace. Offerings include Cloud Storage, virtual private server, load balancers, databases, backup, and monitoring.

Eucalyptus is a paid and open-source computer software for building Amazon Web Services (AWS)-compatible private and hybrid cloud computing environments, originally developed by the company Eucalyptus Systems. Eucalyptus is an acronym for Elastic Utility Computing Architecture for Linking Your Programs To Useful Systems. Eucalyptus enables pooling compute, storage, and network resources that can be dynamically scaled up or down as application workloads change. Mårten Mickos was the CEO of Eucalyptus. In September 2014, Eucalyptus was acquired by Hewlett-Packard and then maintained by DXC Technology. After DXC stopped developing the product in late 2017, AppScale Systems forked the code and started supporting Eucalyptus customers.

<span class="mw-page-title-main">Gladinet</span>

Gladinet, Inc. is an American corporation co-founded by Jerry Huang and Franklyn Peart in 2008 in Lake Worth, Florida. The company provides cloud computing software, including an on-premises, enterprise file synchronization and sharing (EFSS) platform. The on-premises version of the platform is available for service providers and enterprises. There is also a team edition.

<span class="mw-page-title-main">OpenStack</span> Cloud computing software

OpenStack is a free, open standard cloud computing platform. It is mostly deployed as infrastructure-as-a-service (IaaS) in both public and private clouds where virtual servers and other resources are made available to users. The software platform consists of interrelated components that control diverse, multi-vendor hardware pools of processing, storage, and networking resources throughout a data center. Users manage it either through a web-based dashboard, through command-line tools, or through RESTful web services.

<span class="mw-page-title-main">OpenNebula</span> Cloud-computing platform for managing heterogeneous distributed infrastructure

OpenNebula is an open source cloud computing platform for managing heterogeneous data center, public cloud and edge computing infrastructure resources. OpenNebula manages on-premises and remote virtual infrastructure to build private, public, or hybrid implementations of infrastructure as a service (IaaS) and multi-tenant Kubernetes deployments. The two primary uses of the OpenNebula platform are data center virtualization and cloud deployments based on the KVM hypervisor, LXD/LXC system containers, and AWS Firecracker microVMs. The platform is also capable of offering the cloud infrastructure necessary to operate a cloud on top of existing VMware infrastructure. In early June 2020, OpenNebula announced the release of a new Enterprise Edition for corporate users, along with a Community Edition. OpenNebula CE is free and open-source software, released under the Apache License version 2. OpenNebula CE comes with free access to patch releases containing critical bug fixes but with no access to the regular EE maintenance releases. Upgrades to the latest minor/major version is only available for CE users with non-commercial deployments or with significant open source contributions to the OpenNebula Community. OpenNebula EE is distributed under a closed-source license and requires a commercial Subscription.

CloudStack is open-source Infrastructure-as-a-Service cloud computing software for creating, managing, and deploying infrastructure cloud services. It uses existing hypervisor platforms for virtualization, such as KVM, VMware vSphere, including ESXi and vCenter, XenServer/XCP and XCP-ng. In addition to its own API, CloudStack also supports the Amazon Web Services (AWS) API and the Open Cloud Computing Interface from the Open Grid Forum.

Google Compute Engine (GCE) is the infrastructure as a service (IaaS) component of Google Cloud Platform which is built on the global infrastructure that runs Google's search engine, Gmail, YouTube and other services. Google Compute Engine enables users to launch virtual machines (VMs) on demand. VMs can be launched from the standard images or custom images created by users. Google Compute Engine can be accessed via the Developer Console, RESTful API or command-line interface (CLI).

Backend as a service (BaaS), sometimes also referred to as mobile backend as a service (MBaaS), is a service for providing web app and mobile app developers with a way to easily build a backend to their frontend applications. Features available include user management, push notifications, and integration with social networking services. These services are provided via the use of custom software development kits (SDKs) and application programming interfaces (APIs). BaaS is a relatively recent development in cloud computing, with most BaaS startups dating from 2011 or later. Some of the most popular service providers are AWS Amplify and Firebase.

Elliptics is a distributed key–value data storage with open source code. By default it is a classic distributed hash table (DHT) with multiple replicas put in different groups. Elliptics was created to meet requirements of multi-datacenter and physically distributed storage locations when storing huge amount of medium and large files.

<span class="mw-page-title-main">Syncthing</span> Free and open-source peer-to-peer file synchronization application

Syncthing is a free and open source peer-to-peer file synchronization application available for Windows, macOS, Linux, Android, Solaris, Darwin, and BSD. It can sync files between devices on a local network, or between remote devices over the Internet. Data security and data safety are built into its design. Version 1.0 was released in January 2019 after five years in beta.

Kubernetes is an open-source container orchestration system for automating software deployment, scaling, and management. Originally designed by Google, the project is now maintained by a worldwide community of contributors, and the trademark is held by the Cloud Native Computing Foundation.

ObjectiveFS is a distributed file system developed by Objective Security Corp. It is a POSIX-compliant file system built with an object store backend. It was initially released with AWS S3 backend, and has later implemented support for Google Cloud Storage and object store devices. It was released for beta in early 2013, and the first version was officially released on August 11, 2013.

A cloudlet is a mobility-enhanced small-scale cloud datacenter that is located at the edge of the Internet. The main purpose of the cloudlet is supporting resource-intensive and interactive mobile applications by providing powerful computing resources to mobile devices with lower latency. It is a new architectural element that extends today's cloud computing infrastructure. It represents the middle tier of a 3-tier hierarchy: mobile device - cloudlet - cloud. A cloudlet can be viewed as a data center in a box whose goal is to bring the cloud closer. The cloudlet term was first coined by M. Satyanarayanan, Victor Bahl, Ramón Cáceres, and Nigel Davies, and a prototype implementation is developed by Carnegie Mellon University as a research project. The concept of cloudlet is also known as follow me cloud, and mobile micro-cloud.

Harvester is a cloud native hyper-converged infrastructure (HCI) open source software. Harvester was announced in 2020 by SUSE.

References

  1. ↑ "Synnefo v0.19.1 Released" . Retrieved 13 July 2017.
  2. ↑ "Synnefo timeline". synnefo.org. Retrieved 4 April 2014.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.