Cloud storage gateway

Last updated

A cloud storage gateway is a hybrid cloud storage device, implemented in hardware or software, which resides at the customer premises and translates cloud storage APIs such as SOAP or REST to block-based storage protocols such as iSCSI or Fibre Channel or file-based interfaces such as NFS or SMB. [1] [2]

Contents

According to a 2011 report by Gartner Group, cloud gateways were expected to increase the use of cloud storage by lowering monthly charges and eliminating the concern of data security. [3]

Technology

Features

Modern applications (aka "cloud native applications") use network attached storage by means of REST and SOAP with hypertext transfer protocol on the protocol layer. The related storage is provided from arrays that offer these as object storage. Classic applications use network attached storage by means of Network File System NFS, iSCSI or Server Message Block SMB. To make use of all the advantages of object storage, existing applications need to be rewritten and new applications must be object storage aware, which is not the case by default. This problem is addressed by cloud storage gateways. They offer object storage via classic native storage protocols like Network File System NFS or Server Message Block SMB (and a very few offer iSCSI as well). As a rule of thumb, classic applications with cloud native object storage can now be used with cloud storage gateways.

Functionality

In enterprise infrastructures NFS is mainly used by Linux systems whereas Windows systems are using SMB. Object storage needs data in form of objects rather than files. [4] For all cloud storage gateways it is mandatory to cache the incoming files and destage them to object storage on a later step. The time of destaging is subject to the gateway and a policy engine allows functions like

Combinations of these functions are usual. [5] Default sorting schematics spanning the retrieval interface generally rely on zero-fault content processing, which carries the obvious requirement that two or more of the above functions are synchronized. [6]

Extensions

Nearly all object storage gateways support Amazon S3 protocol as a quasi-standard. Some offer as well Microsoft Azure Blob, Google Storage, or Openstack SWIFT. Most gateways support public cloud storage e.g. from Amazon or Microsoft as an object store and Dropbox as a file drive store, there are as well a lot of vendors that support private cloud storage as well – including off and on prem storage. [7]

Deployment methods

There are multiple variants to deploy such gateways – and some vendors support as well different variants as of their product line:

Software appliances as well as FUSE-based gateways can be installed on public cloud infrastructures.

Advantages

Cloud storage gateways avoid the need to change existing applications by providing a standard interface.

Additionally, IT users are used to existing protocols – like SMB or NFS. They can make use of cloud storage with the advantage of still using their existing infrastructures (including e.g. Active Directory, LDAP integration, file share functions etc.).

While cloud storage gateways initially covered a niche only, they got more attraction as of multi-cloud technologies. As an example: It is possible to run a cloud storage gateway in form of a software appliance on top of a public or private cloud infrastructure by offering docker volume drivers that enable containers for automatic provisioning of storage used by these containers in a consistent form. They are using the hypervisors disks as a cache only, but destage data on least recently used algorithm to the underlying cloud storage.

The de facto standard for object storage is Amazon S3 – it had the most popularity and capacity installed on object storage. [8] But every object storage vendor can (and most of them do) offer Amazon S3 storage – even there is no real "standard" S3 API: Every vendor is a little bit different in implementing S3 API (as seen from the different cloud storage gateway vendors supporting the "specific" APIs of the different object storage vendors). Since 2018, an increasing number of cloud storage gateways hide this complexity by offering S3 on northbound (as of networking technologies, southbound relates to the storage used by a gateway, whereas northbound is the storage provided by the gateway). As such, one may utilize a richer S3 implementation on northbound than the southbound supports.

Disadvantages

By using cloud storage gateways the complexity to use object storage is hidden, but that also hides some of the advantages of object storage:

As applications change to cloud-aware applications (aka called cloud native applications), cloud storage gateways will change from multiprotocol gateways to multi-cloud gateways, providing access to multiple cloud providers as well as multiple southbound protocols and act as relays between different clouds.

Market

As of 2020 the cloud storage gateway market was valued at over USD 2 billion and was predicted to reach USD 11 billion by 2026, based on a report by the market research firm Mordor intelligence. [9]

See also

Related Research Articles

In computing, a file server is a computer attached to a network that provides a location for shared disk access, i.e. storage of computer files that can be accessed by the workstations that are able to reach the computer that shares the access through a computer network. The term server highlights the role of the machine in the traditional client–server scheme, where the clients are the workstations using the storage. A file server does not normally perform computational tasks or run programs on behalf of its client workstations.

Network File System (NFS) is a distributed file system protocol originally developed by Sun Microsystems (Sun) in 1984, allowing a user on a client computer to access files over a computer network much like local storage is accessed. NFS, like many other protocols, builds on the Open Network Computing Remote Procedure Call system. NFS is an open IETF standard defined in a Request for Comments (RFC), allowing anyone to implement the protocol.

<span class="mw-page-title-main">Network-attached storage</span> Computer data storage server

Network-attached storage (NAS) is a file-level computer data storage server connected to a computer network providing data access to a heterogeneous group of clients. The term "NAS" can refer to both the technology and systems involved, or a specialized device built for such functionality.

NetApp, Inc. is an American data storage and data management services company headquartered in San Jose, California. It has ranked in the Fortune 500 from 2012 to 2021. Founded in 1992 with an initial public offering in 1995, NetApp offers cloud data services for management of applications and data both online and physically.

A gateway is a piece of networking hardware or software used in telecommunications networks that allows data to flow from one discrete network to another. Gateways are distinct from routers or switches in that they communicate using more than one protocol to connect multiple networks and can operate at any of the seven layers of the open systems interconnection model (OSI).

A virtual storage area network is a logical representation of a physical storage area network (SAN). A VSAN abstracts the storage-related operations from the physical storage layer, and provides shared storage access to the applications and virtual machines by combining the servers' local storage over a network into a single or multiple storage pools.

A virtual tape library (VTL) is a data storage virtualization technology used typically for backup and recovery purposes. A VTL presents a storage component as tape libraries or tape drives for use with existing backup software.

In computer science, storage virtualization is "the process of presenting a logical view of the physical storage resources to" a host computer system, "treating all storage media in the enterprise as a single pool of storage."

A clustered file system is a file system which is shared by being simultaneously mounted on multiple servers. There are several approaches to clustering, most of which do not employ a clustered file system. Clustered file systems can provide features like location-independent addressing and redundancy which improve reliability or reduce the complexity of the other parts of the cluster. Parallel file systems are a type of clustered file system that spread data across multiple storage nodes, usually for redundancy or performance.

oVirt Free, open-source virtualization management platform

oVirt is a free, open-source virtualization management platform. It was founded by Red Hat as a community project on which Red Hat Virtualization is based. It allows centralized management of virtual machines, compute, storage and networking resources, from an easy-to-use web-based front-end with platform independent access. KVM on x86-64, PowerPC64 and s390x architecture are the only hypervisors supported, but there is an ongoing effort to support ARM architecture in a future releases.

Sun Open Storage was an open source computer data storage platform developed by Sun Microsystems. Sun Open Storage was advertised as avoiding vendor lock-in.

<span class="mw-page-title-main">Storage area network</span> Network which provides access to consolidated, block-level data storage

A storage area network (SAN) or storage network is a computer network which provides access to consolidated, block-level data storage. SANs are primarily used to access data storage devices, such as disk arrays and tape libraries from servers so that the devices appear to the operating system as direct-attached storage. A SAN typically is a dedicated network of storage devices not accessible through the local area network (LAN).

In computer storage, a global file system is a distributed file system that can be accessed from multiple locations, typically across a wide-area network, and provides concurrent access to a global namespace from all locations. In order for a file system to be considered global, it must allow for files to be created, modified, and deleted from any location. This access is typically provided by a cloud storage gateway at each edge location, which provides access using the NFS or SMB network file sharing protocols.

<span class="mw-page-title-main">StarWind Software</span> American computer storage company

StarWind Software, Inc. is a privately held Beverly, Massachusetts-based computer software and hardware appliance company specializing in storage virtualization and software-defined storage.

<span class="mw-page-title-main">Dell Fluid File System</span> Filesystem by Dell

Dell Fluid File System, or FluidFS, is a shared-disk filesystem made by Dell that provides distributed file systems to clients. Customers buy an appliance: a combination of purpose-built network-attached storage (NAS) controllers with integrated primary and backup power supplies attached to block level storage via the iSCSI or Fiber Channel protocol. A single Dell FluidFS appliance consists of two controllers operating in concert connecting to the back-end storage area network (SAN). Depending on the storage capacity requirements and user preference, FluidFS version 4 NAS appliances can be used with Compellent or EqualLogic SAN arrays. The EqualLogic FS7600 and FS7610 connect to the client network and to Dell's EqualLogic arrays with either 1 Gbit/s (FS7600) or 10 Gbit/s (FS7610) iSCSI protocol. For Compellent, FluidFS is available with either 1 Gbit/s or 10 Gbit/s iSCSI connectivity to the client network and connection to the backend Compellent SAN can be either 8 Gbit/s Fibre Channel or 10 Gbit/s iSCSI.

VM-aware storage (VAS) is computer data storage designed specifically for managing storage for virtual machines (VMs) within a data center. The goal is to provide storage that is simpler to use with functionality better suited for VMs compared with general-purpose storage. VM-aware storage allows storage to be managed as an integrated part of managing VMs rather than as logical unit numbers (LUNs) or volumes that are separately configured and managed.

Software-defined storage (SDS) is a marketing term for computer data storage software for policy-based provisioning and management of data storage independent of the underlying hardware. Software-defined storage typically includes a form of storage virtualization to separate the storage hardware from the software that manages it. The software enabling a software-defined storage environment may also provide policy management for features such as data deduplication, replication, thin provisioning, snapshots and backup.

Object storage is a computer data storage that manages data as objects, as opposed to other storage architectures like file systems which manages data as a file hierarchy, and block storage which manages data as blocks within sectors and tracks. Each object typically includes the data itself, a variable amount of metadata, and a globally unique identifier. Object storage can be implemented at multiple levels, including the device level, the system level, and the interface level. In each case, object storage seeks to enable capabilities not addressed by other storage architectures, like interfaces that are directly programmable by the application, a namespace that can span multiple instances of physical hardware, and data-management functions like data replication and data distribution at object-level granularity.

ONTAP or Data ONTAP or Clustered Data ONTAP (cDOT) or Data ONTAP 7-Mode is NetApp's proprietary operating system used in storage disk arrays such as NetApp FAS and AFF, ONTAP Select, and Cloud Volumes ONTAP. With the release of version 9.0, NetApp decided to simplify the Data ONTAP name and removed the word "Data" from it, and remove the 7-Mode image, therefore, ONTAP 9 is the successor of Clustered Data ONTAP 8.

<span class="mw-page-title-main">Hybrid cloud storage</span>

Hybrid cloud storage, in data storage, is a term for a storage infrastructure that uses a combination of on-premises storage resources with a public cloud storage provider. The on-premises storage is usually managed by the organization, while the public cloud storage provider is responsible for the management and security of the data stored in the cloud.

References

  1. ↑ Gary Orenstein (22 June 2010). "GigaOm, Show Me the Gateway — Taking Storage to the Cloud". Archived from the original on 15 May 2019. Retrieved 18 July 2013.
  2. ↑ Boles, Jeff. Cloud file storage pros and cons. 15 March 2011
  3. ↑ Stanley Zaffos, Adam W. Couture. Hybrid Cloud Gateway Appliances Expand Cloud Storage Use Cases. Gartner Group Research G00209453. http://storageswiss.com/2014/05/09/hybrid-cloud-storage-vs-the-enterprise/10 January 2011[ permanent dead link ]
  4. ↑ SNIA. "Object Storage – Understanding the What, How and Why" (PDF).
  5. ↑ "5 Ways cloud technology improves your business | Austin Technology | Perth". Austin Technology. 2017-02-10. Retrieved 2019-04-16.
  6. ↑ Spillner, J (2012). "Flexible data distribution policy language and gateway architecture". 2012 IEEE Latin America Conference on Cloud Computing and Communications (LatinCloud). pp. 1–6. doi:10.1109/LatinCloud.2012.6508149. ISBN   978-1-4673-5163-8. S2CID   16839846.
  7. ↑ Kovacs, Gali. "How to Mount Amazon S3 Buckets as a Local Drive". cloud.netapp.com. Retrieved 2019-04-11.
  8. ↑ Columbus, Louis. "Roundup Of Cloud Computing Forecasts, 2017". Forbes. Retrieved 2019-04-16.
  9. ↑ CLOUD STORAGE GATEWAY MARKET - GROWTH, TRENDS, COVID-19 IMPACT, AND FORECASTS (2021 - 2026)
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.