Endace

Last updated

Endace Ltd
Type Private
Industry Network monitoring
Founded2001
Headquarters Auckland,
New Zealand
Key people
Stuart Wilson: CEO
Website www.endace.com

Endace Ltd is a privately owned network monitoring company, based in New Zealand and founded in 2001. [1] It provides network visibility and network recording products to large organizations. The company was listed on the London Stock Exchange in 2005 and then delisted in 2013 when it was acquired by Emulex. [2] In 2016 Endace was spun out of Emulex and is currently a private company. [3]

Contents

In October 2016, The Intercept revealed that some Endace clients were intelligence agencies, including the British GCHQ (known for conducting massive surveillance on network communications) and the Moroccan DGST, likewise known for mass surveillance of its citizens.

Background and history

Endace was founded after the DAG project at the School of Computing and Mathematical Sciences at the University of Waikato in New Zealand. [1] [4] The first cards designed at the university were intended to measure latency in ATM networks. [5]

In 2006, Endace transitioned from component manufacturer to appliance manufacturer to managed infrastructure provider. The company now sells network visibility fabrics, based on its range of network recorders, to large corporations and government agencies.

Endace was the first New Zealand company to list on London's Alternative Investment Market when it floated in mid-June 2005 [6] a move which was not without controversy. [7] Poor share price performance in the early years and a seeming failure to attract a broad enough shareholder base lent weight to the criticism that Endace should have focused initially on developing its local profile (via NZX) rather than pushing for overseas investment (via London AIM).

Endace is headquartered in Auckland, New Zealand, and has an R&D centre in Hamilton, New Zealand, and offices in Australia, United States and Great Britain.

Key innovations of the DAG

The DAG project grew from academic research at Waikato University. Having found that software measurements of ATM cells (or packets) were unsatisfactory, both for reasons of accuracy and lack of certainty about packet loss, the research group set about developing their own hardware to generate better quality recordings. [5] This hardware and its subsequent iterations introduced two fundamental innovations: hardware timestamping and hardware accounting for packet loss.

Hardware timestamping

Conventionally, each packet or cell is given a timestamp by the host machine's kernel (i.e. in software) when the kernel driver is notified that a new packet has arrived. This approach results in poor quality timestamps for several reasons, among them the considerable latency and jitter between the packet arriving at the network interface and receipt by the kernel driver and uncertainty caused by interrupt coalescing wherein one host interrupt signifies the arrival of several packets. Such poor quality limits what research can usefully be done on network performance and related fields.

To solve this, the DAG generates timestamps in the hardware as close to the network interface as possible. Not only does this obviate latency, jitter and problems caused by interrupt coalescing, the hardware is capable of much greater accuracy and precision than software-generated timestamps. Precision comes from the freedom of custom hardware to assign as many bits to the timestamp as required and accuracy is assured by reference to an external time source such as GPS which is accurate to ± 40 nanoseconds. [8] In contrast, the accuracy of NTP (by which kernel clocks can be corrected over the Internet) is in the order of milliseconds (about 100,000 times less accurate), depending on the conditions involved.

The DAG produces 64 bit timestamps in fixed-point format with 32 fractional bits, giving a potential precision of seconds or 233 picoseconds. The actual precision offered varies with the particular model of DAG, the oldest giving 24 fractional bits (60 nanoseconds) and better precisions offered in DAGs for higher bandwidth networks. [9]

The timestamp is derived from a free-running clock provided by a crystal oscillator but the accuracy of crystals drift with both temperature and age. The DAG's solution is to use direct digital synthesis using the 1 Hz pulse-per-second output that many GPS receivers provide as its reference clock. This mechanism is described in §5.5.3 of Stephen Donnelly's PhD thesis [10] which also describes in detail the pre-commercial era models of DAG.

Crucially, and an academically significant contribution of the DAG, the ability to use an external reference such as globally synchronised GPS makes it possible to do one-way time-of-flight measurements. This is of immense interest to academic researchers because packets flowing between two points on the Internet are neither guaranteed to follow the same path in each direction nor guaranteed to have the same timing characteristics in each direction.

Outside of the academic world, timestamp accuracy has commercial applications in the enforcement and compliance with law such as the EU Markets in Financial Instruments Directive 2004.

Packet loss

Almost as important as timestamp accuracy is guaranteeing 100% cell or packet capture and, where loss is unavoidable, knowing not only that packets have been lost but where. The "where" is important because, when analysing a packet trace, it's important to be able to compensate for lost packets when calculating inter-arrival times.

Most commercial NICs keep a count of dropped packets, but they can't indicate where packets were lost. The DAG prepends a header [11] which, amongst other things, indicates how many packets were dropped between that packet and the previously accepted packet.

The DAG is also engineered to deliver recorded packets to the host with the greatest possible efficiency. That, together with the interstitial loss counter, is what makes the DAG so appealing for surveillance applications. The interstitial loss counter also finds application in forensics; a prosecutor needs to be able to prove that the record is complete or, if it is not, where it is not.

Controversy and surveillance

In October 2016, The Intercept published an article showing that Endace customers include intelligence agencies, including the GCHQ, Canadian and Australian intelligence agencies, and the DGST (Morocco's domestic surveillance agency). [12] Edward Snowden documents have shown that the GCHQ has installed massive surveillance of network communications in UK, using the over-sea cable between Europe and North America.

Awards

In March 2020, Endace received awards for "Most Innovative", "Best Product" and "Hot Company" categories at the Cyber Defense Magazine InfoSec Awards. [13] [14]

Also in March 2020, Endace was awarded the "Grand Trophy Winner" for winning several categories in the Info Security Products Guide Global Excellence Awards. The company was awarded the Gold award for "Best Security Hardware", "Most Innovative Security Hardware of the Year", "Network Security and Management", and "Critical Infrastructure Security" categories as well as the Silver award for "Best Security Solution" and "Network Visibility, Security & Testing" categories. [15] [14]

Related Research Articles

Latency, from a general point of view, is a time delay between the cause and the effect of some physical change in the system being observed. Lag, as it is known in gaming circles, refers to the latency between the input to a simulation and the visual or auditory response, often occurring because of network delay in online games.

The Transmission Control Protocol (TCP) is one of the main protocols of the Internet protocol suite. It originated in the initial network implementation in which it complemented the Internet Protocol (IP). Therefore, the entire suite is commonly referred to as TCP/IP. TCP provides reliable, ordered, and error-checked delivery of a stream of octets (bytes) between applications running on hosts communicating via an IP network. Major internet applications such as the World Wide Web, email, remote administration, and file transfer rely on TCP, which is part of the Transport Layer of the TCP/IP suite. SSL/TLS often runs on top of TCP.

In electronics and telecommunications, jitter is the deviation from true periodicity of a presumably periodic signal, often in relation to a reference clock signal. In clock recovery applications it is called timing jitter. Jitter is a significant, and usually undesired, factor in the design of almost all communications links.

Voice over Internet Protocol (VoIP), also called IP telephony, is a method and group of technologies for voice calls, the delivery of voice communication sessions over Internet Protocol (IP) networks, such as the Internet.

<span class="mw-page-title-main">Network Time Protocol</span> Standard protocol for synchronizing time across devices

The Network Time Protocol (NTP) is a networking protocol for clock synchronization between computer systems over packet-switched, variable-latency data networks. In operation since before 1985, NTP is one of the oldest Internet protocols in current use. NTP was designed by David L. Mills of the University of Delaware.

<span class="mw-page-title-main">Network interface controller</span> Hardware component that connects a computer to a network

A network interface controller is a computer hardware component that connects a computer to a computer network.

TCP offload engine (TOE) is a technology used in some network interface cards (NIC) to offload processing of the entire TCP/IP stack to the network controller. It is primarily used with high-speed network interfaces, such as gigabit Ethernet and 10 Gigabit Ethernet, where processing overhead of the network stack becomes significant. TOEs are often used as a way to reduce the overhead associated with Internet Protocol (IP) storage protocols such as iSCSI and Network File System (NFS).

Network performance refers to measures of service quality of a network as seen by the customer.

Professional video over IP systems use some existing standard video codec to reduce the program material to a bitstream, and then to use an Internet Protocol (IP) network to carry that bitstream encapsulated in a stream of IP packets. This is typically accomplished using some variant of the RTP protocol.

AES47 is a standard which describes a method for transporting AES3 professional digital audio streams over Asynchronous Transfer Mode (ATM) networks.

The Precision Time Protocol (PTP) is a protocol used to synchronize clocks throughout a computer network. On a local area network, it achieves clock accuracy in the sub-microsecond range, making it suitable for measurement and control systems. PTP is employed to synchronize financial transactions, mobile phone tower transmissions, sub-sea acoustic arrays, and networks that require precise timing but lack access to satellite navigation signals.

Capacity management's goal is to ensure that information technology resources are sufficient to meet upcoming business requirements cost-effectively. One common interpretation of capacity management is described in the ITIL framework. ITIL version 3 views capacity management as comprising three sub-processes: business capacity management, service capacity management, and component capacity management.

Bandwidth management is the process of measuring and controlling the communications on a network link, to avoid filling the link to capacity or overfilling the link, which would result in network congestion and poor performance of the network. Bandwidth is described by bit rate and measured in units of bits per second (bit/s) or bytes per second (B/s).

In capital markets, low latency is the use of algorithmic trading to react to market events faster than the competition to increase profitability of trades. For example, when executing arbitrage strategies the opportunity to "arb" the market may only present itself for a few milliseconds before parity is achieved. To demonstrate the value that clients put on latency, in 2007 a large global investment bank has stated that every millisecond lost results in $100m per annum in lost opportunity.

HPC Challenge Benchmark combines several benchmarks to test a number of independent attributes of the performance of high-performance computer (HPC) systems. The project has been co-sponsored by the DARPA High Productivity Computing Systems program, the United States Department of Energy and the National Science Foundation.

PTPd is an open source implementation of the Precision Time Protocol for Unix-like computers.

RTP-MIDI is a protocol to transport MIDI messages within Real-time Transport Protocol (RTP) packets over Ethernet and WiFi networks. It is completely open and free, and is compatible both with LAN and WAN application fields. Compared to MIDI 1.0, RTP-MIDI includes new features like session management, device synchronization and detection of lost packets, with automatic regeneration of lost data. RTP-MIDI is compatible with real-time applications, and supports sample-accurate synchronization for each MIDI message.

<span class="mw-page-title-main">Bullrun (decryption program)</span> Code name of a decryption program run by the NSA

Bullrun is a clandestine, highly classified program to crack encryption of online communications and data, which is run by the United States National Security Agency (NSA). The British Government Communications Headquarters (GCHQ) has a similar program codenamed Edgehill. According to the Bullrun classification guide published by The Guardian, the program uses multiple methods including computer network exploitation, interdiction, industry relationships, collaboration with other intelligence community entities, and advanced mathematical techniques.

AES67 is a technical standard for audio over IP and audio over Ethernet (AoE) interoperability. The standard was developed by the Audio Engineering Society and first published in September 2013. It is a layer 3 protocol suite based on existing standards and is designed to allow interoperability between various IP-based audio networking systems such as RAVENNA, Livewire, Q-LAN and Dante.

chrony Network Time Protocol implementation

chrony is an implementation of the Network Time Protocol (NTP). It is an alternative to ntpd, a reference implementation of NTP. It runs on Unix-like operating systems and is released under the GNU GPL v2. It is the default NTP client and server in Red Hat Enterprise Linux 8 and SUSE Linux Enterprise Server 15, and available in many Linux distributions.

References

  1. 1 2 "The DAG Project". Archived from the original on 29 November 2001.
  2. ↑ "ENDACE LTD (EDA:NL): Company Description - BusinessWeek". Bloomberg Businessweek investing database. Bloomberg L.P. Archived from the original on 10 October 2012. Retrieved 9 February 2011.
  3. ↑ "Endace Spins off from Emulex in Management-led Buyout". New Zealand: Endace. 10 March 2016. Retrieved 13 March 2016.
  4. ↑ "Yoke Har Lee: Life's a bit of a DAG for hi-tech firm". The New Zealand Herald . 24 August 2009. Retrieved 11 September 2011.
  5. 1 2 Cleary, John; Donnelly, Stephen; Graham, Ian; McGregor, Anthony; Pearson, Murray. Design Principles for Accurate Passive Measurement (PDF) (Report). Waikato University. Retrieved 13 May 2017.
  6. ↑ "Growth Business: Endace poised to take AIM". Archived from the original on 19 November 2005.
  7. ↑ Inder, Richard (5 June 2006). "Endace's performance on UK AIM listing gives fuel to critics". The New Zealand Herald . Retrieved 11 September 2011.
  8. ↑ "§A.4.8 UTC(USNO) Offset Accuracy". Global Positioning System Standard Positioning Service Performance Standard (PDF) (Report) (4th ed.). US Department of Defense. September 2008. p. A-16. Retrieved 13 May 2017.
  9. ↑ Micheel, Jörg; Donnelly, Stephen; Graham, Ian (2001). "Precision timestepping of network packets" (PDF). Proceedings of the First ACM SIGCOMM Workshop on Internet Measurement – IMW '01. Waikato University. p. 273. doi:10.1145/505202.505236. ISBN   1581134355. S2CID   14567389. Archived from the original (PDF) on 25 February 2018. Retrieved 13 May 2017.
  10. ↑ Donnelly, Stephen F. (2002). High Precision Timing in Passive Measurements of Data Networks (PhD). CiteSeerX   10.1.1.136.1730 .
  11. ↑ ""Extensible Record Format" header description". WireShark. Retrieved 13 May 2017.
  12. ↑ "The Little-Known Company That Enables Worldwide Mass Surveillance". The Intercept. 23 October 2016. Retrieved 2 November 2016.
  13. ↑ "INFOSEC AWARDS FOR 2020 – WINNERS". cyberdefenseawards.com. Retrieved 3 March 2020.
  14. 1 2 "Endace Wins Big in Cyber Defense Magazine and Info Security Products Guide Awards". businesswire.com. 2 March 2020. Retrieved 3 March 2020.
  15. ↑ "2020 Winners". Info Security Products Guide. Retrieved 3 March 2020.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.