Website monitoring

Last updated

Website monitoring is the process of testing and verifying that end-users can interact with a website or web application as expected. Website monitoring are often used by businesses to ensure website uptime, performance, and functionality is as expected.

Contents

Website monitoring companies provide organizations the ability to consistently monitor a website or server function and observe how it responds. The monitoring is often conducted from several locations around the world to a specific website, or server, to detect issues related to general Internet latency, and network hop issues, and to prevent false positives caused by local or inter-connect problems. Monitoring companies generally report on these tests in a variety of reports, charts, and graphs. When an error is detected monitoring services send out alerts via email, SMS, phone, SNMP trap, a pager that may include diagnostic information, such as a network traceroute, code capture of a web page's HTML file, a screenshot of a webpage, and even a video of a website failing. These diagnostics allow network administrators and webmasters to correct issues faster.

Monitoring gathers extensive data on website performance, such as load times, server response times, and page element performance that is often analyzed and used to further optimize website performance.

Purpose

Monitoring is essential to ensure that a website is available to users, downtime is minimized, and performance can be optimized. Users that rely on a website or an application for work or pleasure will get frustrated or even stop using the application if it is not reliably available. [1] Monitoring can cover many things that an application needs to function, like network connectivity, Domain Name System records, database connectivity, bandwidth, and computer resources like free RAM, CPU load, disk space, events, etc. Commonly measured metrics are response time and availability (or uptime), but consistency and reliability metrics are gaining popularity. Measuring a website's availability and reliability under various amounts of traffic is often referred to as load testing.

Website monitoring also helps benchmark the website against the performance of a competitor to help determine how well a site is performing. Website speed is also used as a metric for search engine rankings. [2]

Website monitoring can be used to hold web hosting providers accountable for their service-level agreements. Most web hosts offer a 99.9% uptime guarantee and when uptime is less than that, individuals can be refunded for the excessive downtime. Not all hosts will refund individuals for excessive downtime so one must become familiar with the terms of service of their host. [3]

Most paid website monitoring services will also offer security features such as virus and malware scanning which is of growing importance as websites become more complicated and integral to the business.

Internal vs. external

Website monitoring can be done from both inside and outside of a corporate firewall. Traditional network management solutions focus on inside the firewall monitoring, whereas external performance monitoring will test and monitor performance issues across the Internet backbone and in some cases all the way to the end-user. Third-party website performance monitoring solutions can monitor internal (behind the firewall), external (customer-facing), or cloud-based Web applications.

Inside the firewall, monitoring is done by special hardware appliances which can help you determine if your internal applications’ sluggish performance is caused by: the design of applications, internal infrastructure, internal applications or connections to any public Internet.

External performance monitoring is also known as end-user monitoring or end-to-end performance monitoring. [4]

Real user monitoring measures the performance and availability experienced by actual users, diagnoses individual incidents, and tracks the impact of a change.

Measures of website availability

Availability Nines Downtime per year
90%1 nine876 hours
95%438 hours
99%2 nines87 hours, 36 minutes
99.9%3 nines8 hours, 45 minutes, 36 seconds
99.99%4 nines52 minutes, 33.6 seconds
99.999%5 nines5 minutes, 15.36 seconds
99.9999%6 nines31.68 seconds

Types of protocol

A website monitoring service can check other internet protocols besides HTTP pages and HTTPS such as FTP, SMTP, POP3, ActiveSync, IMAP, DNS, SSH, Telnet, SSL, TCP, PING, UDP, SOAP, Domain Name Expiry, SSL Certificate Expiry and a range of ports. Monitoring frequency occurs at intervals of once every 4-hours to every 15-seconds. Typically, most website monitoring services test a server, or application, between a once-per-hour per once-per-minute.

Advanced monitoring services capture browser interactions with websites using macro recorders, or browser add-ons such as Selenium or iMacros. These services test a website by running a web browser through a typical website transaction (such as a shopping cart) or a custom scenario, in order to check for user experience issues, performance problems, and availability errors. Browser-driven monitoring services detect not only network and server issues, but also webpage object issues (such as slow loading JavaScript, or third-party hosted page elements).

Implementation of time performance monitoring for the Apache HTTP Server is the mod_arm4 [5] module.

Types of monitoring

Users of website monitoring (typically network administrators, webmasters, web operations personnel) may monitor a single page of a website, but can also monitor a complete business process (often referred to as multi-step transactions). [6]

Servers monitoring from around the globe

Website monitoring services usually have a number of servers around the globe – South America, Africa, North America, Europe, Africa, Asia, Australia and other locations. By having multiple servers in different geographic locations, a monitoring service can determine if a web server is available across continents over the Internet. Some vendors claim that the more locations the better picture of your website availability while others say that three globally distributed stations are sufficient and more stations do not give more information.

Types

There are two main types of website monitoring

Levels of Website Monitoring

There are different levels of website monitoring, the more complex your website, the more complex your monitoring needs:

Notification options: alerts

As the information brought by website monitoring services is in most cases urgent and may be of crucial importance, various notification methods, often known as "alerts" are used: e-mail, IM, regular and cell phones, SMS, fax, pagers, Skype, RSS Feed, SNMP trap, URL notifications, etc.

Website monitoring services

The Website monitoring market is very competitive. There are 150+ active service providers [7] and more than 100 are documented to have gone out of business. [8] Most of the providers offer a free plan with low-frequency monitoring.

In recent years Synthetic Monitoring services have become widely available commercially, providing another vehicle to monitor the performance of specific web properties at a granular level.

See also

Related Research Articles

In computing, traceroute and tracert are computer network diagnostic commands for displaying possible routes (paths) and measuring transit delays of packets across an Internet Protocol (IP) network. The history of the route is recorded as the round-trip times of the packets received from each successive host in the route (path); the sum of the mean times in each hop is a measure of the total time spent to establish the connection. Traceroute proceeds unless all sent packets are lost more than twice; then the connection is lost and the route cannot be evaluated. Ping, on the other hand, only computes the final round-trip times from the destination point.

In software quality assurance, performance testing is in general a testing practice performed to determine how a system performs in terms of responsiveness and stability under a particular workload. It can also serve to investigate, measure, validate or verify other quality attributes of the system, such as scalability, reliability and resource usage.

<span class="mw-page-title-main">Load balancing (computing)</span> Set of techniques to improve the distribution of workloads across multiple computing resources

In computing, load balancing is the process of distributing a set of tasks over a set of resources, with the aim of making their overall processing more efficient. Load balancing can optimize the response time and avoid unevenly overloading some compute nodes while other compute nodes are left idle.

<span class="mw-page-title-main">Proxy server</span> Computer server that makes and receives requests on behalf of a user

In computer networking, a proxy server is a server application that acts as an intermediary between a client requesting a resource and the server providing that resource. It improves privacy, security, and performance in the process.

<span class="mw-page-title-main">Web hosting service</span> Service for hosting websites

A web hosting service is a type of Internet hosting service that hosts websites for clients, i.e. it offers the facilities required for them to create and maintain a site and makes it accessible on the World Wide Web. Companies providing web hosting services are sometimes called web hosts.

In computer security, a DMZ or demilitarized zone is a physical or logical subnetwork that contains and exposes an organization's external-facing services to an untrusted, usually larger, network such as the Internet. The purpose of a DMZ is to add an additional layer of security to an organization's local area network (LAN): an external network node can access only what is exposed in the DMZ, while the rest of the organization's network is protected behind a firewall. The DMZ functions as a small, isolated network positioned between the Internet and the private network.

<span class="mw-page-title-main">Load testing</span> Process of putting demand on a system and measuring its response

Load testing is the process of putting demand on a structure or system and measuring its response.

<span class="mw-page-title-main">Port forwarding</span> Computer networking feature

In computer networking, port forwarding or port mapping is an application of network address translation (NAT) that redirects a communication request from one address and port number combination to another while the packets are traversing a network gateway, such as a router or firewall. This technique is most commonly used to make services on a host residing on a protected or masqueraded (internal) network available to hosts on the opposite side of the gateway, by remapping the destination IP address and port number of the communication to an internal host.

Internet security is a branch of computer security. It encompasses the Internet, browser security, web site security, and network security as it applies to other applications or operating systems as a whole. Its objective is to establish rules and measures to use against attacks over the Internet. The Internet is an inherently insecure channel for information exchange, with high risk of intrusion or fraud, such as phishing, online viruses, trojans, ransomware and worms.

An application firewall is a form of firewall that controls input/output or system calls of an application or service. It operates by monitoring and blocking communications based on a configured policy, generally with predefined rule sets to choose from. The two primary categories of application firewalls are network-based and host-based.

Network monitoring is the use of a system that constantly monitors a computer network for slow or failing components and that notifies the network administrator in case of outages or other trouble. Network monitoring is part of network management.

A dedicated hosting service, dedicated server, or managed hosting service is a type of Internet hosting in which the client leases an entire server not shared with anyone else. This is more flexible than shared hosting, as organizations have full control over the server(s), including choice of operating system, hardware, etc.

High availability (HA) is a characteristic of a system that aims to ensure an agreed level of operational performance, usually uptime, for a higher than normal period.

In software design, web design, and electronic product design, synthetic monitoring is a monitoring technique that is done by using a simulation or scripted recordings of transactions. Behavioral scripts are created to simulate an action or path that a customer or end user would take on a site, application, or other software. Those paths are then continuously monitored at specified intervals for performance, such as functionality, availability, and response time measures.

The term downtime is used to refer to periods when a system is unavailable. The unavailability is the proportion of a time-span that a system is unavailable or offline. This is usually a result of the system failing to function because of an unplanned event, or because of routine maintenance.

<span class="mw-page-title-main">Proxy list</span>

A proxy list is a list of open HTTP/HTTPS/SOCKS proxy servers all on one website. Proxies allow users to make indirect network connections to other computer network services. Proxy lists include the IP addresses of computers hosting open proxy servers, meaning that these proxy servers are available to anyone on the internet. Proxy lists are often organized by the various proxy protocols the servers use. Many proxy lists index, which can be used without changing browser settings.

An application delivery controller (ADC) is a computer network device in a datacenter, often part of an application delivery network (ADN), that helps perform common tasks, such as those done by web accelerators to remove load from the web servers themselves. Many also provide load balancing. ADCs are often placed in the DMZ, between the outer firewall or router and a web farm.

DNS hijacking, DNS poisoning, or DNS redirection is the practice of subverting the resolution of Domain Name System (DNS) queries. This can be achieved by malware that overrides a computer's TCP/IP configuration to point at a rogue DNS server under the control of an attacker, or through modifying the behaviour of a trusted DNS server so that it does not comply with internet standards.

In computing, a firewall is a network security system that monitors and controls incoming and outgoing network traffic based on predetermined security rules. A firewall typically establishes a barrier between a trusted network and an untrusted network, such as the Internet.

Web performance refers to the speed in which web pages are downloaded and displayed on the user's web browser. Web performance optimization (WPO), or website optimization is the field of knowledge about increasing web performance.

References

  1. Amazon Downtime Cost $66,000 - Forbes
  2. Speed Affects Website Usage - Google Research Blog
  1. Stevenson, Seth (June 2012). "What You Hate Most About Waiting in Line". Slate . Retrieved 25 October 2014.
  2. Costill, Albert (25 July 2014). "SEO 101: How Important is Site Speed in 2014?". Search Engine Journal. Retrieved 25 October 2014.
  3. Tryn, Joe. "For newbies : 99.5% / 99.9% uptime guarantee explained" . Retrieved 29 October 2014. Also, do know how committed a host is in honouring their guarantee. Do they just mention it in their marketing channels, eg websites etc. Or do they really specify it clearly in their "terms of service" (ToS) ? What kind of compensation they plan to honour if they exceed the agreed upon max downtime ? Not all host are willing to compensate for exceeding the max downtime. Honest and responsible hosts will refund the hosting fees you paid for a particular month upon you reporting to them that they had exceeded the max downtime - and this is stated clearly in their ToS.
  4. In-House or Remote Network Monitoring
  5. Apache ARM 4 module
  6. "Network Monitoring Tools". www.slac.stanford.edu. Retrieved 2021-04-09.
  7. "The Updated List of 200+ Website Monitoring Services". 12 October 2015.
  8. "230+ Website Monitoring Services that Shut Down". 20 October 2015.