A news server is a collection of software used to handle Usenet articles. [1] It may also refer to a computer itself which is primarily or solely used for handling Usenet. Access to Usenet is only available through news server providers.
A news server is a collection of software used to handle Usenet articles. [1] It may also refer to a computer itself which is primarily or solely used for handling Usenet. Access to Usenet is only available through news server providers.
End users often use the term "posting" to refer to a single message or file posted to Usenet. For articles containing plain text, this is synonymous with an article. For binary content such as pictures and files, it is often necessary to split the content among multiple articles. Typically through the use of numbered Subject: headers, the multiple-article postings are automatically reassembled into a single unit by the newsreader. Most servers do not distinguish between single and multiple-part postings, dealing only at the level of the individual component articles. [2]
Each news article contains a complete set of header lines, but in common use the term "headers" is also used when referring to the News Overview database. [2] The overview is a list of the most frequently used headers, and additional information such as article sizes, typically retrieved by the client software using the NNTP XOVER command. Overviews make reading a newsgroup faster for both the client and server by eliminating the need to open each individual article to present them in list form.
If non-overview headers are required, such as for when using a kill file, it may still be necessary to use the slower method of reading all the complete article headers. [1] Many clients are unable to do this, and limit filtering to what is available in the summaries. [2]
Among the operators and users of commercial news servers, common concerns are the continually increasing storage and network capacity requirements and their effects. [2] Completion (the ability of a server to successfully receive all traffic), retention (the amount of time articles are made available to readers) and overall system performance. With the increasing demands, it is common for the transit and reader server roles to be subdivided further into numbering, storage and front end systems. These server farms are continually monitored by both insiders and outsiders, and measurements of these characteristics are often used by consumers when choosing a commercial news service.
Speed, in relation to Usenet, is how quickly a server can deliver an article to the user. The server that the user connects to is typically part of a server farm that has many servers dedicated to multiple tasks. How fast the data can move throughout this farm is the first thing that affects the speed of delivery.[ citation needed ]
The speed of data traveling throughout the farm can be severely bottlenecked through hard drive operations. Retrieving the article and overview information can cause massive stress on hard drives.[ citation needed ] To combat this, caching technology and cylindrical file storage systems have been developed. [ citation needed ]
Once the farm is able to deliver the data to the network, then the provider has limited control over the speed to the user. Since the network path to each user is different, some users will have good routes and the data will flow quickly. Other users will have overloaded routers between them and the provider which will cause delays. About all a provider can do in that case is try moving the traffic through a different route. If the ISP has limited connectivity to the network, routing changes may have little effect.
Frequently a user can reduce the impact of network problems by using multiple connections. Some servers allow as many as 60 simultaneous connections, but this varies widely based on the provider. [3]
Article sizes are limited to what each news server will accept. The larger the article size, the more space it occupies, and thus the fewer articles on each server. This generally means that a server can run with less overhead which makes for a more efficient server, but gives less articles for users to access.[ citation needed ]
Retention is simply defined as how long the server keeps articles. [4] Historically, most users want retention to be long enough so that they don't need to access the server every day but not overly long retention that can overwhelm users with slow computers or network connections. [1] In the modern era, high speed connections, large storage capacity, and advanced search tools allows users to utilize extensive retention without any drawbacks.
Retention is generally quoted separately for text and binary articles, though it may also vary between different groups within these categories. The times vary greatly according to the amount of storage available on the servers and continually increasing traffic. As of 2009, it is common for average news providers to have text retention of over 1000 days and binary retention of over 200 days.[ citation needed ] Large news providers offer text retention up to 2480 days and binary retention of 850 days or more.[ citation needed ] It's important to understand that retention time varies between different newsgroups within the text and binary categories. Omicron's HW Media is currently the Usenet server with the highest amount of binary retention, while Google is the Usenet server with the highest amount of text retention.[ citation needed ]
It can be difficult for end users to accurately measure the retention of a server. One common method is to examine the oldest articles in a group and examine the date, but this is not always accurate. Some articles in a group may be retained for longer than others, articles from remote servers do not always arrive promptly, and at times the date headers are simply incorrect. A sampling of many or all articles, preferably in more than one newsgroup, is required to detect such anomalies.
News servers do not have unlimited storage, and due to this fact they can only hold posts for a length of time before they must delete them in order to make room for new posts. This is a particular problem to binary newsgroups which transmit large volumes of articles.
For news servers provided by Internet Service Providers as part of a user's subscription package, typical retention rates are usually only 2–4 days.[ citation needed ] To deal with the increase of Usenet traffic, many providers turn to a hybrid system, in which old articles not found on the provider's server will request the article from another server with longer retention.
Given the large number of articles transferred between servers and the large size of individual articles, their complete propagation to any one server farm is not guaranteed. The term "completion" is used to describe how well a service is keeping up with the traffic.[ citation needed ]
The primary obstacle to calculating the completion percentage is how many articles were posted. Looking at only one server, one cannot know how many articles were actually inserted throughout the network.[ citation needed ] Articles may never make their way outside the originating server, or may fail to find their way out to the transit cloud. Very large articles are frequently dropped, and tend to propagate less well than smaller ones.[ citation needed ]
One way to measure completion is to access multiple servers and retrieve lists of articles. Because Message-ID: headers are nominally unique throughout the network, comparison of the lists is mostly a straightforward task. Practical limitations to this type of measurement include the impossibility of obtaining lists from all servers worldwide, the fact that many servers filter out spam or employ Usenet Death Penalties, and that some servers mask incompletion by hiding multipart binary sets with missing articles.[ citation needed ] It is also necessary to take into account propagation times and retention; an article may simply have not yet arrived at a given server, or it may have been present but already expired. [ citation needed ]
All Usenet servers peer with one or more other servers in order to exchange articles. Occasionally, new servers appear. Although there are several web resources which may aid in finding peers, a better resource is the newsgroup news.admin.peering (Google Groups portal). [ citation needed ]
As of 2020, text feeds can usually be attained for free, while full binary feeds can be free or paid (depending on how many articles each server sends to the other). Due to the large amount of data in a full binary+text Usenet feed (can be high as 30 terabytes a day) and the high costs of transmitting that data through an IP transit provider like Cogent, Telia, or Zayo, most Usenet providers will only engage in binary peering when they are interconnected at an Internet exchange like AMS-IX, SIX, or DeCIX.
When the server stores the body of an article, it places it in a disk storage area generically called a "spool". [2] There are several common ways in which the spool may be organized:
A reader server provides an interface to read and post articles, generally with the assistance of a news client. A transit server exchanges articles with other servers. Most servers can provide both functions.
Modern transit servers usually use NNTP to exchange news continually over the Internet and similar always-on connections. In the past, servers normally employed the UUCP protocol, which was designed for intermittent dial-up connections. Other ad hoc protocols, including e-mail, are less commonly seen. News servers normally connect with multiple peers, with the redundancy helping to spread loads and ensure that articles are not lost. Smaller sites, called leaf nodes, are connected to one other major server. [2]
Articles are routed based on information found in the header lines defined in RFC 1036.[ citation needed ] Of particular interest to a transit server are:
In most cases, the sending server controls the article transfer process. It compares the Newsgroups and Distribution of each newly arrived article against a set of patterns called newsfeeds, listing each remote server and the newsgroups its operator wishes to receive. Some senders also examine the Path; if the receiving server appears in this line, it is not offered. Other local rules may also be added. The sender transmits matching articles' Message-IDs to the receiving server. The receiver indicates which Message-IDs it has not yet stored locally, and those articles are sent. [2]
The receiving server examines the incoming articles. A message is normally discarded if the Message-ID is duplicated by an article already received (i.e., another server sent it in the meantime), the Date or Expires lines indicate that the article is too old, the header syntax appears to be invalid, the Approved header is missing for a moderated newsgroup, or additional local rules disallow it.[ citation needed ] Most servers also maintain a list of active newsgroups. If the Newsgroups header of a new article does not match the active list, it may be discarded or placed in a special "junk" newsgroup. Once the article is stored, the server attempts to retransmit it to any servers in its own newsfeed list. [2]
Articles with Control lines are given special handling. They are typically filed in special "control" newsgroups and may cause the server to automatically carry out exceptional actions. The newgroup and rmgroup commands can cause newsgroups to be created or removed; checkgroups can be used to reconcile the local active list with a commonly accepted set; and cancel commands are used to request the deletion of a specific article. ihave and sendme are sometimes used with UUCP to transmit lists of offered and wanted Message-IDs. Other commands (version, sendsys, and uuname) are requests for server configuration details. Once used to create network maps, they now are generally obsolete. [2]
A reader server is one that makes the articles available in the hierarchical disk directory format originated by B News 2.10, or offers the NNTP or IMAP commands, for use by newsreaders. A reader server typically also works as a transit server, but it may operate independently or serve as an alternative interface to an Internet forum. When receiving news, this type of server must perform the additional steps of filing articles into newsgroups and assigning sequential numbers within each group. An Xref line is usually added, listing all the groups where the message appears and the sequence numbers. Unlike Message IDs, the numbers and ordering of articles will differ on each server; but related servers may force agreement by operating in a slave mode, re-using their siblings' Xref lines. Reader servers typically also maintain a News Overview (NOV) database that allows newsreaders to quickly obtain message summaries and present messages in threaded form. [2]
Most reader servers support posting, either through NNTP or a special inews program.[ citation needed ] When an article is posted, the process is much the same as when a transit server receives news, but with additional checks. For posting, the server will normally fill in missing Path and Message-ID lines and check the syntax of headers intended for human readers, such as From and Subject. If the article is posted to a moderated group, the server will attempt to mail it to the newsgroup moderator if the Approved header is absent. Additional identity checks and filters are also typically applied at this point. [2]
Smaller sites with limited network bandwidth may operate "sucking" or cache servers. These perform the same reader server role as conventional news servers, but themselves act as newsreaders to exchange articles with other reader servers.[ citation needed ] Hybrid servers allow greater flexibility for the server operator in that received groups can be adjusted without manual intervention by operators. They may also be the only available means to obtain articles from remote servers that do not offer conventional feeding.
Because hybrid servers usually use the posting function to send news, article headers are reformatted by the posting function and tracing information can be lost. Also, the delayed sucking process can result in excess activity on the remote reader servers. For these reasons, the use of hybrid servers is often discouraged or disallowed without prior agreement. [2]
A Usenet newsgroup is a repository usually within the Usenet system, for messages posted from users in different locations using the Internet. They are discussion groups and are not devoted to publishing news. Newsgroups are technically distinct from, but functionally similar to, discussion forums on the World Wide Web. Newsreader software is used to read the content of newsgroups.
rn is a news client written by Larry Wall and originally released in 1984. It was one of the first newsreaders to take full advantage of character-addressable CRT terminals. Previous newsreaders, such as readnews, were mostly line-oriented and designed for use on the printing terminals which were common on the early Unix minicomputers where the Usenet software and network originated. Later variants of the original rn program included rrn, trn, and strn.
The Network News Transfer Protocol (NNTP) is an application protocol used for transporting Usenet news articles (netnews) between news servers, and for reading/posting articles by the end user client applications. Brian Kantor of the University of California, San Diego, and Phil Lapsley of the University of California, Berkeley, wrote RFC 977, the specification for the Network News Transfer Protocol, in March 1986. Other contributors included Stan O. Barber from the Baylor College of Medicine and Erik Fair of Apple Computer.
The alt.* hierarchy is a major class of newsgroups in Usenet, containing all newsgroups whose name begins with "alt.", organized hierarchically. The alt.* hierarchy is not confined to newsgroups of any specific subject or type, although in practice more formally organized groups tend not to occur in alt.*. The alt.* hierarchy was created by John Gilmore and Brian Reid.
A News, or Netnews Version A, originally known simply as news, was the first widely distributed program for serving and reading Usenet newsgroups. The program, written at Duke University by Steve Daniel and Tom Truscott, was released on a tape given out at the June 1980 USENIX conference held at the University of Delaware. Steve Daniel from Duke offered a presentation on the then-new Usenet network and invited attendees to join.
Sporgery is the disruptive act of posting a flood of articles to a Usenet newsgroup, with the article headers falsified so that they appear to have been posted by others. The word is a portmanteau of spam and forgery, coined by German software developer, and critic of Scientology, Tilman Hausherr.
X-No-Archive, also known colloquially as xna, is a newsgroup message header field used to prevent a Usenet message from being archived in various servers.
On Usenet, the Usenet Death Penalty (UDP) is a final penalty that may be issued against Internet service providers or single users who produce too much spam or fail to adhere to Usenet standards. It is named after the death penalty, as it causes the banned user or provider to be unable to use Usenet, essentially "killing" their service. Messages that fall under the jurisdiction of a Usenet Death Penalty will be cancelled. Cancelled messages are deleted from Usenet servers and not allowed to propagate. This causes users on the affected ISP to be unable to post to Usenet, and it puts pressure on the ISP to change their policies. Notable cases include actions taken against UUNET, CompuServe, and Excite@Home.
Google Groups is a service from Google that provides discussion groups for people sharing common interests. The Groups service also provides a gateway to Usenet newsgroups via a shared user interface.
yEnc is a binary-to-text encoding scheme for transferring binary files in messages on Usenet or via e-mail. It reduces the overhead over previous US-ASCII-based encoding methods by using an 8-bit encoding method. yEnc's overhead is often as little as 1–2%, compared to 33–40% overhead for 6-bit encoding methods like uuencode and Base64. yEnc was initially developed by Jürgen Helbing, and its first release was early 2001. By 2003 yEnc became the de facto standard encoding system for binary files on Usenet. The name yEncode is a wordplay on "Why encode?", since the idea is to only encode characters if it is absolutely required to adhere to the message format standard.
Forté Agent is an email and Usenet news client used on the Windows operating system. Agent was conceived, designed and developed by Mark Sidell and the team at Forté Internet Software in 1994 to address the need for an online/offline newsreader which capitalized on the emerging Windows GUI framework. By 1995, Agent had expanded to become a full-featured email client and remains a widely used application for integrating news and email communication on Windows. Agent supports POP email but not IMAP.
Crossposting is the act of posting the same message to multiple information channels; forums, mailing lists, or newsgroups. This is distinct from multiposting, which is the posting of separate identical messages, individually, to each channel,. Enforcement actions against crossposting individuals vary from simple admonishments up to total lifetime bans. In some cases, on email lists and forums, an individual is put under a Stealth Ban where their posts are distributed back to them as if they were being distributed normally, but the rest of the subscribers are not sent the messages. This is easily detected if the Stealthed individual has two different, and totally non-associated identities in the channel, such that the non-stealthed identity will see a different set of messages, lacking the posts of the stealthed individual, in their view of the channel.
tin is an open-source text-based and threaded news client, used to read and post messages on the Usenet global communications network.
A newsreader is an application program that reads articles on Usenet distributed throughout newsgroups. Newsreaders act as clients which connect to a news server, via the Network News Transfer Protocol (NNTP), to download articles and post new articles. In addition to text-based articles, Usenet is also used to distribute binary files, generally in dedicated "binaries" newsgroups.
Control messages are a special kind of Usenet post that are used to control news servers. They differ from ordinary posts by a header field named Control. The body of the field contains control name and arguments.
Usenet Explorer is a news client for the Microsoft Windows operating system. It is designed to handle binary and text Usenet posts, and is capable of handling newsgroups as large as hundred million headers. A Usenet indexing service with 3000 day retention and boolean wildmat as search pattern language is integrated into the program.
Usenet is a worldwide distributed discussion system available on computers. It was developed from the general-purpose Unix-to-Unix Copy (UUCP) dial-up network architecture. Tom Truscott and Jim Ellis conceived the idea in 1979, and it was established in 1980. Users read and post messages to one or more topic categories, known as newsgroups. Usenet resembles a bulletin board system (BBS) in many respects and is the precursor to the Internet forums that have become widely used. Discussions are threaded, as with web forums and BBSs, though posts are stored on the server sequentially.
Easynews, Inc is a HW Media Usenet/newsgroup reseller. Founded in 1994, Easynews service is available to individual users through a subscription model and as an outsourced service to internet service providers. Easynews offers Usenet access both through traditional Network News Transfer Protocol (NNTP) servers as well as a web interface using a standard web browser.
Spotnet is a protocol on top of Usenet, providing a decentralized alternative to usenet indexing websites, and the NZB format in general. Spotnet allows users to create and browse private 'newsservers', or decentralized repositories of files and information. Members share spots with one another, similar to the seeding process in torrent sharing. Spotnet experienced tremendous growth since the closure of FTD in 2011.