What is Usenet?

Long-form reference · 4,174 words · about an 18-minute read

Duke, Chapel Hill and a set of shell scripts

Usenet was conceived in 1979 by Tom Truscott and Jim Ellis, graduate students at Duke University, as a replacement for the local announcement program on their department's machine. It was established the following year over a dial-up link to the nearby University of North Carolina at Chapel Hill, carried by Bourne shell scripts that Steve Bellovin had written. The first public release replaced those scripts with compiled software written by Steve Daniel and Truscott — the program remembered as A News, designed on the assumption that the whole network would carry one or two articles a day.

There was no funding and no owner. Usenet rode on UUCP, the Unix-to-Unix Copy program that already shipped with the operating system and moved files between machines over ordinary telephone lines; the result was described at the time as a poor man's ARPANET. Departments with no prospect of an ARPANET connection could exchange news by dialling one another at night, when charges were lowest. That habit is still legible in the design: articles are pushed by the sender rather than fetched by the reader, and a site that connects twice a day is a legitimate participant.

The name contracts users' network, and carried a hope that the USENIX association would take an active part in running it. The first group was NET.general, soon lower-cased to net.general. In 1980 the system reached the ARPANET through the University of California, Berkeley; Mary Ann Horton, the graduate student who set up the link, began feeding ARPANET mailing lists into Usenet under the prefix fa, for from ARPANET. Fifty sites joined in the first year; more than 500 hosts carried news by 1983 and 940 by 1984.

The software was rewritten repeatedly as traffic outgrew it. A News gave way to B News, produced by Matt Glickman and Mary Ann Horton at Berkeley, which coped with the roughly fifty articles a day arriving by late 1983 and adopted a message format compatible with internet mail. C News, by Geoff Collyer and Henry Spencer at the University of Toronto, matched its features and ran far faster. In the early 1990s Rich Salz wrote InterNetNews, or INN, which assumed a continuous stream of messages rather than the batched exchanges UUCP implied; it is still developed and still widely run.

How an article actually moves

A Usenet article is a plain text message shaped much like an e-mail: header lines, a blank line, then the body. The current specification, RFC 5536, Netnews Article Format of November 2009, requires exactly one each of six header fields — Date, From, Message-ID, Newsgroups, Path and Subject. The format has been standardised continuously since RFC 850, superseded by RFC 1036 in 1987 and in turn by RFC 5536 and its companion RFC 5537.

The Message-ID is the load-bearing part: a globally unique identifier written in the shape of an e-mail address. The standard is emphatic that two articles bearing the same identifier are treated as the same article, however their headers or bodies differ. That one rule is what makes the distribution scheme safe, because a server which has already seen an identifier declines the copy the next peer offers.

Threading uses the same identifiers. A reply carries, in a References header, the identifiers of its parent and of that parent's own ancestors, oldest first, and a newsreader builds a conversation by following that chain rather than by matching subject lines. A thread therefore survives a change of subject halfway through and reassembles correctly when articles arrive out of order or never arrive at all.

The Path header records the route an article has taken since injection: every agent that handles it prepends its own identity, and the standard tells an agent not to relay an article to a peer whose name already appears there. That check saves the offer; the identifier history behind it is what finally stops duplicates.

Distribution is flood-fill. Each server offers everything it has accepted to each of its peers, and each peer takes whatever it does not hold, until every participating server has a copy or has refused one. There is no central server, no master copy and no register of who is taking part. The flood terminates because relaying software declines articles whose date falls outside a cutoff interval — the standard recommends no less than seven days for Usenet — and because each server expires what it stores.

Transport moved onto the internet in March 1986, when Brian Kantor of the University of California, San Diego and Phil Lapsley of Berkeley wrote RFC 977, the Network News Transfer Protocol, with contributions from Stan Barber of the Baylor College of Medicine and Erik Fair of Apple. NNTP resembles the mail protocol SMTP but is shaped for newsgroup articles, and it separated reader from server: rather than logging in to the machine holding the spool, a user could run a newsreader on a personal computer. It uses TCP port 119, and port 563 when wrapped in TLS.

What a newsgroup is, mechanically

A newsgroup is not a place, an account or a list of members. It is a name written in the Newsgroups header of an article. A group exists on a given server because that server's administrator has configured it to accept and store articles bearing that name, and it exists on the network to the extent that other administrators have made the same decision. Subscribing is a private note kept by the reader's own software about which groups to display and which articles have been read.

No server is obliged to carry anything — it may refuse a group because it consumes too much disk, because the traffic trips a spam filter, or for no stated reason — so a group can be well propagated in one country and absent in another, complete on one server and full of holes on the next. And the copies are genuinely separate: there is no original.

Administrative actions travel as ordinary articles carrying a Control header: newgroup asks servers to create a group, rmgroup to remove one, cancel to withdraw a specified article. The standards note plainly that there is at present no standardised means of authenticating the sender of a control message. Forged cancels were abused early and often, so most servers now ignore them outright and treat creation and removal as advisory. Deleting something from Usenet is therefore not one operation but a request made, separately and voluntarily, of every administrator holding a copy.

A minority of groups are moderated: submissions are mailed to a moderator, who injects the approved ones bearing an Approved header, without which conforming servers will not propagate them. Moderated groups originally had to live in a hierarchy of their own called mod.*; since the reorganisation described below they can sit anywhere, usually marked by a .moderated suffix.

One article may name several groups at once. Cross-posting stores a single copy that appears in each named group, and netiquette prefers it to separate copies, because a reader who has seen the article in one group is not shown it again in the next.

The Great Renaming and the founding of alt.*

For its first six years Usenet's names were, by common admission, haphazard. Groups fell into three hierarchies: net.* for unmoderated discussion, mod.* for moderated discussion and fa.* for the mailing lists gatewayed from ARPANET. The scheme was a product of software limits as much as of taste: a moderated group had to sit under the mod. prefix and be contributed to by e-mail, and the flat storage layout required the first fourteen characters of every group name to be unique.

Those limits fell away during 1986 with B News version 2.11, released by Rick Adams, which dropped the prefix requirement, allowed ordinary newsreaders to post to moderated groups and abolished the flat spool. Reorganisation became practical, and in 1987 it happened; Adams is generally considered the initiator of the Great Renaming. The stated motive was the difficulty of maintaining a list of every group in existence. An alternative account, widely repeated since, is that European networks refused to pay for high-volume, low-content groups on subjects such as religion and racism and wanted them separable.

The renaming produced seven hierarchies — comp.*, misc.*, news.*, rec.*, sci.*, soc.* and talk.* — open to anyone but subject to shared rules on naming and distribution, and known as the Big Seven. In April 1995 an eighth, humanities.*, was added, giving the grouping its present name of the Big Eight.

What made a rename of that scale enforceable was the backbone cabal: an informal association of administrators of the large sites whose machines, unlike most, stayed reachable around the clock and so carried the bulk of long-distance propagation. Mary Ann Horton recruited its membership and designed the original backbone topology in 1983; Gene Spafford created the mailing list on which its decisions were taken, and Brian Reid, Richard Sexton, Chuq von Rospach and Rick Adams were among its members. Because a group the backbone declined to carry did not reach most of the network, its preferences had the force of policy in a system that formally had none. It broke up in late 1988, having denied throughout that it existed.

The alt.* hierarchy was created in reaction to exactly that. Brian Reid proposed it as an alternative to talk.*, carried outside the backbone's influence so that the backbone's willingness to carry a group would no longer decide whether it could exist. He created it with John Gilmore, and the first group in it was Reid's own alt.gourmand.

A Big Eight group is created by a written proposal called a Request for Discussion, which must name the group and state whether it is moderated and by whom; discussion follows, and the Big-8 Management Board votes on approval. In alt.* there is no board and no vote: anyone capable of issuing a newgroup message can create a group, and whether it survives depends on whether individual administrators choose to carry it. Most proposals are still discussed first in alt.config, because groups that skip the step propagate badly.

Other hierarchies are distributed alongside these and are Usenet in every practical sense: national and regional ones such as de.*, fr.* and japan.*, and corporate ones run for product support. This directory's hierarchy pages — comp, rec, sci, soc, talk, news and alt — follow the same divisions, and the full group list shows what the gateway carried.

What the medium invented

The FAQ is the clearest case, and the usual account needs one correction. The question-and-answer digest was worked out in 1982 on the ARPANET's SPACE mailing list, where regulars were expected to read the archive before asking and mostly did not, and Eugene Miya of NASA coined the acronym for that list some time between 1982 and 1985. What Usenet did was industrialise the form: FAQs were first posted at monthly intervals, Jef Poskanzer is credited with the first weekly one, posted to net.graphics and comp.graphics, and the moderated *.answers groups were created to collect FAQs cross-posted from the groups beneath them.

Netiquette, a portmanteau of network and etiquette, is the name for the conventions that grew up around all this: keep signatures short, do not multipost or cross-post carelessly, stay on topic, do not hijack another person's thread, quote only enough of the message you are answering to make the reply intelligible, and do not type in capitals, which reads as shouting. Some of it was courtesy; much of it was about not wasting disk, bandwidth and reading time, which in this network are shared and finite.

The killfile is Usenet's contribution to moderation, and it inverts the usual model. Rather than an administrator removing a poster for everyone, each reader keeps a private file of patterns — authors, subjects, other header fields — and matching articles are never displayed to that reader. It was first implemented in rn, Larry Wall's newsreader; later readers elaborated it into scoring systems where overlapping rules give an article a numeric weight and anything below a threshold disappears. Filtering is the reader's business, not the publisher's.

The signature block outlived the medium that standardised it. On Usenet it was a few lines of monospaced text holding a name, an address, sometimes a quotation generated by the fortune program and sometimes a piece of ASCII art. The standards separate it from the body by a line of exactly two hyphens and a space — the sig dashes — so that software can find and strip it automatically.

Spoiler space applies the same instinct to plot endings. The term reached the internet in 1979 and was common on Usenet by the late 1980s, and the local solution was typographic: precede the revelation with enough blank lines to push it off the bottom of a twenty-five-line terminal. The alternative was ROT13, a trivial cipher rotating each letter thirteen places through the alphabet; almost every newsreader had a key for it.

Godwin's law belongs to the same period. Mike Godwin promulgated it in 1990, of Usenet discussions specifically, and has said he introduced it as an experiment in memetics: as an online discussion grows longer, the probability of a comparison involving Nazis or Hitler approaches one.

Words that got out

Several terms in ordinary English were made or made popular here, and the folk etymologies are numerous.

Spam derives from the 1970 Monty Python sketch in which a cafe menu consists almost entirely of the tinned meat and a party of Vikings drowns the dialogue by singing its name. In the 1980s it was applied on bulletin boards and in multi-user dungeons to users who typed spam repeatedly to scroll everyone else's text off the screen. It arrived on Usenet meaning excessive multiple posting — the repeated posting of the same message — and only later narrowed to unsolicited commercial mail: the volumetric rather than the commercial character of the offence is what it originally described.

Flame is a hostile posting; Usenet writers marked the hostile passage off with the commands FLAME ON and FLAME OFF. The standard explanations for why text-only discussion produces so many are anonymity and the thin transfer of social cues, which reduce the fear of social sanction, and the disinhibition that follows from both. A flame war can outlast the disagreement that began it and split a group permanently.

Troll comes from the fishing sense: to troll is to drag a baited line slowly behind a moving boat. The internet usage grew out of the phrase trolling for newbies in alt.folklore.urban in the early 1990s, where regulars would post an overdone question to see who took it seriously. The Oxford English Dictionary's earliest attestation for the internet sense is from 1992.

Lurker is far older than any network: the verb, meaning to lie concealed, is traced to the fourteenth century. It was applied to online communities in the mid-1980s on bulletin board systems, where a user occupying the single telephone line without contributing was resented as a free-rider. The kinder reading is the one Usenet arrived at first: lurking is how a newcomer learns a group's conventions before speaking.

Sockpuppet, for a second identity operated by someone already present in a discussion, and FAQ itself belong to the same list of terms Usenet gave to or popularised for the wider networked world.

Turning points, 1988 to 1994

On the evening of 2 November 1988 a self-replicating program was released from a machine at the Massachusetts Institute of Technology by Robert Tappan Morris, then a graduate student at Cornell. The often-quoted figure of six thousand infected Unix machines is a disputed estimate, derived from a guess that some sixty thousand computers were attached to the internet and that roughly a tenth were hit. The effect on connectivity is not disputed: the internet was partitioned for several days as regional networks disconnected from the NSFNet backbone and from each other. Gene Spafford created the Phage list to coordinate the response, and in the aftermath DARPA funded the establishment of the CERT Coordination Centre at Carnegie Mellon University.

Usenet was where the incident was discussed, at length and for years, but the emergency response itself ran on a mailing list. Usenet propagation is asynchronous and was then frequently batched, many sites exchanging news only once or twice a day, so an article could take a day or more to reach the far edges of the network.

The Eternal September is the name for what happened between late 1993 and early 1994. Until then the influx of newcomers had been seasonal: each September a fresh intake of university students received accounts, arrived without knowing the conventions, and was absorbed over a few weeks by the regulars. When commercial providers began offering Usenet to the general public that pattern ended, and America Online's gateway, opened in March 1994, made the arrival of newcomers permanent. The phrase the September that never ended appears in Usenet postings in January 1994 and the compressed form Eternal September in February.

Then, on 12 April 1994, the law firm of Laurence Canter and Martha Siegel advertised their green-card lottery services by posting the same message to at least 5,500 newsgroups, using a Perl script written by a programmer identified only as Jason. Individually posted copies rather than a single cross-post meant the advertisement appeared in every group separately and could not be filtered as one article. The firm's internet provider received complaints in volumes that crashed its mail servers repeatedly over the next two days, and it terminated their account. The campaign is generally taken as the first mass commercial spamming of the network.

Usenet's answer was collective. The Usenet Death Penalty is a declaration, announced in advance and only after the operator of an offending server has been contacted and given repeated chances to act, that articles originating there will be cancelled by cancelbots or not accepted at all until the problem is fixed. It was applied to UUNET on 1 August 1997 and called off on 6 August, and to CompuServe on 18 November 1997, lifted the following day. It is enforcement by agreement among administrators, which is the only kind of enforcement the network has.

Archives, binaries and the withdrawal of the ISPs

Usenet was archived from early on, in fragments and by individuals, and the wish for a searchable index of it is nearly as old. Web-based archiving on a serious scale began in March 1995, when Steve Madere founded Deja News in Austin, Texas: free public access through a browser, search across all newsgroups at once, and retention of messages indefinitely rather than for the few days a news server kept them. A poster could no longer assume a remark would scroll off every server within the week.

Google acquired the Deja archive in February 2001, having begun archiving Usenet on its own account from the second week of August 2000, and put it behind Google Groups. The collection was then pushed backwards in time. Posts from May 1981 to June 1991 were donated by the University of Western Ontario, having originally been archived by Henry Spencer — the same Henry Spencer who wrote C News — in the University of Toronto's zoology department; late 1991 to early 1995 came from Kent Landfield's NetNews CD series and from Jürgen Christoffel at GMD. The result is a record of a public conversation running from 1981 onward.

The other thing that happened to Usenet was binaries. The system was built for seven-bit ASCII text, but uuencode, a utility from the UUCP package, represents arbitrary bytes as printable characters, and once that was routine any file could travel as an article. Because articles were commonly limited to around sixty thousand characters in the late 1980s, large files were split into numbered parts the reader reassembled. The practice hardened into a toolchain of RAR archives with Parchive parity files, so that a download missing a few parts can still be reconstructed.

Binary posting was confined to alt.binaries.* precisely so administrators could refuse it, and that hierarchy became the largest on Usenet, accounting for over ninety-nine per cent of article data. Text traffic did not disappear but became statistically invisible: by 2010 all the text posted in the whole of the Big Eight averaged about 1,800 messages an hour, against a binary feed measured in terabytes a day.

That imbalance is the background to the withdrawal of the general internet providers. In 2008 Verizon, Time Warner Cable and Sprint Nextel signed an agreement with the New York Attorney General, Andrew Cuomo, undertaking to cut off access to sources of child abuse imagery; Cuomo said his office had found such material in eighty-eight of the hundred thousand groups in alt.*. He never named Usenet as such, but the providers cut parts of it: Time Warner Cable stopped offering Usenet altogether, Verizon narrowed its service to the Big Eight, Sprint dropped alt.* and AT&T dropped alt.binaries.*. The reasons the providers themselves gave ran to falling readership, cost, and the liability attached to file-sharing traffic and spam.

The rest followed within two years. AT&T ended its Usenet service on 15 July 2009 and Verizon on 30 September 2009; the United Kingdom's academic network JANET withdrew its service with effect from 31 July 2010; Microsoft began closing its public newsgroups on 1 June 2010, finishing that autumn. In May 2010 Duke University decommissioned the news server whose predecessor had started the whole thing thirty years earlier, and on 4 February 2011 the news service at the University of North Carolina at Chapel Hill was retired after thirty-two years.

What Usenet is now, and how to read it

The most recent break was Google's. Google Groups had been the way most casual readers reached Usenet for two decades, and by the 2020s it had also become a substantial source of spam entering the network, to the point that a Usenet Death Penalty was called against it in October 2023. In January 2024 the service carried a notice that, effective from 22 February 2024, it would no longer support new Usenet content: posting and subscribing would be disallowed and new content from Usenet peers would not appear. Google's NNTP server and peering arrangements were withdrawn at the same time. Viewing and searching the historical archive continues.

What remains is the network itself, which was never Google's and never anyone's. Servers still peer, articles still flood-fill, and NNTP is still NNTP. The binaries hierarchies are still heavily used, and are what most surviving commercial providers sell. The text hierarchies are quiet, unevenly carried and heavily spammed in the groups nobody tends, but a number are still genuinely active.

Reading it today takes one of three forms. A newsreader plus an account on a news server is the original arrangement; some servers carry the text hierarchies without charge, and commercial providers sell access by the month. This directory keeps a page of free news servers and a page of web-based services that will show you groups in a browser. The practical instructions are on the access page, and the conventions worth knowing before you post are set out under how to post.

Usenet was the first internet community and the venue for a striking number of the announcements that shaped what came after: it is where Tim Berners-Lee announced the launch of the World Wide Web, where Linus Torvalds announced the project that became Linux, and where Marc Andreessen announced Mosaic and the image tag that turned the web into a graphical medium. Those were not press releases sent to a list of journalists. They were messages sent to a topic-named group by someone who thought the readers would be interested — which is the mechanism this page has described.