This article relies too much on references to primary sources. (October 2015) (Learn how and when to remove this template message)
|Type of business||501(c)(3) nonprofit|
Type of site
|Founded||May 12, 1996[notes 1]|
San Francisco, California, U.S.
|Services||Archive-It, Open Library, Wayback Machine (since 2001), Netlabels, NASA Images, Prelinger Archives|
|Revenue||$17.5 million (2016)|
|Alexa rank||253 (February 2019[update])|
The Internet Archive is a San Francisco–based nonprofit digital library with the stated mission of "universal access to all knowledge."[notes 2][notes 3] It provides free public access to collections of digitized materials, including websites, software applications/games, music, movies/videos, moving images, and millions of public-domain books. In addition to its archiving function, the Archive is an activist organization, advocating for a free and open Internet.
The Internet Archive allows the public to upload and download digital material to its data cluster, but the bulk of its data is collected automatically by its web crawlers, which work to preserve as much of the public web as possible. Its web archive, the Wayback Machine, contains hundreds of billions of web captures.[notes 4] The Archive also oversees one of the world's largest book digitization projects.
The Archive is a 501(c)(3) nonprofit operating in the United States. It has an annual budget of $10 million, derived from a variety of sources: revenue from its Web crawling services, various partnerships, grants, donations, and the Kahle-Austin Foundation.
Its headquarters are in San Francisco, California. From 1996 to 2009, headquarters were in the Presidio of San Francisco, a former U.S. military base. Since 2009, headquarters have been at 300 Funston Avenue in San Francisco, a former Christian Science Church.
At some points in time, most of its staff worked in its book-scanning centers. The Archive has data centers in three Californian cities: San Francisco, Redwood City, and Richmond. To prevent losing the data in case of e.g. a natural disaster, the Archive attempts to create copies of (parts of) the collection at more distant locations, currently including the Bibliotheca Alexandrina[notes 5] in Egypt and a facility in Amsterdam. The Archive is a member of the International Internet Preservation Consortium and was officially designated as a library by the state of California in 2007.[notes 6]
Brewster Kahle founded the archive in May 1996 at around the same time that he began the for-profit web crawling company Alexa Internet.[notes 7] In October 1996, the Internet Archive had begun to archive and preserve the World Wide Web in large quantities,[notes 8] though it saved the earliest pages in May 1996. The archived content wasn't available to the general public until 2001, when it developed the Wayback Machine.
In late 1999, the Archive expanded its collections beyond the Web archive, beginning with the Prelinger Archives. Now the Internet Archive includes texts, audio, moving images, and software. It hosts a number of other projects: the NASA Images Archive, the contract crawling service Archive-It, and the wiki-able library catalog and book information site Open Library. Soon after that, the archive began working to provide specialized services relating to the information access needs of the print-disabled; publicly accessible books were made available in a protected Digital Accessible Information System (DAISY) format.[notes 9]
According to its website:[notes 10]
Most societies place importance on preserving artifacts of their culture and heritage. Without such artifacts, civilization has no memory and no mechanism to learn from its successes and failures. Our culture now produces more and more artifacts in digital form. The Archive's mission is to help preserve those artifacts and create an Internet library for researchers, historians, and scholars.
In August 2012, the archive announced that it has added BitTorrent to its file download options for more than 1.3 million existing files, and all newly uploaded files. This method is the fastest means of downloading media from the Archive, as files are served from two Archive data centers, in addition to other torrent clients which have downloaded and continue to serve the files.[notes 11] On November 6, 2013, the Internet Archive's headquarters in San Francisco's Richmond District caught fire, destroying equipment and damaging some nearby apartments. According to the Archive, it lost a side-building housing one of 30 of its scanning centers; cameras, lights, and scanning equipment worth hundreds of thousands of dollars; and "maybe 20 boxes of books and film, some irreplaceable, most already digitized, and some replaceable". The nonprofit Archive sought donations to cover the estimated $600,000 in damage.
In November 2016, Kahle announced that the Internet Archive was building the Internet Archive of Canada, a copy of the archive to be based somewhere in Canada. The announcement received widespread coverage due to the implication that the decision to build a backup archive in a foreign country was because of the upcoming presidency of Donald Trump. Kahle was quoted as saying:
On November 9th in America, we woke up to a new administration promising radical change. It was a firm reminder that institutions like ours, built for the long-term, need to design for change. For us, it means keeping our cultural materials safe, private and perpetually accessible. It means preparing for a Web that may face greater restrictions. It means serving patrons in a world in which government surveillance is not going away; indeed it looks like it will increase. Throughout history, libraries have fought against terrible violations of privacy—where people have been rounded up simply for what they read. At the Internet Archive, we are fighting to protect our readers' privacy in the digital world.
Since 2018, the Internet Archive visual arts residency, which is organized by Amir Saber Esfahani and Andrew McClintock, helps connect artists with the archive's over 48 petabytes[notes 12] of digitized materials. Over the course of the yearlong residency, visual artists create a body of work which culminates in an exhibition. The hope is to connect digital history with the arts and create something for future generations to appreciate online or off. Previous artists in residence include Taravat Talepasand and Jenny Odell.
The Internet Archive capitalized on the popular use of the term "WABAC Machine" from a segment of The Adventures of Rocky and Bullwinkle cartoon (specifically Peabody's Improbable History), and uses the name "Wayback Machine" for its service that allows archives of the World Wide Web to be searched and accessed. This service allows users to view some of the archived web pages. The Wayback Machine was created as a joint effort between Alexa Internet and the Internet Archive when a three-dimensional index was built to allow for the browsing of archived web content.[notes 13] Millions of web sites and their associated data (images, source code, documents, etc.) are saved in a database. The service can be used to see what previous versions of web sites used to look like, to grab original source code from web sites that may no longer be directly available, or to visit web sites that no longer even exist. Not all web sites are available because many web site owners choose to exclude their sites. As with all sites based on data from web crawlers, the Internet Archive misses large areas of the web for a variety of other reasons. A 2004 paper found international biases in the coverage, but deemed them "not intentional".
The use of the term "Wayback Machine" in the context of the Internet Archive has become common in popular culture; e.g., in the television show Law and Order: Criminal Intent ("Legacy", first run August 3, 2008), a computer tech uses the "Wayback Machine" to find an archive of a student's Facebook-style web site.
Snapshots used to take at least 6–18 months to be added, but sites eventually were able to be added in real time by request. A "Save Page Now" archiving feature was made available in October 2013, accessible on the lower right of the Wayback Machine's main page.[notes 14] Once a target URL is entered and saved, the web page will become part of the Wayback Machine. Through the Internet address web.archive.org, users can upload to the Wayback Machine a large variety of contents, including .pdf and data compression file formats. The Wayback Machine creates a permanent local URL of the upload content, that is accessible in the web, even if not listed while searching in the http://archive.org official website.
It October 2016, it was announced that the way web pages are counted would be changed, resulting in a decrease of the archived pages counts shown.
|Year||2005||2006||2007||2008||2009||2010||2011||2012||2013||2014||2015||2016||Oct. 2016 (see above)||2017||2018|
|Number of all archived pages
|40[notes 15]||85[notes 16]||85[notes 17]||85[notes 18]||150[notes 19]||150[notes 20]||150[notes 21]||150[notes 22]||373[notes 23]||430||479[notes 24]||510[notes 25]||273||286[notes 26]||344[notes 27]|
Created in early 2006, Archive-It is a web archiving subscription service that allows institutions and individuals to build and preserve collections of digital content and create digital archives. Archive-It allows the user to customize their capture or exclusion of web content they want to preserve for cultural heritage reasons. Through a web application, Archive-It partners can harvest, catalog, manage, browse, search, and view their archived collections.
In terms of accessibility, the archived web sites are full text searchable within seven days of capture. Content collected through Archive-It is captured and stored as a WARC file. A primary and back-up copy is stored at the Internet Archive data centers. A copy of the WARC file can be given to subscribing partner institutions for geo-redundant preservation and storage purposes to their best practice standards. Periodically, the data captured through Archive-It is indexed into the Internet Archive's general archive.
As of March 2014[update], Archive-It had more than 275 partner institutions in 46 U.S. states and 16 countries that have captured more than 7.4 billion URLs for more than 2,444 public collections. Archive-It partners are universities and college libraries, state archives, federal institutions, museums, law libraries, and cultural organizations, including the Electronic Literature Organization, North Carolina State Archives and Library, Stanford University, Columbia University, American University in Cairo, Georgetown Law Library, and many others.
The Internet Archive Text Archive collection includes digitized books and special collections from various libraries and cultural heritage institutions from around the world. The Internet Archive operates 33 scanning centers in five countries, digitizing about 1,000 books a day for a total of more than 2 million books, financially supported by libraries and foundations.[notes 28] As of July 2013[update], the collection included 4.4 million books with more than 15 million downloads per month. As of November 2008[update], when there were approximately 1 million texts, the entire collection was greater than 0.5 petabytes, which includes raw camera images, cropped and skewed images, PDFs, and raw OCR data. Between about 2006 and 2008, Microsoft had a special relationship with Internet Archive texts through its Live Search Books project, scanning more than 300,000 books that were contributed to the collection, as well as financial support and scanning equipment. On May 23, 2008, Microsoft announced it would be ending the Live Book Search project and no longer scanning books. Microsoft made its scanned books available without contractual restriction and donated its scanning equipment to its former partners.
Around October 2007, Archive users began uploading public domain books from Google Book Search.[notes 29] As of November 2013[update], there were more than 900,000 Google-digitized books in the Archive's collection;[notes 30] the books are identical to the copies found on Google, except without the Google watermarks, and are available for unrestricted use and download. Brewster Kahle revealed in 2013 that this archival effort was coordinated by Aaron Swartz, who with a "bunch of friends" downloaded the public domain books from Google slow enough and from enough computers to stay within Google's restrictions. They did this to ensure public access to the public domain. The Archive ensured the items were attributed and linked back to Google, which never complained, while libraries "grumbled". According to Kahle, this is an example of Swartz's "genius" to work on what could give the most to the public good for millions of people. Besides books, the Archive offers free and anonymous public access to more than four million court opinions, legal briefs, or exhibits uploaded from the United States Federal Courts' PACER electronic document system via the RECAP web browser plugin. These documents had been kept behind a federal court paywall. On the Archive, they had been accessed by more than six million people by 2013.
|Number of all texts
(June 2, 2016)
|Number of texts
(November 27, 2015)
|6,553,945[notes 32]||358,721[notes 33]||344,810[notes 34]||134,170[notes 35]||84,147[notes 36]||66,786[notes 37]||30,237[notes 38]||25,938[notes 39]||22,731[notes 40]||14,978[notes 41]||14,795[notes 42]|
|Number of texts
(November 27, 2015)
|39,842[notes 43]||51,151[notes 44]||79,476[notes 45]||105,021[notes 46]||127,649[notes 47]||180,950[notes 48]||210,574[notes 49]||214,505[notes 50]||285,984[notes 51]||370,726[notes 52]|
|Number of texts
(November 27, 2015)
|504,000[notes 53]||455,539[notes 54]||185,876[notes 55]||70,190[notes 56]||85,062[notes 57]||81,192[notes 58]||125,977[notes 59]||206,870[notes 60]||181,129[notes 61]||272,848[notes 62]||579,905[notes 63]||855,253[notes 64]|
The Open Library is another project of the Internet Archive. The wiki seeks to include a web page for every book ever published: it holds 25 million catalog records of ions. It also seeks to be a web-accessible public library: it contains the full texts of approximately 1,600,000 public domain books (out of the more than five million from the main texts collection), as well as in-print and in-copyright books, which are fully readable, downloadable and full-text searchable; it offers a two-week loan of e-books in its Books to Borrow lending program for over 647,784 books not in the public domain, in partnership with over 1,000 library partners from 6 countries after a free registration on the web site. Open Library is a free and open-source software project, with its source code freely available on GitHub.
As of December 2018, over 50 sponsors helped the Internet Archive provide over 5 million scanned books (text items). Of these, over 2 million were scanned by Internet Archive itself, funded either by itself or by MSN, the University of Toronto or the Internet Archive's founder's Kahle/Austin Foundation.
The collections for scanning centers often include also digitisations sponsored by their partners, for instance the University of Toronto performed scans supported by other Canadian libraries.
|Sponsor||Main collection||Number of texts sponsored|
|University of Toronto||||176,888|
|U.S. Department of Agriculture, National Agricultural Library||150,984|
|University of Alberta Libraries||||100,511|
|China-America Digital Academic Library (CADAL)||||91,953|
|The Library of Congress||||79,132|
|University of Illinois Urbana-Champaign||||72,269|
|Princeton Theological Seminary Library||66,442|
|Boston Library Consortium Member Libraries||59,562|
|Jisc and Wellcome Library||55,878|
|LYRASIS Members and Sloan Foundation||||54,930|
|Boston Public Library||54,067|
|Nazi War Crimes and Japanese Imperial Government Records Interagency Working Group||51,884|
|Getty Research Institute||||46,571|
|Greek Open Technologies Alliance through Google Summer of Code||45,371|
|University of Ottawa||44,808|
|Naval Postgraduate School, Dudley Knox Library||37,727|
|University of Victoria Libraries||37,650|
|The Newberry Library||37,616|
|Brigham Young University||33,784|
|Lyrasis Members and Sloan Foundation||32,676|
|Columbia University Libraries||31,639|
|University of North Carolina at Chapel Hill||29,298|
|Montana State Library||25,372|
|Allen County Public Library Genealogy Center||||24,829|
|University of Illinois Urbana-Champaign Alternates||22,726|
|Institute of Botany, CAS and Internet Archive||21,468|
|University of Florida, George A. Smathers Libraries with support from LYRASIS and the Sloan Foundation||20,827|
|Environmental Data Resources, Inc.||20,259|
|Eric P. Newman Numismatic Education Society||18,781|
|NIST Research Library||18,739|
|Open Knowledge Commons, U.S. National Library of Medicine||18,091|
|Biodiversity Heritage Library||||17,979|
|Ontario Council of University Libraries and Member Libraries||17,880|
|Corporation of the Presiding Bishop, The Church of Jesus Christ of Latter-day Saints||16,880|
|Leo Baeck Institute Archives||16,769|
|University of Florida, George A. Smathers Libraries||15,859|
|North Carolina Digital Heritage Center||||14,355|
|California State Library Califa/LSTA Grant||14,149|
|Duke University Libraries||14,122|
|The Black Vault||13,765|
|Buddhist Digital Resource Center||13,460|
|John Carter Brown Library||12,943|
|Harvard University, Museum of Comparative Zoology, Ernst Mayr Library||||10,196|
|AFS Intercultural Programs||10,114|
In addition to web archives, the Internet Archive maintains extensive collections of digital media that are attested by the uploader to be in the public domain in the United States or licensed under a license that allows redistribution, such as Creative Commons licenses. Media are organized into collections by media type (moving images, audio, text, etc.), and into sub-collections by various criteria. Each of the main collections includes a "Community" sub-collection (formerly named "Open Source") where general contributions by the public are stored.
The Audio Archive includes music, audiobooks, news broadcasts, old time radio shows, and a wide variety of other audio files. There are more than 200,000 free digital recordings in the collection. The subcollections include audio books and poetry, podcasts, non-English audio, and many others.[notes 65] The sound collections are curated by B. George, director of the ARChive of Contemporary Music.
The Live Music Archive sub-collection includes more than 170,000 concert recordings from independent musicians, as well as more established artists and musical ensembles with permissive rules about recording their concerts, such as the Grateful Dead, and more recently, The Smashing Pumpkins. Also, Jordan Zevon has allowed the Internet Archive to host a definitive collection of his father Warren Zevon's concert recordings. The Zevon collection ranges from 1976–2001 and contains 126 concerts including 1,137 songs.
The Great 78 Project aims to digitize 250,000 78 rpm singles (500,000 songs) from the period between 1880 and 1960, donated by various collectors and institutions. It has been developed in collaboration with the Archive of Contemporary Music and George Blood Audio, responsible for the audio digitization.
This collection contains more than 880,000 items.[notes 67] Cover Art Archive, Metropolitan Museum of Art - Gallery Images, NASA Images, Occupy Wall Street Flickr Archive, and USGS Maps and are some sub-collections of Image collection.
The Cover Art Archive is a joint project between the Internet Archive and MusicBrainz, whose goal is to make cover art images on the Internet. This collection contains more than 330,000 items.[notes 68]
The NASA Images archive was created through a Space Act Agreement between the Internet Archive and NASA to bring public access to NASA's image, video, and audio collections in a single, searchable resource. The IA NASA Images team worked closely with all of the NASA centers to keep adding to the ever-growing collection. The nasaimages.org site launched in July 2008 and had more than 100,000 items online at the end of its hosting in 2012.
This collection contains creative commons licensed photographs from Flickr related to the Occupy Wall Street movement. This collection contains more than 15,000 items.[notes 70]
One of the sub-collections of the Internet Archive's Video Archive is the Machinima Archive. This small section hosts many Machinima videos. Machinima is a digital artform in which computer games, game engines, or software engines are used in a sandbox-like mode to create motion pictures, recreate plays, or even publish presentations or keynotes. The archive collects a range of Machinima films from internet publishers such as Rooster Teeth and Machinima.com as well as independent producers. The sub-collection is a collaborative effort among the Internet Archive, the How They Got Game research project at Stanford University, the Academy of Machinima Arts and Sciences, and Machinima.com.[notes 72]
This collection contains approximately 160,000 items from a variety of libraries including the University of Chicago Libraries, the University of Illinois at Urbana-Champaign, the University of Alberta, Allen County Public Library, and the National Technical Information Service.[notes 74][notes 75]
The Internet Archive holds a collection of approximately 3,863 feature films.[notes 76] Additionally, the Internet Archive's Moving Image collection includes: newsreels, classic cartoons, pro- and anti-war propaganda, The Video Cellar Collection, Skip Elsheimer's "A.V. Geeks" collection, early television, and ephemeral material from Prelinger Archives, such as advertising, educational, and industrial films, as well as amateur and home movie collections.
Subcategories of this collection include:
The Archive has a collection of freely distributable music that is streamed and available for download via its Netlabels service. The music in this collection generally has Creative Commons-license catalogs of virtual record labels.[notes 79]
Open Educational Resources is a digital collection at archive.org. This collection contains hundreds of free courses, video lectures, and supplemental materials from universities in the United States and China. The contributors of this collection are ArsDigita University, Hewlett Foundation, MIT, Monterey Institute, and Naropa University.[notes 80]
In September 2012, the Internet Archive launched the TV News Search & Borrow service for searching U.S. national news programs.[notes 81] The service is built on closed captioning transcripts and allows users to search and stream 30-second video clips. Upon launch, the service contained "350,000 news programs collected over 3 years from national U.S. networks and stations in San Francisco and Washington D.C." According to Kahle, the service was inspired by the Vanderbilt Television News Archive, a similar library of televised network news programs. In contrast to Vanderbilt, which limits access to streaming video to individuals associated with subscribing colleges and universities, the TV News Search & Borrow allows open access to its streaming video clips. In 2013, the Archive received an additional donation of "approximately 40,000 well-organized tapes" from the estate of a Philadelphia woman, Marion Stokes. Stokes "had recorded more than 35 years of TV news in Philadelphia and Boston with her VHS and Betamax machines."
Voicing a strong reaction to the idea of books simply being thrown away, and inspired by the Svalbard Global Seed Vault, Kahle now envisions collecting one copy of every book ever published. "We're not going to get there, but that's our goal", he said. Alongside the books, Kahle plans to store the Internet Archive's old servers, which were replaced in 2010.
The Internet Archive has "the largest collection of historical software online in the world", spanning 50 years of computer history in terabytes of computer magazines and journals, books, shareware discs, FTP web sites, video games, etc. The Internet Archive has created an archive of what it describes as "vintage software", as a way to preserve them.[notes 82] The project advocated for an exemption from the United States Digital Millennium Copyright Act to permit them to bypass copy protection, which was approved in 2003 for a period of three years.[notes 83] The Archive does not offer the software for download, as the exemption is solely "for the purpose of preservation or archival reproduction of published digital works by a library or archive." The exemption was renewed in 2006, and in 2009 was indefinitely extended pending further rulemakings. The Library reiterated the exemption as a "Final Rule" with no expiration date in 2010. In 2013, the Internet Archive began to provide abandonware video games browser-playable via MESS, for instance the Atari 2600 game E.T. the Extra-Terrestrial. Since December 23, 2014, the Internet Archive presents, via a browser-based DOSBox emulation, thousands of DOS/PC games[notes 84] for "scholarship and research purposes only".[notes 85]
In November 2005, free downloads of Grateful Dead concerts were removed from the site. John Perry Barlow identified Bob Weir, Mickey Hart, and Bill Kreutzmann as the instigators of the change, according to an article in The New York Times. Phil Lesh commented on the change in a November 30, 2005, posting to his personal web site:
It was brought to my attention that all of the Grateful Dead shows were taken down from Archive.org right before Thanksgiving. I was not part of this decision making process and was not notified that the shows were to be pulled. I do feel that the music is the Grateful Dead's legacy and I hope that one way or another all of it is available for those who want it.
A November 30 forum post from Brewster Kahle summarized what appeared to be the compromise reached among the band members. Audience recordings could be downloaded or streamed, but soundboard recordings were to be available for streaming only. Concerts have since been re-added.[notes 87]
On November 28, 2016, it was revealed that a second FBI national security letter had been successfully challenged that had been asking for logs on another undisclosed user.
On August 17, 2011, the Israeli-American Middle East Media Research Institute published "Al-Qaeda, Jihadis Infest the San Francisco, California-Based 'Internet Archive' Library", which detailed how members can post anonymously and enjoy free uncensored hosting.
In a story at his Web site, entitled "What the heck is going on at Internet Archive?", author Steven Saylor noted: "Sometime in 2012, the entire run of Omni magazine was uploaded (and made available for download) at Internet Archive ... Since those old issues must contain hundreds of works still under copyright by numerous contributors, how is this legal?" At least one contributor to the magazine, author Steve Perry, has publicly complained that he never gave permission for his work to be uploaded ("they didn't say a word in my direction"), and it has been noted that all issues containing the work of Harlan Ellison apparently have been taken down. Glenn Fleishman, investigating the question "Who Owns Omni?", writes that "Almost all of the authors, photographers, and artists whose work appeared in the magazine had signed contracts that granted only short-term rights. ... [No one] could simply reprint or post the content from older issues."
The Internet Archive blacked out its web site for 12 hours on January 18, 2012, in protest of the Stop Online Piracy Act and the PROTECT IP Act bills, two pieces of legislation in the United States Congress that they claimed would "negatively affect the ecosystem of web publishing that led to the emergence of the Internet Archive". This occurred in conjunction with the English Wikipedia blackout, as well as numerous other protests across the Internet.
The Internet Archive is a member of the Open Book Alliance, which has been among the most outspoken critics of the Google Book Settlement. The Archive advocates an alternative digital library project.
In February 2016, Internet Archive had begun archiving digital copies of Nintendo Power, Nintendo's official magazine for their games and products, which ran from 1988 to 2012. The first 140 issues had been collected, before Nintendo had the archive removed on August 8, 2016. In response to the takedown, Nintendo told gaming website Polygon, "[Nintendo] must protect our own characters, trademarks and other content ... The unapproved use of Nintendo's intellectual property can weaken our ability to protect and preserve it, or to possibly use it for new projects". The collection was re-added in 2017.
In August 2017, the Government of India blocked the Internet Archive along with other file-sharing websites, citing piracy concerns after copies of two Bollywood films were allegedly shared via the service.. The block was lifted subsequently.
The Great Room of the Internet Archive features a collection of more than 100 ceramic figures representing employees of the Internet Archive. This collection, inspired by the statues of the Xian warriors in China, was commissioned by Brewster Kahle, sculpted by Nuala Creed, and is ongoing.
The Internet Archive has a ginormous collection of free, downloadable music in their NetLabels category ...
Brewster Kahle, the man behind a project to file every webpage, now wants to gather one copy of every published book
Computer programs and video games distributed in formats that have become obsolete and that require the original media or hardware as a condition of access, when circumvention is accomplished for the purpose of preservation or archival reproduction of published digital works by a library or archive. A format shall be considered obsolete if the machine or system necessary to render perceptible a work stored in that format is no longer manufactured or is no longer reasonably available in the commercial marketplace.
|Wikimedia Commons has media related to Internet Archive.|
Access to the Archive's Collections is provided at no cost to you and is granted for scholarship and research purposes only.