The Internet as a National Security Studies Resource

by William M. Arkin

The Internet is an electronic embarrassment of riches and a truly revolutionary change in the way information flows in our society. Yet as a research tool, the Internet is not everything. It is not comprehensive, nor very historical, and it is by no means selective. What is online is there because someone has placed it there, not because it has been published or because it is deemed important by the academic or expert world. Good and bad information therefore reside comfortably together, and what is online is hardly a comprehensive reflection of what there is. The very culture of the Internet is also transitory, material coming and going, often undated and uncited, with an informality that detracts from usefulness for academic citation. And there is no methodical catalogue of content, even in discrete fields.

In starting any research project, the number-one task is identifying the most relevant resource likely to produce the required information. Students and other Internet enthusiasts may think that if it is not on the Internet, it isn't important. Yet although the Internet may be increasingly huge, and the best finding aid for a paper resource, it is not THE only resource. Many Internet books and magazines speak of an "old way" and a "net way" to do things and find information, but no researcher should consider the "push" of the Internet superior to the "pull" of methodical research and inquiry which seeks out data needed to answer a question, rather than just using available data. The Internet cannot replace traditional data collection as a means to understand an issue comprehensively. And in a field like national security studies, where much information remains classified, no researcher can afford to overlook documents in archives alone or those originally obtained under mandatory declassification or the Freedom of Information Act (FOIA).

That said, the Internet is already a superior source for certain types of information. From official government resources to the world of international institutions; to academic research centers, departments, professors, and libraries; non-governmental think tanks, organizations, and associations; corporations; the news media; citizens groups and individual citizens, it is clear that the Internet has become THE place to communicate and debate. Every institution and special interest has a website, and even when it comes to doing things the "old way," people can be found for interviews using online search resources; biographies and obituaries can be searched more efficiently online; and even some Freedom of Information Act requests can be made electronically. Finally, the Internet is not merely a facsimile of what is available on paper. Internet "pages" or documents are themselves increasingly serious resources.

While one cannot overemphasize the importance of discipline and a clear research agenda in using the Internet, part of its joy is discovering truly unique resources. Where else could one quickly find out the positions of the Hezbollah or the Serbian government and read their own propaganda and views first hand? How would someone find a street map of Baghdad or Belgrade any other place than online? Where does one go for an Army field manual or an Air Force regulation? Or the text of a recent speech by President Clinton, or the defense stance of the presidential candidates? Or the text of a NATO communique or UN Security Council Resolution from 1989? Or the constitution of Lithuania? What is more, all of this is available from the office or at home, and one can retain an electronic copy for later retrieval.

One may argue that the web is only relevant for the instant and contemporary, but it has also acquired surprising depth and breadth considering its youth. Historical documents, even the full text of historic books, are online from around the world, and growth in the depth of material shows no signs of slowing. Though the World Wide Web itself tends to have little historical memory (the depth of contemporary material falls off notably before 1994), websites dealing with all aspects of history, politics, regional and country studies, and science and technology have sprung up, and each of these is slowly building its own archive.

And even in areas by far favoring paper, where there has been little digitization, such as military history, the Internet is quickly becoming a vital resource. The finding aids of archival collections, such as those of the National Archives, the presidential libraries, the Library of Congress, or official military history centers are increasingly online, and the military libraries and war colleges, of which there are dozens, have each made significant contributions to online wealth. Older documents are increasingly being converted for online viewing. New and old studies are online. One can find, for instance, volumes of the World War II Strategic Bombing Survey online.

In one area in particular, the Internet has assumed a leadership role: that of accessing the world of "grey" literature. Grey literature refers to those studies, position papers, reports, proceedings, dissertations, etc., that are not books or periodicals neatly catalogued by traditional libraries. It is the output and the lifeblood of government and public policy institutions and actors. Before the web, it was material that really only circulated to the world of wonks and inside the Beltway operators. Libraries never really found a satisfactory method to catalogue, let alone collect, grey material (it was essentially only available through the system of inter-library loan). Now it is freely available and virtually circulating via the Internet.

There is so much material, and so much more is being added every day from all corners of the globe, some words of warning are in order. The search engines are businesses, and they deal with the enormity of the web by serving their main customers, who are not national security-oriented expert users. Their bias is towards the popular rather than the obscure, towards services rather than perfection in databases. And while "independent" sources of content have sprung up on the web, from general to highly specialized news services, many in the national security field, these are also businesses. They face two realities. The first is the need to serve the customer. Paying customers tend to be corporate users, and reliable and original online information tends as a result to be in the financial sector. The second is that traditional media and entertainment companies, such as NBC or Disney, have been buying up many of these players after they have failed to turn a profit independently. In fact, stratification on the web looks more and more like society all of the time. The end result? There may be quality control that comes with affiliation with traditional news media, but there is also a further bias towards the majority and most clamorous users, with expert users suffering.

The Bias of Internet Content

What is really revolutionary about the Internet is not just computing power distributed in such a way that it gives anyone access to a global grid of information resources, it is the concept of the availability of primary sources. Sure the web is a medium over which one can read The New York Times or The Washington Post (or for that matter, The Jerusalem Post), but it also provides potential access to the primary source, the transcript of the briefing or speech, the original U.S., UN, or NATO document, the original study, the original data that is the basis for the news report. When the mainstream news media emerged on the web, there were examples (the San Jose Mercury News series on the CIA-crack cocaine connection) of using the power of the web to hyperlink to the original sources behind the story. For a number of business, personnel, and tradecraft issues relating to journalism, the full potential of the depth of the Internet still remains to be realized. That, however, does not mean that the primary sources aren't available. They are. Increasingly governments themselves, from local to international, are placing their raw output online. And if governments do not post original documents, some webmaster, in many cases a mere individual or hobbyist operating a web "shrine" in an obscure corner of the Internet, will have posted the material.

Every societal event, controversy, and crisis, from the death of Princess Diana or the downing of TWA Flight 800, to wars in Sierra Leone and Yugoslavia, triggers the creation of new web resources and databases of information, from the traditional media to governments to advocacy organizations to these so-called shrines. In early 1998, as war loomed in the Persian Gulf, special websites from the U.S. military, foreign governments, the news media, think tanks, arms control and counter-proliferation organizations, human rights, peace movement, and advocacy groups all across the political spectrum, sprung up overnight. When an issue such as human rights in China and permanent most-favored-nation status is debated, all the players: the Chinese Embassy in Washington, the State Department, the Tibetan government-in-exile, human rights organizations, post special pages to showcase their positions.

The proliferation of special websites is hardly just a U.S. reflection of the world. There is no longer any region or country that lacks authentic local information on the Internet. The 1999 war in Yugoslavia was probably the first high-profile conflict in which both adversaries were completely "wired" societies. Not only then did the now-expected special websites emerge (in fact, there were so many it was difficult even to track them all), but the Internet itself became a genuine factor in the war. The Yugoslav government deftly used the web to disseminate releases and photographs relating to civilian collateral damage. Suppressed Yugoslav media used the web from outside the country to continue their journalism. There were claims of hackers, and there was genuine government interference in Internet access. Computer network attack was a part of NATO's arsenal. It was truly the first Internet war. But, characteristic of the web, much of the electronic legacy is unfortunately already disappearing into the ether because of the transitory nature of the medium.

What survives after a crisis has abated, after the big media has moved on to another issue, and after governments have shifted their focus, are not just the academic and NGO specialists of the region or country who are interested in a subject regardless of the spotlight. On the web, the "committed"—the adherents of extreme politics and conspiracy—tend to get as much attention as reputable institutions. The squeaky wheels, using websites with persistence and energy, do get the grease, at least as it relates to attention from portals and search engines which hardly have the ability to follow discrete subjects in national security, let alone any specialized subjects. If one were to take a snapshot of the most energetic national security topics on the web, for instance, the National Security Agency’s so-called Echelon surveillance network or the Gulf War Syndrome and anti-anthrax vaccination crowd, one might have a very distorted view of reality.

What's on the Web?

It is not an exaggeration to say that a little bit of everything about everything is online. With library catalogues and archival indexes connected to the web, some including highly specialized collections, the Internet really does have relevance to every research task. Even for Luddites committed to paper research, online resources speed the process of traditional library research.

In a number of areas, the Internet excels, with rich resources, some of which are merely online equivalents of information and data available elsewhere, some of which are completely original:

Ready reference: The Internet has quickly developed into one of the finest sources of ready reference material. Encyclopedias, dictionaries, maps, coordinates and locations, acronyms and abbreviations, and equipment nomenclature indexes are now all readily available online. People finders, worldwide telephone directories, and mapping services truly unleash some of the computing power of the web. Traditional libraries long ago recognized that information technology was transforming their role in the world, and librarians have been quick to try to provide some order in the chaos of the Internet, with some of the best links pages, bibliographies, and other research aids.

Data: Statistical and demographic data, much of it compiled authoritatively by U.S. government agencies and international organizations, now exists online for every country and region of the world, and on every aspect of national security affairs. Some of the data are a mere online (though far more convenient) example of what is available on paper, for example, the CIA World Fact Book; other data are completely new and configured for Internet presentation. Agencies such as the Census Bureau, Department of Energy, World Bank, and the UN, as well as private think tanks (the Stockholm International Peace Research Institute is one that comes to mind) have excelled in putting material online.

News: The real-time nature of the Internet medium and its unlimited storage capacity make it particularly suited for delivery of up-to-date news. In the space of less than six years, virtually every wire service, newspaper, magazine, or newsletter of note worldwide has found it necessary to be represented on the net. Television and radio have established a foothold as well. In fact, CNN, MSNBC, and ABC are three of the most popular destinations on the web. Like so much else about the Internet, the reach is global, and thousands of previously unavailable publications and news sources are circulating for the first time to a mass audience. Most of the major media websites are far more than merely representations of what is available on paper or on the air. Archives tend to be limited, but in the case of special coverage, which often includes wars, resources tend to be more comprehensive and historical.

Official information: Government and military commitment to use of the Internet as a means of public dissemination has made a body of national security-related documents readily available. These include speeches and testimony, official papers and studies, reports from military commands and the various war colleges, technical reports from laboratories, regulations and directives, and doctrinal and procedural manuals. Some material, such as budget and contracting data, inspector general and audit reports, or "lessons learned" are really available for the first time because of the Internet. Standard government publications and periodicals are now online, as well as the public affairs publishing output and web dissemination efforts of countless government agencies. Among more traditional and core government publications, the full text of Commerce Business Daily, the Congressional Record, the Federal Register, and the Code of Federal Regulations, are all available online in fully searchable forms.

The policy debate: From reputable think tanks and academic research centers, to special interest organizations, to private individuals, the Internet is a source of mainstream, alternative, and unconventional views and opinions. The latter are provided via direct access to the websites and discussion groups of revolutionary, peace, veterans, human rights, and hate groups, and the huge world of conspiracy theories. Much of this is merely the "brochure ware" of organizations and causes, but other material is complete and rich, including the grey literature, special mailing lists and discussion groups, and expertly maintained homepages and communities in discrete areas. Specialized newsletters and journals in every aspect of national security affairs and international relations, some only available online, make the Internet a true element of the policy debate.

Search Dilemmas and Internet "Intuition"

Unlike libraries or journals with editors and editorial boards, there is no one with overall charge of web acquisitions, and there is no "balance" in terms of what is on the web. The mix of authoritative and truly crazy does not disqualify the Internet for academic use, but it does present special challenges. Using the Internet is not like using a library. You walk into a library in Cambridge or Calcutta and there is a basic organization and a school-age-learned intuition and orientation. Now contemplate entering the Internet. For most people, the same instincts do not exist. The experience instead is the equivalent of walking into a library and finding all of the books heaped into a pile in the center of the room. As a result, to most, the vast holdings of the net are nothing more than a library of frustration. First, there is the reality that unlike bookshops or libraries, the Internet itself has not been neatly organized by information professionals. Second, the very scope of the web defies simple comprehension—one cannot keep up with the resources on the web, even in the most discrete field. And third, the web is fleeting. It is a dynamic medium, a living resource where something found yesterday may be gone tomorrow.

This January, Inktomi and the NEC Research Institute completed a new study estimating that the web was now made up of more than one billion unique pages. Halfway through the year, another study claimed two billion pages. Since April 1998, when the first comprehensive study was done by the NEC Research Institute, the Internet has either tripled in size or is seven times larger. Or maybe it has not grown that much. Or maybe it has grown even more. Regardless, no one denies that the web is growing by leaps and bounds. It is estimated that more than one million pages were being added to the Internet daily in mid-2000. No one knows really how large the Internet is. What is more, the billion estimate comes nowhere near to estimating how much material is available via the web. For instance, the traditional proprietary information vendors, such as Lexis-Nexis, Dialog, and Dow Jones News Retrieval, all have interfaces to the web. Dialog’s content alone is estimated at more than six billion pages of searchable text, three times the "size" of the Internet. Add to all this the contents of over four thousand library card catalogues connected to the web, with billions upon billions of records.

In The Internet and Strategic Studies, I said that indexing specialized academic, scientific, and military-related web content was suffering more and more as the net grew. This continues to be the case, at least as it applies to the commercial search engines. But luckily, at the same time, those search engines are getting better, and specialized search services (many in the military field) have emerged where the general tools leave off. This sub-field has come to be called the world of meta-information, information to find information. Broadly, meta-information can be categorized in three groups: search engines, directories, and primary sources. Whether the field is Africa, intelligence and espionage, or naval forces, the Internet presents dozens of meta-information sites to assist (and confound) the researcher.

So, the good news is that while much expert content is not or cannot be indexed by the major search engines (this will be explained in more detail in Chapter III), what is indexed is increasingly easier to get to if one uses the proper search methods and query syntax. The good news is also that approximately 12 to 15 percent of Internet content is located in the combined government, military, and educational domains and largely accessible via the major search engines, which have generally made a priority of indexing this material.

What is not indexed is accessible, if one also builds skills to use the Internet without dependence on the search engines. I call this Internet "intuition." Internet intuition is having that basic orientation and confidence that one has upon entering a library or bookstore, that without having to examine the spine of every volume, the right volume can be found, and the information being sought can be narrowed further. There are skills that will help to build Internet intuition, but the most important is thinking of the Internet as the world of information. It is the information itself that is the target. The Internet is merely the medium in which the information resides.