The Institute for Defense Analyses: Advancing Operations Research and Mathematics for America’s Defense and Intelligence Communities
In the decade following the Second World War, the defense and intelligence communities of the United States faced an increased demand for operations research methods and solutions. OR applicability had proven useful in areas beyond the wartime problem sets of 1941-1945. In the realm of efficient procurement, the Weapons Systems Evaluation Group (WSEG) was established in 1948 by the U.S. Department of Defense to meet this demand. This venture ultimately fell short of achieving its goals. At the time, there existed a successful counterpart of sorts in the private sector – the RAND Corporation, an independent contractor that, among other things, absorbed the R&D load for various branches of the armed forces. WSEG was never allocated the budget nor contracts that enabled RAND’s success. One identified problem for WSEG was that, as an organization internal to the Department of Defense, its hands were tied by federal bureaucracy. The powers that be realized that to get the most of an operations research group, they needed one free from the direct administration and oversight of the United States government. From this idea in April 1956 and with financial assistance from the Ford Foundation, the Institute for Defense Analyses was born.
The Institute for Defense Analyses (IDA) was established for supporting WSEG as a university-supported organization comprising civilian scientists and mathematicians. Within a few years, the institute expanded its operations to cover fields beyond its primary charter. IDA worked closely with DOD’s Advanced Research Projects Agency (ARPA) and provided research on ambitious new military technologies with “JASON”, a study group of elite academic scientists (Thomas 2015). From 1956 until 1964, the number of universities providing academic oversight over IDA grew from four to twelve. These formal academic relationships ended at the height of the Vietnam War in 1968 as anti-war demonstrations at member universities forced collegiate leaders to distance their institutions from the military-intelligence complex. At Princeton, especially, student and township protestors targeted the IDA Communications Research Division (CRD) center located on the university’s campus. In 1970, CRD staff threatened to sever from IDA and negotiate its own contract with DOD if the university failed to compensate the organization for costs incurred surrounding the demonstrations. CRD ultimately rode out its initial lease until 1975 and moved to an off-campus location (Neuwirth 2009). Though IDA lost direct university oversight, the academic culture fostered in its first fifteen years set the stage for a research organization that operated far differently from the bureaucratic government agencies it supported, hence achieving its original purpose.
The Systems and Analyses Center (SAC) in Virginia is the largest and oldest of IDA’s Federally Funded Research and Developments Centers (FFRDCs). Initially, the staff here was largely focused on assisting DOD’s acquisitions office and ARPA. IDA was not a direct and immediate replacement for WSEG but rather a supplement. The relationship between the two organizations was complementary. This is evident in papers published by IDA researchers in the 1950s and 1960s. In an article which shared an algorithm that examined the trade-off between performance versus cost versus time of obtainability, Karl Seiler gave thanks to Colonel Lawrence Bowley of WSEG for “valuable suggestions pertaining to this paper (Seiler 1966).” Other collaborative work at the time included queuing with impatient customers and ordered service (Barrer 1957). While other organizations with ties to the federal customer, like RAND, were becoming major players in civilian work, IDA was fully committed to serving DOD. In the late 1960s, then IDA president Maxwell Taylor, a retired Army General and former US Ambassador to South Vietnam, quite definitely assured his staff that this would remain the case.
Though much of IDA’s work at the time was (and remains) classified, some public articles and letters draw attention to the work done in the 1950s through 1980s. Paul J. Schweitzer of IDA, for instance, published research on optimal replacement policies in Operations Research, comparing the use of “block replacement” for items with hyperexponentially distributed lifetimes and uniformly distributed lifetimes. In this brief article, he compares individually replacing lightbulbs as they fail in a system to the periodic replacement of all lightbulbs in a system. He shows that since lightbulbs fail on an individual and unpredictable basis, block replacement is never the preferred option (Schweitzer 1967). It is not difficult to replace lightbulbs in this scenario with other groups of products pertinent to the DOD with similarly individualized life spans, such as certain weapons or electronics equipment in the field. This style of problem characterized many of the acquisition puzzles the organization was tasked with solving. In a chapter on systems analysis in defense logistics, IDA’s Grace Kelleher acknowledged for “reasons of security classification,” research on weapons systems could not be included in unclassified publications, opting to share instead “the field of defense logistics, a field that can be discussed openly and significantly involves problems akin to those faced by other governmental agencies such as the Department of Transportation and the Post Office (Kelleher 1970).” On other occasions, the defense and security applicability of publicized research was more obviously stated. In 1965, Kelleher gave a presentation at the 28th National Meeting of ORSA on a model for allocating nuclear shelters in preparation for a “potential threat by such countries as Communist China.” Kelleher’s model provided minimal cost shelter creation with the aim of limiting fatalities from a nuclear strike, considering the distances from ground zero, the distribution of persons in a populated area, and each element of a population matrix. This involved not only the number and location of shelters, but their strength against a blast considering their respective distances from the initial strike site (Kelleher 1967).
Not unlike researchers at RAND and other trailblazing institutions of the 20th Century, IDA persons were involved in the growing operations research community of the late 1950s and early 1960s. In 1957-1959, three IDA researchers served as paper referees on the editorial board of Operations Research (ORSA 1959). IDA’s Al Blumstein, served as chair of the Nominating Committee in 1969 (ORSA 1969). During his time at with the institute, Blumstein directed a task force under LBJ’s Commission on Law Enforcement and Administration of Justice, an appointment that may have been tied to his service as chair of the Washington OR Committee (WORC), a local ORSA chapter. Blumstein chose to run the task force from his office at IDA as he and his team became responsible for bringing methods “that characterize the hallmark of OR” to criminal justice. After leaving IDA in 1969, he went on to serve as president of ORSA (1978-78), TIMS (1987-1988), and INFORMS (1996). Beyond contributions to OR itself, IDA staff members also studied greater trends that affected systems analysis and other elements in their field. Kelleher edited a book on the topic for the Operations Research Society of America. In the Preface, she acknowledges IDA for “sponsoring the editing of this book under their independent research program (Kelleher 1970).” Importantly, IDA was and remains committed to hiring persons from a wide array of backgrounds, including operations research, systems analysis, economics, cost analyses, mathematics, and computer science, and actively recruited from the greater OR community (IDA 1966).
Where SAC’s sponsor and client was DOD, the Communications Research Division (CRD) served the National Security Agency and operated separately from the efforts in Virginia. The Center for Communications Research (CCR) in Princeton, the Center for Communications Research, La Jolla (CCRL or “CCR-West”), and the Center for Computing Sciences (CCS) in Maryland were all established as the cryptology/cryptanalysis divisions of IDA. In 1962, Dr. Richard Leibler became the director of CRD, a position he held for fifteen years. Leibler was inducted into the NSA Hall of Honor for his contributions to the “Venona” project (an effort to decipher theoretically unexploitable Soviet espionage methods). The Soviet system used so-called “one time pads.” NSA discovered, via statistical analysis, that some of the pads were in fact re-used, which led to the deciphering of some messages. The “Kullback-Leibler divergence” for measuring similarity between populations may contain some similarity to the classified methods he developed. A number of brilliant mathematicians worked under Leibler during his tenure, including Jim Simmons (1964-1968), who used studies on pattern recognition to become one of the most successful hedge fund managers and investors on Wall Street. Another was Donald Knuth, who worked as a Staff Mathematician in the Communications Research Division of IDA in 1968–1969, before moving to Stanford and becoming a major figure in advancing computer software and programming languages. In the summers, prominent personalities in the fields of computer science and mathematics would attend annual workshops as visiting staff members. IDA promoted a “thinkers first” culture, modeled largely after European academic traditions. Like their colleagues at the United Kingdom’s Government Communications Headquarters (GCHQ), IDA’s “crypies” took afternoon tea. Whenever one of the mathematicians “cracked a big one” and solved a major problem, champagne toasts were held as colleagues saluted one another’s accomplishments. It was important that the mathematicians first and foremost enjoyed what they were doing, solving the nation’s pressing problems how they saw fit and at their own pace. In the 1980s, IDA personnel would spend summers at GCHQ in Cheltenham, working along their colleagues in the “Five Eyes” intelligence alliance (UK, USA, Canada, Australia, and New Zealand). Many of the methods developed in the mid-20th century (both public and classified) are still used by today’s cybersecurity experts.
In 1989, CCR La Jolla was established to “foster stronger technical interactions with the West Coast academic and industrial research community,” striving to capture the bright minds that did not want to leave the familiar comforts of the Pacific Coast (Lieberman 1989). For at least the decade prior, IDA has already sent its CRD researchers on summer retreats to Southern California and the Naval Postgraduate School in Monterey, so the stage was already set for a formal presence in the area. CCS in Bowie opened in 1985 to put a strong computing center near NSA headquarters at Fort Meade. All three centers have hosted some of the world’s most powerful and groundbreaking computers, including those designed by the pioneering Seymour Cray. Today, IDA researchers in communications and computation are actively involved in network security, machine learning for intelligence, and large-scale data analytics.
First and foremost, IDA has sought bright minds interested in the intersection of intellectual challenge and national purpose. The federal defense and intelligence communities have come to rely on IDA for advice and research on which to base a range of decisions pertaining to the national security of the United Sates. Leadership there has fostered a culture of supportive peers, not competitive careerists focused on climbing the ladder. In present day, IDA is a member of the INFORMS Roundtable, a group of institutional INFORMS members that have demonstrated leadership in the application of OR and advanced analytics.
Compiled by: Reed E. Devany
Note: The author would like to thank Al Blumstein and Coke S. Reed for their helpful contributions and comments.
Links and References
Albert, J. G., Kamrases M., & J. A. Navarro (1967) Evaluating Aircraft Requirement in the Light of Varying or Uncertain Mission Mixes. Operations Research, 15(4):738-751.
Barrer, D. Y. (1957) Queueing with Impatient Customers and Ordered Service. Operations Research, 5(5): 650-656.
Ceruzzi, P.E. (2008) Internet Alley: High Technology in Tysons Quarter, 1945-2005. MIT Press: Cambridge, MA.
Hunter D. E. (2015) Roundtable profile: analytics at IDA. OR/MS Today, 42(4). (link)
IDA (1966) [advertisement]. Operations Research, 14(4): i.
Kelleher, G. J. (1967) A Damage-Limiting Shelter-Allocation Strategy. Operations Research, 15(2), 211-220.
Kelleher, G. J., ed. (1970) The Challenge to Systems Analysis: Public Policy and Social Change. Publications in Operations Research, No 20. John Wiley & Sons: New York.
Lieberman, D. I. (1989) Letter to the editor: Center for Communications Research. The Institute of Mathematical Statistics Bulletin, 18(6): 581.
Neuwirth, L. (2009) Nothing Personal: The Vietnam War in Princeton, 1965-1975. BookSurge Publishing: Charleston, SC.
ORSA (1959) Back Matter. Operations Research, 7(3): 415-422.
ORSA (1969) Front Matter. Operations Research, 17(4): i-iv.
Ponturo J (1979) Analytical Support for the Joint Chiefs of Staff: The WSEG Experience, 1948 - 1976. IDA Study 507, Institute for Defense Analyses (Arlington VA)
Pugh, G. E. (1960) Operations Research for the Secretary of Defense and the Joint Chiefs of Staff. Operations Research, 8(6): 839-846.
Schweitzer, P. J. (1967) Optimal Replacement Policies for Hyperexponentially and Uniformly Distributed Lifetimes. Operations Research, 15(2), 360-362.
Seiler, K. (1966) A Cost Effectiveness Comparison Involving a Tradeoff of Performance, Cost, and Obtainability. Operations Research, 14(3): 528-531.
Thomas, W. (2015) Chapter 16 – The Challenge of Rational Procurement. in Rational Action: The Sciences of Policy in Britain and America, 1940-1960. MIT Press: Boston, MA.
Associated Historic Individuals
Barlow, Richard E.Blumstein, Alfred
Borsting, Jack R.
Cushen, W. Edward
Gass, Saul I.
Gross, Donald
Isaacs, Rufus
Knuth, Donald
Koopman, Bernard
Montroll, Elliott W.
Morse, Philip M.
Rinehart, Robert F.