Connections before connectionism
Exploring the institutional links in the "Perceptrons controversy"
Image by the National Museum of the U.S. Navy - source
The debate around artificial intelligence's (AI) split into two rival research strands, Symbolic AI and Neural Networks (later called ‘Connectionism’), is among the most notable developments in AI history. According to the popular view, after Marvin Minksy and Seymour Papert published their mathematical proofs showing the impossibility of using neural networks in Perceptrons in 1969, Symbolic AI became the winning paradigm for about two decades, during which funding for neural networks was virtually non-existent.
This essay looks beyond the main figures on the opposing sides of the controversy (Marvin Minsky and Frank Rosenblatt) towards the broader context of the early years of AI and attempts to unpack some of the folklore surrounding it. It starts from the assertion that “scientists of the Symbolic AI approach claimed that it would never be possible to create intelligent machines based on neural networks.”[1] It aims to unearth the extent to which Minsky “managed to convince the main funding organisations that supporting the Symbolic AI approach would be much more promising than funding research in neural networks” and “skilfully used ... personal contacts within these funding organizations.”[2]
One of the earliest books mentioning the dispute was published by journalist Pamela McCorduck in 1979[3]. Written for a broader public and punctuated by interviews with significant figures like Edward Feigenbaum, John McCarthy, Hubert Dreyfus and Marvin Minsky, it framed the event in a decades-long history of AI. This early account was published before the debate re-opened in the 1980s and is not presented as a major event at this point. McCorduck paid little attention to the reasons why Minsky, then a leading figure in computer science, criticized the perceptron so fiercely other than him being “irritated” by Rosenblatt [4]. McCorduck’s retelling also seems to have gotten some details wrong, including Rosenblatt’s being “a classmate of Minsky’s at Bronx [High School of] Science.”[5] In fact, Minsky, who graduated in 1945, was a grade behind Rosenblatt.[6]
Later sociological studies such as by Olazaran and Guice attempt to correct popular versions of “the rise and fall of ‘perceptrons’” which, according to Guice, have been passed down as “professional folklore.”[7] In his 1993 study, Olazaran re-evaluates the arguments in Perceptrons, arguing that it was not the proofs themselves that relegated neural networks to the sidelines of mainstream research, but a process of legitimation for the Symbolic approach, which happened with the participation of institutional players like ARPA.[8] Guice's (1998) socio-historical account of the debate, based on historical literature, contemporaneous sources, and interviews with researchers and ARPA officials, is perhaps the most well-researched. That same year, the first oral history monograph focusing specifically on of neural nets was also published.[9]
First ventures into automated intelligence
The development of AI in the early days of computing was taking place against the backdrop of the Cold War. After the USSR defeated the US in the space race with the launch of Sputnik in 1957, that same year, President Eisenhower approved funding for the Defense Department’s Advanced Research Projects (DARPA). Indeed, the first period of DARPA’s activity was “driven by the need to regain what at the time appeared to be lost U.S. technological leadership in space and strategic systems.”[10] As we will see, DARPA and the first directors of its Information Processing Techniques Office (ITPO), founded in 1962, played an important role in steering the direction of AI research in the early days.
The precipitating event for the perceptrons controversy was Frank Rosenblatt’s demonstration of the perceptron, which caused a media frenzy in 1958 after being revealed by the Office of Naval Research (ONR), the project’s funder. Rosenblatt, at the time a research psychologist at Cornell Aeronautical Lab, claimed the machine was “capable of having an original idea” and “of perceiving, recognizing and identifying its surroundings without any human training or control.”[11] These claims were perceived as exaggerations and reportedly irritated a lot of people.[12] One of these was Minsky, who was irked by “the wall-to-wall coverage of Rosenblatt and his machine” and may have felt that Rosenblatt didn’t belong on the “turf” of hard science, given that his training was in psychology and not mathematics (in which Minsky had a PhD).[13] Minsky, who had pioneered the first neurocomputer (the SNARC) [14] but abandoned this line of research, became quite hostile towards the neural networks approach and began campaigning against them.[15]
One of the means for this was the book Perceptrons, in which they brought forth mathematical proofs against the single-layer perceptron — a simplified, less powerful variant of what Rosenblatt had presented. The authors had circulated manuscripts before publication which, according to Robert Hecht Nielsen, wove in something of a “personal attack on Rosenblatt” and its authors later “expunged the vitriol” before publication.[16] Although copies of the original manuscript are not known to have survived, the introduction to Perceptrons still bears some traces of condescension towards the “flourish of romanticism” that marked the start of the sciences of computation of which neural networks were a part of.[17] This criticism is even re-iterated in the added prologue to the book’s 1988 edition, where the authors attack the supposed reluctance of some “romantic” proponents of connectionism to assume a more critical analysis, “perhaps because the spirit of connectionism seems itself to go somewhat against the grain of analytic rigor.”[18]
As Papert admits, there was “some hostility in the energy behind the research reported in Perceptrons,” but this drive was partly due to the fact that “funding and research energy were being dissipated on ... misleading attempts to use connectionist methods in practical applications” (emphasis added).[19] Although the real impact of the book is contested, with some maintaining that it was “hard to see why this was taken as a real critique of neural networks”[20] given that it attacked the most basic version of the perceptron, Minksy and Papert’s claims nevertheless had an impact on the trends in the AI research community and especially on those who cared about Minksy’s opinion as a leading figure. As Finnish researcher Teuvo Kohonen points out,
“it went roughly this way: you start telling somebody about your work, and this visitor or whoever you talk to says, ‘Don't you know that this area is dead?’”[21]
There is clear evidence of animosity between the two research groups, with many remembering “as great spectator sport the quarrels Minsky and Rosenblatt had on the platforms of scientific conferences in the late 1950s and early 1960s.”[22] Given Minsky’s notoriety in the then-small world of AI research, and his outspoken opposition to neural networks, it’s plausible that at least part of the fall from grace of neural networks could be attributed to Minsky’s claims and his use of his social influence. Yet, understanding field’s quiet years also requires an insight into the funding bodies and their reliance on the opinions of a few key leaders located in ‘centres of excellence.’
From MIT to DARPA and back
The first head of the Information Processing Techniques Office (ITPO) at DARPA (1962-1964) was J.C.R Licklider. From 1950 until 1957, he was associate professor in the Department of Electrical Engineering and, later on, in the Department of Economics and Social Science at MIT.[23] By this time, Minsky “knew him well,” as he had been a student of Licklider’s at Harvard.[24] As Guice found out, before starting at IPTO in 1962, Licklider was vice-president of BBN, a consultancy firm founded by two acousticians from MIT. At BBN, Licklider was the main organizer of a time-sharing system for the PDP-1 computer “along with MIT staff including Minsky and McCarthy.”[25]
For ARPA’s first AI contract, Project MAC, the funding process seems to have been rather informal, with Licklider going back to MIT in search of people to start a big computers lab.[26] In need of an “an engineer of stature with broad experience,” Licklider asked a former colleague (Robert Fano of MIT, who, according to himself, volunteered to head the project) to bring to him in his new role at ARPA[27] a proposal for an improved version of the elementary version of the time-sharing system that was in operation at MIT at the time.[28] Once the project got underway, Minsky’s group became the first one to receive funding through Project MAC, which “appears to have helped established cooperative relations between the emerging IPTO community and the emerging AI community.”[29]
Nevertheless, it must be noted that Licklider had a strong vision for computer research even before he was invested at DARPA. He exposed this in a now-seminal paper titled Man-Computer Symbiosis.[30] For him, computer research had implications beyond military customers and should have resulted into potentially large payoffs for the Department of Defense’s processing needs.[31] In essence, his ideas involved interactive computing and a focus on making computers effective at handling administrative calculations, while giving programmers fast enough feedback to code and debug programs efficiently. DARPA attributed its success to this vision in some areas, and it underlay DARPA’s Information Science and Technology Office program until at least 1990.[32]
This direction, coupled with the agency’s explicit policy of seeking out ‘centres of excellence,’ led to “considerable focus on a few academic centres, most notably MIT, Stanford and Carnegie-Mellon.”[33] The preference for research with high potential payoff and Licklider’s interest in real-time computing motivated ITPO’s first projects to focus on “time-sharing, computer languages, programming and, under various other budget lines, artificial intelligence.”[34] According to ONR research manager Marvin Denicoff, Rosenblatt’s work “never attracted that kind of money, because he wasn’t offering a large payoff...in the application sense, world problem solving.”[35]
Guice’s assessment shows that there was a clear preference at DARPA for supporting MIT-related research groups. To some extent, this is tied to the agency’s policy that favoured centres of excellence and high-payoff projects.[36] This approach is seen by some as a strategy of picking winners that leads to a tendency to “supporting dinosaurs.”[37] Beyond being a declared institutional strategy, it was also an approach that allowed the use of personal social capital. Renowned research centres like MIT, where Licklider himself knew many people,[38] were fertile ground for the creation of social affinities, which Licklider made use of in his new role at DARPA.
In the acknowledgements of Perceptrons, Minsky and Papert express their gratitude towards the IPTO directors that funded them through Project MAC as “the imaginative band of men who built [the Information Sciences Branch]” – J.C.R Licklider, I.E. Sutherland, R.W. Taylor, and L.G. Roberts.[39] It might be interesting to investigate the relations that Minsky had with the other three directors in order to untangle possible connections. At least in the early years of the controversy (1960-1964), which we looked at, Minsky cannot be said to have “skilfully used his personal contacts” or clearly influenced AI funding priorities based on the reviewed evidence. At the time, AI was subsumed under computer research funding, itself already impregnated with the ideas of institutional leaders like Licklider. In that sense, for Minsky, it’s perhaps a case of being in the right place, at the right time to get the funding – at least as far as Project MAC is concerned. There’s no doubt that Licklider’s background at MIT and connection with the likes of Minsky had an impact on his views of what kind of research is worth pursuing, but it’s unlikely he will have been “used” by Minsky – especially since he had his own ideas of what projects to pursue and moreover, Minsky had been his student, not the other way round.
As for the book, the circulation of the preprints, and his other speaking engagement, Minsky can be said to have used his influence in the field against neural networks, but it’s not clear to which extent the book played a major part. It’s more plausible that “the fashions in computer science departments had shifted the emphasis away from neural nets to the more symbolic methods of AI.”[40] Though it remains difficult to quantify the influence that DARPA’s continued support of Symbolist approaches had on these ‘fashions,’ it’s reasonable to think that it was not negligible.
This Symbolic AI – Neural Networks controversy underscores a decades-long dispute for the right to lay claim to having pioneered the field of Artificial Intelligence. The controversy spanned over decades and there would be much more to be said about it. But based on what we’ve looked at, it’s too facile to pin the blame on (or give the entire credit to) Minsky. While he was an influential figure, his collaborators, followers, and funders also played a part in the dominance of the Symbolic approach. If there’s one thing this investigation brings to light, it is that interest and funds tend to gravitate towards specific nodes of interest in institutional networks, and social actors collectively contribute to a field’s course of development.
[1] Here both Minsky and Papert are assumed to have used their connections. This essay focuses only on the former. Uli Meyer and Ingo Schulz-Schaeffer, Three Forms of Interpretative Flexibility, 32.
[2] Ibid., 33.
[3] Pamela McCorduck, Machines who think.
[4] Ibid., 106.
[5] Ibid., 105
[6] Melanie Lefkowitz, Professor’s perceptron paved the way for AI – 60 years too soon.
[7] Jon Guice, Controversy and the State: Lord ARPA and Intelligent Computing, 105.
[8] Mikel Olazaran, A Sociological Study of the Official History of the Perceptrons Controversy, 638.
[9] Edward Rosenfeld and James A. Anderson, Talking nets: an oral history of neural networks
[10] Richard H. van Atta, Seymour J. Deitchman and Sidney G. Reed, DARPA Technical Accomplishments, Volume III, S-5.
[11] Frank Rosenblatt, The Design of an Intelligent Automaton.
[12] Pamela McCorduck, Machines who think, 106.
[13] Robert Hecht Nielsen, Chapter 13, 304. In Rosenfeld; Anderson, Talking nets: an oral history of neural networks.
[14] Unknown author, Brief Academic Biography of Marvin Minsky.
[15] Many of those interviewed by Rosenfeld and Anderson remember Minsky being vocal against neural networks during the 60s. Jack D. Cowan, Chapter 5, 109
[16] Robert Hecht Nielsen, Chapter 13, 305. In Rosenfeld; Anderson, Talking nets: an oral history of neural networks.
[17] Marvin Minsky and Seymour Papert, Perceptrons: an introduction to computational geometry, 4.
[18] Marvin Minsky and Seymour Papert, Perceptrons (1988), vii.
[19] Seymour Papert, One AI or Many?, 4–5.
[20] Michael A. Arbib, Chapter 10, 156. In Rosenfeld; Anderson, Talking nets: an oral history of neural networks.
[21] Teuvo Kohonen, Chapter 7. In Rosenfeld; Anderson, Talking nets: an oral history of neural networks.
[22] Pamela McCorduck, Machines who think, 106.
[23] National Academies of Sciences, Engineering, and Medicine, Biographical Memoirs, 194, 201.
[24] Daniel Crevier, A.I: The tumultuous history of the search for artificial intelligence.
[25] Robert S. Englemore, Interview with J.C.R. Licklider, 220, qtd. in Guice (1998).
[26] Robert Fano, Interview by Arthur L. Norberg. April 20, 1989. Cambridge Mass., OH 165, 14.
[27] Karl L. Wildes and Nilo A. Lindgren, A century of electrical engineering and computer science at MIT, 1882-1982, 347.
[28] Robert Fano, Interview by Arthur L. Norberg. April 20, 1989. Cambridge Mass., OH 165, 14–15.
[29] Jon Guice, Controversy and the State: Lord ARPA and Intelligent Computing.
[30] J. C. R. Licklider, Man-Computer Symbiosis.
[31] Richard H. van Atta, Seymour J. Deitchman and Sidney G. Reed, DARPA Technical Accomplishments, Volume III, V-18.
[32] Ibid.
[33] Ibid., IV-5.
[34] Jon Guice, Controversy and the State: Lord ARPA and Intelligent Computing, 116.
[35] Mikel Olazaran, A Sociological Study of the Official History of the Perceptrons Controversy, 638.
[36] Richard H. van Atta, Seymour J. Deitchman and Sidney G. Reed, DARPA Technical Accomplishments, Volume III, V-12.
[37] Bart Kosko, Chapter 17, 410. In Rosenfeld; Anderson, Talking nets: an oral history of neural networks.
[38] Robert Fano, Interview by Arthur L. Norberg. April 20, 1989. Cambridge Mass., OH 165, 14.
[39] Marvin Minsky and Seymour Papert, Perceptrons: an introduction to computational geometry.
[40] Michael A. Arbib, Chapter 10, 222. In Rosenfeld; Anderson, Talking nets: an oral history of neural networks.
Bibliography
Arbib, Michael A. “Chapter 10.” In Rosenfeld; Anderson, Talking nets: an oral history of neural networks.
Cowan, Jack D. “Chapter 5.” In Rosenfeld; Anderson, Talking nets: an oral history of neural networks.
Crevier, Daniel. A.I: The tumultuous history of the search for artificial intelligence. New York: BasicBooks, 1993.
Englemore, Robert S. “Interview with J.C.R. Licklider.” In Expert systems and artificial intelligence: Applications and management. Edited by Thomas C. Bartee. 1st ed., 213–90. Indianapolis Ind. USA: H.W. Sams, 1988.
Fano, Robert. Interview by Arthur L. Norberg. April 20, 1989. Cambridge Mass.
Guice, Jon. “Controversy and the State: Lord ARPA and Intelligent Computing.” Social Studies of Science 28, no. 1 (1998): 103–38. http://www.jstor.org/stable/285752.
Hecht Nielsen, Robert. “Chapter 13.” In Rosenfeld; Anderson, Talking nets: an oral history of neural networks.
Kohonen, Teuvo. “Chapter 7.” In Rosenfeld; Anderson, Talking nets: an oral history of neural networks.
Kosko, Bart. “Chapter 17.” In Rosenfeld; Anderson, Talking nets: an oral history of neural networks.
Lefkowitz, Melanie. “Professor’s perceptron paved the way for AI – 60 years too soon.” Accessed March 11, 2024. https://news.cornell.edu/stories/2019/09/professors-perceptron-paved-way-ai-60-years-too-soon.
Licklider, J. C. R. “Man-Computer Symbiosis.” IRE Transactions on Human Factors in Electronics HFE-1, no. 1 (1960): 4–11. doi:10.1109/THFE2.1960.4503259.
McCorduck, Pamela. Machines who think: A personal inquiry into the history and prospects of artificial intelligence. 25th anniversary update. Natick Mass.: A.K. Peters, 2004.
Meyer, Uli, and Ingo Schulz-Schaeffer. “Three Forms of Interpretative Flexibility.” 2006; Science, Technology & Innovation Studies. doi:10.17877/DE290R-971.
Minsky, Marvin, and Seymour Papert. Perceptrons: an introduction to computational geometry. Cambridge, Mass. u.a.: MIT Press, 1969.
Minsky, Marvin, and Seymour Papert. Perceptrons: An introduction to computational geometry. Expanded ed. Cambridge Mass.: MIT Press, 1988, 1969.
National Academies of Sciences, Engineering, and Medicine, ed. Biographical Memoirs 75. Washington, D.C.: National Academies Press, 1998.
Olazaran, Mikel. “A Sociological Study of the Official History of the Perceptrons Controversy.” Social Studies of Science 26, no. 3 (1996): 611–59. http://www.jstor.org/stable/285702.
Papert, Seymour. “One AI or Many?” In The Artificial intelligence debate: False starts, real foundations. Edited by Stephen R. Graubard. 1st MIT Press ed. Cambridge Mass.: MIT Press, 1988.
Rosenblatt, Frank. “The Design of an Intelligent Automaton.” Research Trends VI, no. 2 (Summer 1958). Accessed March 9, 2023.
Rosenfeld, Edward and James A. Anderson, eds. Talking nets: an oral history of neural networks. Cambridge, Mass.: MIT Press, 1998.
Unknown author. “Brief Academic Biography of Marvin Minsky.” https://web.media.mit.edu/~minsky/minskybiog.html.
van Atta, Richard H., Seymour J. Deitchman, and Sidney G. Reed. DARPA Technical Accomplishments, Volume III: An Overall Perspective and Assessment of the Technical Accomplishments of the Defense Advanced Research Projects Agency: 1958-1990. IDA Paper. Washington, DC, 1991, P-2538. Accessed March 9, 2024. https://apps.dtic.mil/sti/tr/pdf/ADA241680.pdf.
Wildes, Karl L., and Nilo A. Lindgren. A century of electrical engineering and computer science at MIT, 1882-1982. Cambridge Mass.: MIT Press, 1985.