The new memoir by Steve Jobs’ daughter, Lisa Brennan-Jobs, titled Small Fry. The last vestiges of the thin veneer of counter-culture dissention from what used to be called “straight” culture are here stripped away. It will be released to the general public on Sept. 4. Excerpts have been leaked to Vanity Fair and other outlets. The buzz now is about that already famous line, uttered by Lisa’s stepmother, Laurene Powell-Jobs, at a therapy session, when Lisa was a teenager, at which were present Steve, Laurene, and Lisa. Ms. Brennan-Jobs cried at one point and says to the therapist she feels lonely and has wanted her parents to say good night to her. Ms. Powell-Jobs responds to the therapist: “We’re just cold people.”
I want to link this chilling tale with some words and ideas written by Joseph Weizenbaum, whose seminal work Computer Power and Human Reason sounded the alarm concerning a new sociopolitical dynamic way back in 1976. In those days, there was a good deal of resistance and skepticism about the burgeoning computer technology on the part of the general public. But it was only a surface phenomenon, as the following account shows. It tells the tale of the creation of ELIZA, a computer program devised by Mr. Weizenbaum, which factored the response characteristics of a Rogerian psychotherapist, as I have related in these pages before. In the article “Silicon Valley is not Your Friend”, Norm Cohen traces the early development of this trend:
“Interactions between people and their computers were always going to be confusing, and that confusion would be easy for programmers to exploit. John McCarthy, the computer-science pioneer who nurtured the first hackers at M.I.T. and later ran Stanford’s artificial intelligence lab, worried that programmers didn’t understand their responsibilities. ‘Computers will end up with the psychology that is convenient to their designers (and they’ll be fascist bastards if those designers don’t think twice),” he wrote in 1983. “Program designers have a tendency to think of the users as idiots who need to be controlled. They should rather think of their program as a servant, whose master, the user, should be able to control it.’
“Call it the Eliza problem. In 1966, Joseph Weizenbaum, a professor at M.I.T., unveiled a computer program, Eliza, which imitated a psychotherapist. It would, by rote, inquire about your feelings toward your parents or try to get you talking by rephrasing what you said in the form of a question. The program immediately touched a nerve, becoming a national phenomenon, to the surprise of Mr. Weizenbaum. For example, The New York Times swooned: “Computer is Being Taught to Understand English” . Eliza understood nothing, in truth, and could never reach any shared insight with a ‘patient.’ Eliza mechanically responded to whatever appeared on the screen. A typical therapy session quickly devolved into a Monty Python sketch. (Patient: You are not very aggressive, but I think you don’t want me to notice that. Eliza: What makes you think I am not very aggressive? Patient: You don’t argue with me. Eliza: Why do you think I don’t argue with you? Patient: You are afraid of me. Eliza: Does it please you to believe I am afraid of you?)
“Imagine Mr. Weizenbaum’s surprise when his secretary looked up from her computer and interrupted her exchanges with Eliza to say to him, ‘Would you mind leaving the room, please?’ She wanted privacy for a conversation with a machine! Mr. Weizenbaum, appalled, suddenly saw the potential for mischief by programmers who could manipulate computers and potentially the rest of us. He soon switched gears and devoted his remaining years to protesting what he considered the amorality of his computer science peers, frequently referring to his experiences as a young refugee from Nazi Germany.
One thought from the above passage stands out in a blinding flash of sickening realization: Computers will end up with the characteristics of their designers as seen from a psychological standpoint.
Returning to the article by Norm Cohen: “Neither Mr. Weizenbaum nor Mr. McCarthy mentioned, though it was hard to miss, that this ascendant generation were nearly all white men with a strong preference for people just like themselves. In a word, they were incorrigible, accustomed to total control of what appeared on their screens. ‘No playwright, no stage director, no emperor, however powerful,’ Mr. Weizenbaum wrote, ‘has ever exercised such absolute authority to arrange a stage or a field of battle and to command such unswervingly dutiful actors or troops.’”
Howe does such a mindset gain a foothold in the collective consciousness of society? I believe it is instructive to point once again to Herbert Marcuse’s “Some Implications of Modern Technology” of 1941, which traces the course of a world-view that, developing wholly from within established rational frameworks, made it possible to pervert the rationalistically configured presuppositions animating the liberal autonomous individual into something approaching its opposite. I reproduce the relevant passage from this essay below:
“The principle of competitive efficiency favors the enterprises with the most highly mechanized and rationalized industrial equipment. Technological power tends to the concentration of economic power, to “large units of production, of vast corporate enterprises producing large quantities and often a striking variety of goods, of industrial empires owning and controlling materials, equipment, and processes from the extraction of raw materials to the distribution of finished products, of dominance over an entire industry by a small number of giant concerns. . . .” And technology “steadily increases the power at the command of giant concerns by creating new tools, processes and product. Efficiency here called for integral unification and simplification, for the removal of all “waste,” the avoidance of all detours, it called for radical coordination. A contradiction exists, however, between the profit incentive that keeps the apparatus moving and the rise of the standard of living which this same apparatus has made possible. “Since control of production is in the hands of enterprisers working for profit, thcy will have at their disposal whatever emerges as surplus after rent, interest, labor, and other costs are met. These costs will be kept at the lowest possible minimum as a matter of course.”‘ Under these circumstances, profitable employment of the apparatus dictates to a great extent the quantity, form and kind of commodities to be produced, and through this mode of production and distribution, the technological power of the apparatus affects the entire rationality of those whom it serves. Under the impact of this apparatus individualistic rationality has been transformed into technological rationality. It is by no means confined to the subjects and objects of large scale enterprises but characterizes the pervasive mode of thought and even the manifold forms of protest and rebellion [italics mine-dw]. This rationality establishes standards of judgment and fosters attitudes which make men ready to accept and even to introcept the dictates of the apparatus.”
To my mind, this last phrase in the last sentence quoted above is key–“attitudes which make men ready to accept and even to introcept the dictates of the apparatus”, coupled with the idea that computers will end up with the characteristics that are convenient to their designers, designers that have thoroughly introcepted what might be called scientistic presuppositions, ones which all but eliminate affect and privilege cold rationality, can only have an impact that replicates these features in the general populace. We are led down a pathway in this examination that leads to many others. One of them implied in the above quote is that traced by Jacques Ellul, whose The Technological Society of 1954 undertakes a deep examination of this drive for efficiency in the context of technological development. Efficiency is expressed in optimally rationalized action, and the implications for a mindset which privileges, nay reifies, efficiency over all other concerns are profound. Computers will end up with the characteristics of their designers, and from there, these characteristics are passed on to the end-user on a scale unseen in human history. A key point in Ellul’s critique is that this “technological imperative” has been allowed to achieve full autonomy, that all other concerns are subsumed into itself, which subordinates ends to means and is fully independent of any specific form of social organization, contrary to what Marcuse and, as well, by most leftist critics of the technocracy would have us believe. Technique, in this reading, follows a protocol that proceeds without regard to context: “The primary aspect of autonomy is perfectly expressed by Frederick Winslow Taylor, a leading technician. He takes, as his point of departure, the view that the industrial plant is a whole in itself, a ‘closed organism’, an end in itself…the complete separation of the goal from the mechanism, the limitation of the problem to the means, and the refusal to interfere in any way with efficiency, all this is clearly expressed by Taylor and lies at the basis of technical automony.”
Considering the totalizing nature of this phenomenon, it is easy to see how the cohort which invests in it could grow almost without limit, given the rise of the new type of personality so memorably voiced by Ms. Laurene Powell-Jobs in the quote cited above. If one accepts the basic argument Marcuse puts forward in “Some Implications of Modern Technology”, especially that this technological rationality “make[s] men ready to accept and even to introcept the dictates of the apparatus”, one begins to grasp the enormity of the problem. Ellul and Marcuse convincingly demonstrate that this technological rationality operates at the level of the deeply held philosophical conviction. This is nothing short of an ideological imperative, an ethos of living, which dovetails into the mechanistic processes of “persuasive design” to encourage the rise of “the cold people” and marginalize the rest of us.