Artificial Intelligence And Aesthetics

Introduction

Firschein et al. [1] describe some products that can be expected to result from research in artificial intelligence. One of these products is "creation and valuation systems" which they describe as computer systems "capable of creative work in such areas as music, art (painting, sculpture, architecture), literature (essays, novels, poetry), and mathematics, and able to evaluate the work of humans" (and presumably the work of other "creation systems"). "Creation" and "valuation" in the arts are discussed traditionally in terms of "design" and "criticism". In this paper, we review some of the work we have done to provide a basis for the development of systems (algorithms) capable of doing design and criticism

in the arta.

The goals of our research include:

1. The development of a simple, basic structure for criticism algorithms and design algorithms in the arts.
2. The investigation of traditional issues and approaches in aesthetics and art theory in terms of this structure.
3. The development of specific criticism algorithms and design algorithms for restricted art forms.

This paper is concerned mainly with our work on goal (1). An overview of the basic structure developed for criticism and design algorithms is presented. Preliminary work on goal (2) is reported in [2-5]. Issues in aesthetics and art theory that are being investigated in terms of the developed structure for criticism and design algorithms include the definition of "work of art"; different ways of understanding objects as works of art: notions of "form and content", "representation", and "expression"; different ways of evaluat.ing objects as works of art: notions of "unity and variety"; problems of copies, multiple performances, forgeries; found objects. Important parts of specific criticism algorithms and design algorithms (work on goal (3)) for number sequences and for non-representational, geometric paintings having generative specifications (6] have been developed. Work on number sequences can be found in [2,5]; work on paintings can be found in [2-5). A comprehensive treatment of our work on aesthetics is expected to appear in [7) .

George Stiny System Science Department

Criticism and Design

The task of criticism algorithms is taken to be the production of a statement of how a given object is interpreted and evaluated as a work of art. The task of design algorithms is taken to be the production of an object as a work of art in response to some specified initial conditions. The overall structure postulated for criticism algorithms is given in Figure la, for design algorithms in Figure lb. (For an interesting, early, but neglected discussion of systems with receptors, internal'~easoning'components, and effectors as models of thought processes see [8].)

Criticism Algorithms

The schema for criticism algorithms given in Figure la consists of:

1. An object as a possible work of art. Here the notion of "object" is used in its widest possible sense to include, for example, musical or theatrical performances as well as paintings or novels.
2. A receptor consisting of a sensory input transducer (shown schematically by an "eye") and linked algorithm. For example, a possible receptor for music is described in [9).
3. The output of the receptor: a description, A , of the object. For example, for music, drama, literature, or architecture, A could resemble the score, script, text, or plan.
4. An aesthetic system, an algorithmic specification of the viewpoint or knowledge used in some approach to art. The aesthetic system encodes the conventions and criteria needed for the determination of whether an object is considered a work of art and if so how it can be interpreted and evaluated as a work of art. The structure of aesthetic systems is described in a subsequent section.
5. An analysis algorithm. The analysis algorithm uses the kn~ledge encoded in the aesthetic system· to interpret and evaluate the object as a work of art (if possible). The task of the analysis algorithm is described more precisely in a subsequent section.
6. The output of the analysis algorithm: the interpretation and evaluation of the object as a work of art.

1. An effector consisting of a transducer (shown schematically by "hands") and linked algorithm.
2. A statement of how the object is interpreted and evaluated as a work of art.

Design Algorithms

The schema for design algorithms given in Figure lb consists of:

1. Some initial conditions, e.g., a person whose portrait is to be painted or the injunction "Write music for the royal water-party between Whitehall and Limehouse to be held on August 22, 1715."
2. A receptor consisting of a sensory input transducer (shown schematically by an "eye") and linked algorithm.
3. The output of the recepto~: a specification of the initial conditions.
4. An aesthetic system as in criticism algorithms.
5. A synthesis algorithm. The synthesis algorithm uses the knowledge encoded in the aesthetic system to construct the description of the best possible object which satisfies the initial conditions. The task of the synthesis algorithm is described more preciaely , in a subsequent section.
6. The output of the synthesis algorithm: the intended description, A , of the work of art to be produced.
7. An effector consisting of a transducer (shown schematically by "hands") and linked algorithm.
8. The object produced by the effector. This object is the work of art produced by the design algorithm.

B

ApProaches in Criticism and Design

Criticism and design in the arts can be done in many different ways. The variety of actual approaches in criticism is apparent when two different observers interpret and evaluate the same object as a work of art in two different ways. For example, consider the disparity among opening night reviews of a given Broadway play. The variety of actual approaches in design is apparent when two different artists produce two different objects as works of art in response to identical initial conditions. For example, consider the disparity between the commissioned portraits of L.B.J.

The aim of our work is not to produce any single, authoritative criticism algorithm or design algorithm, as we consider any approach to art legitimate. Rather, we postulate a structure for criticism algorithms and design algorithms in which a variety of approaches to art c:an be represented. The practice of criticism and design in the arts can be done in many different ways and still be modelled using the postulated structure for criticism algorithms and design algorithms . Different approaches in criticism or design might result in differences in any of the components of criticism algorithms or design algorithms. For example, aesthetic systems corresponding to several different approaches to non­ representational, geometric paintings have been suggested [4,5] . Each of these aesthetic systems would encode different conventions and criteria for interpreting and evaluating paintings as works of art. The interpretation of a painting using one of these aesthetic systems would be done in terma of the shapes in the painting, using a second in terms of the colors in the painting, using a third in terms of the associations attached to the painting, etc:. Of course, there can be many different aesthetic: systems allowing for the interpretation of paintings in terms of shape, color, or associations. Similarly, the evaluation of a painting using one of these aesthetic: systems would be done in a variety of different wayo. If these aesthetic systems were used in criticism algorithms, different statements of how a given painting is interpreted and evaluated would be produced. If these aesthetic systems were used in design algorithms, different paintings would be produced in response to some given initial conditions. None of these aesthetic systems is taken to be definitive. Again, any approach to art is considered legitimate. Our interest is in investigating the many possible approaches to. art in a uniform way.

Aesthetic: Systems

The key component in both criticism algorithms and design algorithms is an aesthetic system. Recall that an aesthetic system is an algorithmic specification of the viewpoint or knowledge used in some approach to art. An aesthetic system encodes the conventions and criteria needed for the determination of whether an object is considered a work of art and if so, how it can be interpreted and evaluated as a work of art.

An aesthetic system cor.sists of four algorithms': an algorithm A which defines a set of interpretations IA , a reference decision algorithm R , an evaluation algorithm E , and an evaluation comparison algorithm C (see Figure 2). Here we are not concerned with the internal structure of these algorithms but rather with the characteristics and inter-relationships of their

inputs and outputs.

The algorithm A in an aesthetic system defines the set of interpretations lA · The set of interpretations lA consists of all input­ output pairs <a,B> for the algorithm A (see Figure 2). AD interpretation in the set lA is a possible way of understanding some object as a work of art using the viewpoint or knowledge specified by the aesthetic system. For example, one component of an interpretation may be a description A of an object and the other component may be a specification of how that description is construed. The case where ·the output compooent, B , of an interpretation is a description of an object provides an interesting paradigm for the study of "form" or "internal coherence" iD the arts. The case where the input component, a , of an interpretation is a description of an object provides an interesting paradigm for the study of "content" or "external evocations" iD the arts. [4,5]

Whether an interpretation refers to an object, i.e. is an interpretation of an object, is determined by the reference decision algorithm R in an aesthetic system (see Figure 2). The inp4t to the reference decision algorithm is an inter­ pretation iD the set IA and the description A of an object. The output of the reference decision algorithm R indicates whether the interpretation refers to an object having the description A

The set of interpretations IA and the reference decision algorithm R .provide the basis for determiDiDg whether an object is a work of art for some aesthetic system. An object is a work of art for an aesthetic system i .f and only if there is an interpretation in the set IA which refers to th' object using the reference decision algorithm R · We believe this definition of work of art has important implications for a variety of current issues in aesthetics and art theory [7].

It must be stressed that a given object may be considered a work of art usinp, one aesthetic system and may not be considered a work of art using a second aesthetic system. Further, when an object is considered a work of art using two different aesthetic systems, its interpretation and evaluation in each of these systems may be quite different.

Aesthetic value is determined by the evaluation algorithm E . The evaluation algorithm assigns an aesthetic value to each interpretation in the set IA (see Figure 2). How an object is evaluated as a work of art depends on how the object is interpreted as a work of art. An interesting evaluation algorithm has been defined in terms of the relative lengths of the components of an interpretation. This evaluation algorithm provides a paradigm for the study of "unity and variety" in the arts and can be characterized in terms of algorithmic information theory [10] . These topics are explored in [2-5).

The relative merit of two aesthetic values is determined by the evaluation comparison algorithm C · One interpretation is aesthetically superior to a second interpretation in an aesthetic system when the aesthetic value assigned the first interpretation is determined by the evaluation comparison algorithm to be superior to the aesthetic value assigned to the second inter­ pretation. An interesting issue is whether the evaluation comparison algorithm should be an order [11) and if so, whether it should be total or partial [7 J.

Specific aesthetic systems ate being developed for number sequences [2,5] and for non­ representational, geometric paintings [2-5)'. The computer . implementation of important parts of a fully developed aesthetic system for painting is described in [4]. Additionally, some traditional aesthetic viewpoints for a variety of art forms are being examined in terms of aesthetic systems.

Analysis and Synthesis Algorithms

In a criticism algorithm, an analysis algorithm is used in conjunction with an aesthetic system to specify how an object having description A is interpreted and evaluated as a work of art (see Figure la). The task of an analysis algorithm is to find the best way to interpret the object as a work of art. For a given aesthetic system, the task of an analysis algorithm is to find the inter­ pretation in the set IA which refers, using the

reference decision algorithm R , to the object having the description A and which is assigned an aesthetic value by the evaluation algorithm E which is maximal in the sense of the evaluation comparison algorithm C · The interpretation found by the analysis algorithm is the beat way to under­ stand the object having the description A in terms of the viewpoint or knowledge specified by the aesthetic system.

In a design algorithm, a synthesis algorithm is used in conjunction with an aesthetic system to construct the description of a work of art (object) satisfying the given initial conditions (see Figure lb). The task of a synthesis algorithm is to construct the description of the best possible work of art which satisfies the initial conditions. For a given aesthetic system, the task of a synthesis algorithm is to find a description A of an object for which (1) the specified initial conditions are satisfied and (2) there is an interpretation in th~ set IA which would refer, using the reference decision algorithm R , to the object and which is assigned an aesthetic value by the evaluation algorithm E which is maximal in the sense of the evaluation comparison algorithm C · The descriptim found by the synthesis algorithm specifies the best work of art, in terms of the viewpoint or knowledge given by the aesthetic system, that satisfies the initial conditions.

Special purpose analysis algorithms and synthesis algorithms have been investigated. Analysis algorithms have been studied for use in conjunction with restricted types of aesthetic systems. Heuristic search meth0ds have been suggested for synthesis algorithms to be used in conjunction with the aesthetic system developed for paintings having generative specifications [2,4,5].

Problems and Prospects

The problem of constructing particular criticism algorithms and design algorithms can be extremely difficult. The process of criticizing or designing a work of art may be very complicated and can involve a full range of mental abilities. The ability to specify a criticism algorit~~ or a design algorithm may well presuppose the ability to formalize a wide range of perceptual and cognitive skills and a wide range of knowledge. For example, a critici"sm algorithm which allows for the interpretation and evaluation of Raphael's School of Athens may involve the ability to recognize painted shapes as people, the ability to recognize those people as representations of Greek philosophers as well as portraits of Italian artists of the 15th and 16th centuries, the ability to associate these people into groups in terms of their philosophical points of view as Greek philosophers as well as their spatial location in the pa~ting, the ability to relate the painting as part of art history, the ability to relate the ideas associated with the painting with the ideas in some cultural context, the ability to identify the emotions evoked by the various aspects of the painting, among many. Formalizing even the first of these abilities would be an extremely difficult task at the present time.

In [1], "creation and valuation systems" were predicted to be one of the last artificial intelligence products to be developed. This prediction seems well-founded in light of the difficulties involved in the specification of particular criticism algorithms and design algorithms and the need to include a wide variety of other artificial intelligence products in this specification. Our hope is that our work provides a productive first step toward the goal of developing "creation and valuation systems" as well as the basis for better understanding general questions of aesthetics and art theory.

1. 0. Firschein, M. Fischler, L. S. Coles, and J. M. Tenenbaum, "Forecasting and Assessing the Impact of Artificial Intelligence on Society", 3IJCAI, Stanford, 1973.
2. G. Stiny and J. Gips, "Formalization of Analysis and Design in the Arts", in W. Spillers (ed.), Basic Questions of Design Theory, North Holland Publishing Co . , Amsterdam, 1974.
3. J. Gips and G. Stiny, "An Investigation of Algorithmic Aesthetics", Leonardo, in press.
4. J. Gips, Shape Grammars and Their Uses, Birkhauser Verlag, Basel, Switzerland, in press. Also, Ph.D. Dissertation, Computer Science Department, Stanford, 1974.
5. G." Stiny, Pictorial and Formal Aspects of Shape and Shape Grammars, Birkhauser Verlag, Basel, Switzerland, in press. Also Ph.D. Dissertation, System Science Department, U.C.L.A., 1975.
6. (6] G. Stiny and J. Gips, "Shape Grammars and the Generative Specification of Painting and Sculpture", Proceedings of IFIPs Congress '71. Also in 0. Petrocelli (ed.), The Best Computer Papers of 1971. Auerbach, Princeton, New Jersey, 1972.
7. G. Stiny and J. Gips, Aesthetics: An Algorithmic Approach, monograph in preparation.
8. K. Craik, The Nature of Explanation, Cambridge University Press, Cambridge, 1943.
9. J. A. Moorer, "On the Segmentation of Continuous Musical Sound by Digital Computer", forthcoming Ph.D. Dissertaion and Computer Science Report, Stanford.
10. A. N. Kolmogorov, "Logical Basis for Information Theory and Probability Theory", IEEE Transactions on Information Theory, Vol. IT-14, No. 5, 1968.
11. (ll] W. S. McCulloch, "Toward Some Circuitry of Ethical Robots", Acta Biotheoretica, Vol. XI, 1956. Also in W. S. McCulloch, Embodiments of Mind, The M.I.T. Press, Cambridge, Mass., 1965.