Diego Mellado Martínez
Electronic Media Review, Volume Six: 2019-2020
ABSTRACT
This article presents a method that allows external data-driven software-based works to keep a record of the data used as input. “Self-documentation” will prevent the artworks from failures when data sources are no longer available. Data stored with timestamps will work as an alternative source and will provide information on how the artwork was at a given past moment. Included in the programming of the artwork from the very beginning, this method is a way of implementing preventive conservation. This strategy will be shown in several case studies: Daniel Canogar’s Cannula (2016), Ernest Edmonds’s Shaping Space (2007), Jose Carlos Martinat’s Ambiente de estéreo Realidad 3 – Brutalismo (2007), and Rafael Lozano-Hemmer’s Pulse Index (2010). Each case study used stored information for different results. In the case of Canogar’s studio, the use of stored data leads to what the studio calls demo-mode: an incomplete version of the artwork, referring to the demos of software and video games, present in computer magazines in the 1980s and 1990s. It is a matter of discussion if the artwork is still “the artwork” when entering this demo-mode.
Introduction
“Self-documentation” as conservation is one of several strategies that the studio of the US-Spanish artist Daniel Canogar (b. 1964) uses to conserve its artworks. These strategies have been a joint effort developed inside the studio by Diego Mellado and are described in his master’s thesis, “Software- and data-based conservation from the artist studio practice: Engineering approaches to documentation as a tool for artwork definition and conservation.” This article explores one of these strategies, aimed to provide protection from failure when an artwork requires data acquired from a source external to the artwork.
The concept of self-documentation consists of storing the information used to render the output of the artwork. After retrieval from its original source, this input data is stored on the computer that runs the artwork, along with a timestamp that defines when that data was accessed or “happened.” If the source of information is not reachable (e.g., the connection is lost, or it is no longer available—e.g., the website from where the data is retrieved is permanently down), the stored data is available.
Although at the technical level using stored videos does not represent a formal change to the artwork’s function, since the stored data is a valid source of data, at a conceptual or symbolic level, there is a big difference. Hence, it is questionable if the artwork is still the artwork, since one of its “significant properties” (Laurenson 2013) has been drastically altered. If, in the words of media theorist Boris Groys (2002), “The art world has shifted its interest away from the artwork and toward art documentation” (p. 53), it is a matter of discussion if the artwork is still “the artwork” when entering this demo-mode. Before moving into that discussion, it would be convenient to illustrate the self-documentation strategy and the demo-mode with an example.
Case Study from Studio Daniel Canogar: Cannula (2016)
Cannula (2016), shown in figure 1, is the first generative work from Daniel Canogar’s studio using real-time data retrieved from the Internet. As described on the artist’s web page, “Cannula´s animation riffs off painterly abstract expressionism. The palette of this artwork is not painting, but rather YouTube videos that correspond to queries introduced by the public. A computer downloads the top 100 videos of these queries, which then get projected as a liquid composition.”
Cannula is presented in a large format, a 75-inch display, running on a Mac Mini computer as of 2016. Figure 2 shows a system representation of the artwork. Considering that the public can interact with the artwork—by making queries—and that the artwork retrieves information from a remote location, this could be understood as an open system or, as in Espenschied (2019), an artwork with “blurry boundaries,” which increases its conservation issues. Figure 3 describes the algorithm through a flowchart. In this case, the algorithm has different “threads” or simultaneous processes running at the same time. The main thread calculates the visual output, the second thread checks the Internet connection periodically, and the third one downloads the contents when required. The third thread also stores the videos in memory.
As described in figure 3, videos are downloaded and stored before being used for the visual composition. If at some point the connection is down, instead of retrieving new videos, those downloaded are used and the source of data changes from external to internal: the system diagram that represents the artwork is now shown in figure 4, since the one in figure 2 is no longer valid. In this case, the algorithm continues to work unaffected by the change, as it received its expected inputs, and the rest of the elements and parts of the system perform in the expected way.
Nevertheless, the source of the information is by no means trivial. It is clear that moving from live data to stored data means a significant conceptual shift, but it will not be less of a change using a different source of videos, even if those could be retrieved in real time. As per the description of the artwork available on the artist’s web page, YouTube was selected as a source of data because of its significance in today’s society, as, in the words of the artist, “the new library paradigm of the 21st century.” Hence, only YouTube as source for the videos will deliver the correct symbolic message. Another alternative, such as Vimeo, would not have represented that “Babel tower” sense that the artist gives to YouTube. In other words, using YouTube has a significance that must be preserved. It is also a significant property of the artwork. In this case, the artist has left a clear statement about what to do in the future, but this is not always the case. A common agreement among stakeholders together with the proper documentation on the actions taken would be needed in such cases.
Espenschied (2019) presents an interesting point regarding the fact that YouTube is more than just the videos that can be download from it: “YouTube the service requires YouTube the organization to provide its full performative potential.” YouTube significance is based on the current historical and social consideration of the service and the organization. That consideration could change into something under censorship or could be abandoned by users. In that case, even the service could be available and its significance could be altered. In such a scenario, the artist stated that “the artwork should be left to evolve, as one of its components, YouTube, will be evolving too. It is based on the content and value of YouTube at execution time, even if that changes.”
If previously downloaded and stored data is recovered, which represents a past moment or iteration of the artwork, it is easy to relate the performance to documentation. Now the algorithm and the parts of the artwork show how the artwork was at a given moment or under certain conditions, but it is very questionable if those—algorithm and parts—are still composing the artwork. Considering that they are representing a past performance of the artwork, it could be possible to talk about documentation.
Usually, documentation is performed in a different support, such as videos of a performance or pictures of an installation. Even when the support is the same—a picture of a picture or a video of a video—the material objects of the artwork and the documentation are different. In this case, the support for the documentation is the same as the original artwork, being as close as possible to the original artwork but not being the artwork. Inside Daniel Canogar’s studio, the result is the “demo-mode”, a limited representation of the artwork, similar to the test versions of software and video games that computer magazines in the 1980s and 1990s used to include, usually in the form of CD-ROMs.
This strategy has been applied to all data works by Daniel Canogar since, and it is a cornerstone for the studio in the approach to conservation of data-based artworks. Especially among collectors, what happens to the artwork when its source of data is no longer available is a major concern. Demo-mode using previously stored contents presents a considerable degree of loss, but lesser than a blank screen, which is positive for all stakeholders.
Case Studies from Outside of the Studio
Neither storing data as a backup nor using it to solve problems related to availability or connections is exclusive to Daniel Canogar’s studio. Other artists have used this approach before for similar purposes. The next three case studies are presented to illustrate different applications of the self-documentation processes.
José Carlos Martinat’s Ambiente de estéreo Realidad 3 – Brutalismo (2007)
José Carlos Martinat’s (b. 1974) Ambiente de estéreo Realidad 3—Brutalismo, shown in figure 5, is described in Tate Modern’s website as:
“a scale model of the Peruvian military headquarters, an example of ‘brutalist’ architecture it was nicknamed the ‘Pentagonito’ (or ‘little Pentagon’). During the Fujimori presidency, the building became notorious for the torture, murders and disappearances conducted by the secret service. The sculpture incorporates a computer which has been programmed to search the internet for references to ‘Brutalismo / Brutalism’, picking up extracts about Latin American and global dictatorships but also on architecture, forging associations between different kinds of ‘brutalism’ which it spews out onto the gallery floor.”
As part of the Tate software-based artwork collection, Brutalismo has been presented in several case studies and papers such as those of Falcão (2010), Laurenson (2016), and Ensom (2018). Brutalismo “[relies] on a constantly changing input of data, and although it is possible to use a previously populated database to feed the piece, the idea of liveness is evoked as a significant or work defining property of the work. This dependency on a live data feed also points to the diffuse nature of the work as a key property of the medium employed” (Laurenson 2016). More related to the scope of this master thesis, as stated by Falcão (2010), “For Martinat, conveying the information about the political and cultural references is the essential part of [Brutalismo]. He went as far as showing the work without the computer element or the printer, due to budget restrictions, and instead displayed a model and pre-printed information about brutalist architecture and the Peruvian dictatorship. The artist prefers the work to be shown with the live search and printers, and sees the ‘offline’ presentation as a ‘sketch’ of the artwork, but this underlines how important the dissemination of the information is for him, more than the technological side of the artwork.”
During an e-mail exchange with the artist in November 2019, J.C. Martinat was asked about the changes suffered by the artwork when using stored data. He referred to the artwork as entering a “suspension state”: “it is still the artwork but in a suspension state, [these] signal failures are to me part of the nature of the network.”
When Martinat was asked about how the shift from live to stored data happens and how it is stored, the artist answered: “[The change] is automatic, if the connection is lost it automatically moves to backup mode. Nevertheless, we always monitor the artwork remotely . . . It does [the storage] constantly. I consider register from the artwork all the information generated and stored during the time the artwork was constantly exhibited, since it represents a ‘moment’ of time, I hope at some moment to be able to compare the information generated for example by Brutalismo in 2008 with that that could be extracted from the same artwork in 2025.”
In this case, the use of self-documentation was provided an alternative, stored source of data, similar to the case of Cannula. Different from that case, the artist still considers the result of using the stored data as the artwork.
Ernest Edmonds’s Shaping Space (2000–)
In 2017, Ernest Edmond (b. 1942), considered a pioneer of computer art, was awarded the ACM SIGGRAPH Distinguished Artist Award for Lifetime Achievement in Digital Art, as well as the ACM SIGCHI 2017 Lifetime Achievement Award for the Practice of Computer Human Interaction. His installation Shaping Space (2001–), shown in figure 6, is the development of several processes and installations, as he explains in his interview with Francesca Franco (Edmonds, Franco 2019). Shaping Space algorithmically generates a visual output composed of squares of different sizes and colors, which are calculated from images of the visitors in the gallery space as captured by a video camera.
In the case of Shaping Space, the information stored was not the first data gathered by the system—the images captured by the video camera—but the result of processing those captured images. If the images captured by the camera could be considered a zero-grade input data (as the original, first input to the system), the result of comparing two subsequent frames to detect motion could be considered first-grade data. The result of performing a specific set of calculations over the first-grade data would be the second-grade data, defined as “LifeVectors” inside the artwork’s algorithm.
In this case, the reason for storing the second-grade data was for statistics and algorithm evaluation. Each data grade is the result of applying a set of operations to the previous one and cannot be undone—there is a loss of information that cannot be recovered. It is not possible to obtain the frames (zero grade) from the activity (first grade) nor the activity from the LifeVector (second grade).
When the artwork was created, storing images was too expensive. In addition, the goal of storing information was not to document the complete process but to test and debug the implementation of the algorithm. Therefore, saving a vector of data like the LifeVectors made much more sense at that moment. If instead of storing the LifeVectors the frames captured by the USB camera had been saved, all of the information used by the algorithm could have been available.
From algorithm implementation used for this artwork, a complete model of the algorithm can be obtained. Together with the stored data, re-creating any output resulting from that input could have been possible. If the images had also been available, a new representation the whole artwork, such as one in a virtual reality space, could re-create the artwork but also study how the different forms of interaction were affecting the artwork and in which ways. Alternative implementations of the artwork could also be tested by comparing the outputs of the original and the new implementations starting from the same input. This process could be understood as calibration. Even more, with enough input data and its corresponding output, the algorithm could have been deduced or, in a worst-case scenario, a new algorithm replicating the behavior of the original could be developed using digital signal processing methods, such as Wiener filters or machine learning processes.
When the artist was asked about what would happen to the artwork if it would use only those previously stored values to generate the visual output and not use the camera frames anymore, Edmonds answered: “No. That would be a completely different artwork. The way it interacts is parts of its nature.”
In this case, the use of self-documentation was intended to provide statistical analysis of the performance of the algorithm and not intended for any other use, although it has been shown how it could be used to represent the artwork without an interactive source.
Rafael Lozano-Hemmer’s Pulse Index (2010)
Rafael Lozano-Hemmer (b. 1967) is one of the major international references for software-based art, and he has received two BAFTA Awards for Interactive Art in London and a Golden Nica at the Prix Ars Electronica in Austria. His 2010 artwork Pulse Index, shown in figure 7, is “an interactive installation that records participants’ fingerprints at the same time as it detects their heart rates. The piece displays data for the last 765 and more participants in a stepped display that creates a horizon line of skin. To participate, people put their finger into a custom-made sensor equipped with a 220x digital microscope and a heart rate sensor . . . As more people try the piece one’s own recording travels upwards until it disappears altogether.”
The artwork, as stated on the web page of the project, admits different presentation with up to 10,952 recorded prints being exhibited at the same time. Any time a print is added, the oldest one is deleted from the artwork. In this case, the reason for doing so is to render the artwork itself. The set of images represents an accumulation of data and documentation, but that was not the first intention for the artwork. When asking the artists about the storage of the images, he answered: “[The images] are stored in the hard drive, but a new recording erases the oldest one . . . At the beginning of the project I did store them and used them in an artwork print, which is illegal . . . There was originally an alternative use, which is why I made Performance Review [2013]. But now I think it is stupid to record and store as the strength of the project is the ephemerality.”
When asked if it were possible to reconstruct the artwork in a different environment for documentation purposes, Lozano-Hemmer was clear: “I do have some pieces that can work ‘unplugged.’ I am fine with that so long as you say that to the public. If the interactivity can’t happen then this piece should NOT be exhibited. For this artwork participation is live and direct.”
In this case, the artwork itself is based on an accumulation of fingerprints, which can be understood as self-documentation in order to render the output of the artwork.
Conclusions
In this article, the strategy of self-documentation as a built-in function and some possible uses have been presented, together with potential consequences of its use. This strategy is based on storing external data with the same format that would be used by the system, together with a timestamp that works as a reference of the moment it was captured. The reason to do this is to have a valid dataset in case access to the initially intended one is no longer possible, no matter the reason.
It is an open question as to what happens when the artwork cannot access the intended source of information: if it changes into something different or not and what pros and cons the artwork can face storing that data.
If stored, then it is available as input for the system with different possible purposes. For Canogar, who implements the self-documentation strategy on all of his data artworks, the primary purpose of this data is to work as “demo-mode.” In this case, stored data will be used whenever the original source is unavailable. As such, the artist considers that the artwork has changed into something different as the data is no longer “live.” That makes the artwork “object” turn into “documentation” of the artwork, the closest that documentation could be to the artwork itself. As stated by Van Saaze (2013), “Documentation in contemporary art conservation is regarded as extremely important and can even come to substitute for the physical artwork.” Groys (2002) makes the difference clear when he mentions this: “Nevertheless, to categorize art documentation as ‘simple’ artwork would be to misunderstand it by overlooking its originality, its identifying feature, which is precisely that it documents art rather than presenting it.”
In the case of Jose Carlos Martinat’s Brutalismo, a database of terms is built as a backup when an Internet connection is not available or is unreliable. If the artwork turns to that backup, the artwork enters a “suspension-state” —in the artist’s words, like being assisted by an external device but still being the artwork.
In the case of Ernest Edmonds’s Shaping Space, the stored data was not intended to be used to reconstruct the work in the future but to test the algorithm and provide information about the system and to the system. Nevertheless, that stored content could have been used to build a virtual re-creation of the artwork, test alternative implementations of the algorithm and calibrate them, or even develop a model for the algorithm.
In the case of Rafael Lozano Hemmer’s Pulse Index, the self-documentation strategy provides all of the functionalities commented earlier, but storing the images is fundamental for rendering the artwork itself. Here, the artwork can happen because of that. It is not an added, built-in functionality but at the core of the algorithm itself.
Self-documentation implies extra storage of data, which implies its own conservation problems. It is outside the scope of this article to deal with those problems, although disk images will provide a solution. It would be interesting to consider the possibility of indexing and storing that data as an external database for historical research, as other artworks through history have provided insights on the time they were produced thanks to the content represented in the artworks. Different databases from different artists working in the same time period could provide interesting historical information in some centuries’ time about the artist’s concerns in our current time.
ACKNOWLEDGEMENTS
This article is part of the research developed for my master’s degree in the MediaArtHistories program at Danube University Krems, Austria. I would like to thank Oliver Grau, head of teaching and coordination for MediaArtHistories, and Wendy Coones, chair professor, as representatives of the Image Science Department for their support. Specially, I would like to thank Patricia Falcão for her work as supervisor of this thesis. In addition, I would like to thank all of the artists who kindly provided insights on their works, making them suitable as case studies.
REFERENCES
Brutalism: Stereo Reality Environment 3. Tate Modern, London. Accessed January 2022. https://www.tate.org.uk/art/artworks/martinat-mendoza-brutalism-stereo-reality-environment-3-t13251.
Edmonds, Ernest. Shaping Space. Accessed January 2022. https://www.ernestedmonds.com/www/Art/Recent/RecentShapingSpace.htm.
Edmonds, E., Franco, F. (2019) “Systems theory, systems art and the computer: Ernest Edmonds interviewed by Francesca Franco” Interdisciplinary science reviews, 2017 vol. 42, nos. 1–2, 168–179.
Ensom, Tom. 2018. “Technical narratives: Analysis, description and representation in the conservation of software-based art.” Ph.D. thesis, Faculty of Arts and Humanities, King’s College, London.
Espenschied, D., Rechert, K. (2019). 207.1 Fencing Apparently Infinite Objects. https://doi.org/10.17605/OSF.IO/6F2NM.
Falcão, Patricia. 2010. “Developing a risk assessment tool for the conservation of software based-artworks.” MA thesis, Hochschule der Künste, Bern.
Laurenson, Pip. 2013. “Emerging Institutional Models and Notions of Expertise for the Conservation of Time-Based Media Works of Art.” Technè 37: 36–42.
Laurenson, Pip. 2016. “Old Media, New Media? Significant Difference and the Conservation of Software-Based Art.” In Preserving and Exhibiting Media Art: Challenges and Perspectives. Amsterdam, Netherlands: Amsterdam University Press. 73–96.
Lozano-Hemmer, Rafael. Pulse Index. Accessed January 2022. http://www.lozano-hemmer.com/artworks/pulse_index.php.
Studio Daniel Canogar. Cannula. Accessed January 2022. http://danielcanogar.com/work/cannula.
Van Saaze, V. 2013. Installation Art and the Museum. Amsterdam, Netherlands: Amsterdam University Press.
FURTHER READING
A case study on Daniel Canogar’s Cannula, presenting an in-depth system analysis of the artwork and introducing the use of the self-documentation strategy at Daniel Canogar’s artworks was presented in Youth in Conservation of Cultural Heritage (YOCOCU) 2016 and can be found at https://ge-iic.com/ojs/index.php/revista/article/view/505/790 (Accessed January 2022).
A case study on Ernest Edmonds’s Shaping Space can be found in the journal article, “The Troubling Affair of Born-Digital Art Conservation: Creative Design for Computational Art Documentation” by Francesca Franco, Ernest Edmonds, and Diego Mellado, included in Computational and Cognitive Models of Creative Design, John S. Gero and Mary Lou Maher (eds.), Springer, 2020 presented at the Heron Island Conference Workshop on Computational and Cognitive Model Creative Design in December 2019.
More details on the conservation strategies adopted by Daniel Canogar’s studio can be found in the author’s MA thesis “Software- and data-based art conservation from the artist studio practice: Engineering approaches to documentation as a tool for artwork definition and conservation,” Danube University Krems.
An interesting discussion -but out of the scope of this paper- about the life of the artwork and the artwork as documentation in the scope of biopolitics, can be found in Boris Groys’ article “Art in the Age of Biopolitics: From Artwork to Art Documentation.” In Art Power. Cambridge, MA: MIT Press, pages 53–65.
AUTHOR
Diego Mellado Martínez
Studio Daniel Canogar Project Manager
Madrid, Spain
diego@danielcanogar.com