Tzu-Chuan Lin
Electronic Media Review, Volume Seven: 2021-2022
Introduction
Max (a.k.a. Max/MSP, both of which will be used throughout this article) is a visual programming language, which means that it offers a graphical environment that allows users/programmers to manipulate object blocks freely and visually, by connecting these blocks to create unique programs, sounds, and animations, and so on, and acquire the feedback in real time. Basically, users do not need to write any proper code or have any programming knowledge; one could simply play with the blocks and discover different ways to create. Due to this advantage, Max is widely used by artists, composers, software designers, and so on; therefore, there are many possible creations, such as an interactive or sound installation, robotic control system, or even a computer music piece.
Conservation Issue
Although the software is user-friendly, Max/MSP-based creations can be simple or complicated depending on how a user develops it—for example, it could be either connected (also communicated) with lots of apparatuses, musical instruments, and so forth, or use some built-in functions to create sounds. However, Max/MSP does not have a clear programming structure or source code like the other textual code programming languages, such as C or Python (Farbowitz 2018). This makes it difficult for conservators to understand Max/MSP contents by common programming logic. In addition, as Max/MSP is commercial software and not open source, it complicates conserving specific software-based works of art. For instance, when the work needs to be updated to a newer version of Max/MSP due to conservation purposes, it will require purchasing a new license. This also points to a strong concern regarding dependence on the software supplier in terms of maintaining the work in the long run.
Development History
How did Max/MSP come to be? The earliest version of the software was simply called Max, and began a real-time control system that was developed by Miller Puckette in 1986 at the Institute for Research and Coordination in Acoustics/Music (IRCAM), a French institute dedicated to the research of music and sound (Favreau et al. 1986). Two years later, the graphical environment—Patcher—was created to control the objects in the “Max” system (Puckette 1988); this visualization has become the most important feature of Max/MSP these days. In 1990, the Opcode system launched a commercial version that was developed by David Zicarelli once the Opcode gained the license from IRCAM. Later in 1996, Miller Puckette would have liked to make several improvements in Max, but instead he developed a redesign open-source program called Pure Data. One year later, Zicarelli acquired the publishing rights from Opcode and founded a new company—Cycling ’74—to continue commercial development until today; meanwhile, he released the first “Max” product called Max/MSP, in which MSP was partly adapted from Pure Data to make Max able to deal with the audio signal processing (Puckette 2002). The development has been going on for more than 30 years through different phases (fig. 1); meanwhile, media art was booming thanks to those technological developments. Artists experimented with Max/MSP to create various video and/or sound installations, as well as other things. Some of them have become signature works of artists.
Contemporary museums have been collecting time-based media and software-based artworks over the years. Hence, Max has been involved in the digital art field over time. The works of software-based art using Max/MSP might just currently be on show in an exhibition room or asleep in a storage room of those museums. This was how I encountered two software-based artworks—Border Patrol and Particle Noise—during my internship research courses. I had learned how to use Max/MSP in a previous class, but at this time, discovering these works while training to be a conservator, I became inspired to investigate how these important pieces of software-based art can be safely preserved.
Case Studies of Software-Based Artwork Using Max/MSP
The two case studies will be introduced by the following structure: brief background of the artwork, the installation’s behavior, and risk assessment. The first one is Border Patrol, created by Paul Garrin, who collaborated with David Rokeby for the first Gwangju Biennale in South Korea from 1994 to 1995 (fig. 2). This large video interactive installation is an early-acquired artwork without documentation from the ZKM | Center for Art and Media Karlsruhe, which means that there were no instructions or references to know how the work functioned before ZKM reinvestigated the work with the artist (Heiss, Stricot, and Vlaminck 2021). When Paul Garrin conceived this work, he was dealing with the issue of national security within the surveillance technology; thus, he turned video cameras into sniper guns which were aiming at visitors’ heads and put the cameras on top of a military barrier to form the whole installation, mimicking a border control station. David Rokeby then implemented Garrin’s concept with his own developed software and hardware.
Based on the ZKM’s revisited project material and second test setup, I was able to draw a logical diagram (fig. 3) to describe the behavior of this video interactive installation. There were two types of cameras. One group was color robotic cameras mounted with two motors each that could move and follow visitors. The other one was a group of black-and-white stationary tracking cameras mounted on top of robotic cameras. They functioned as visual sensors and sent the video image to the SCSI interface, which was custom made by Rokeby. This interface analyzed the image of the visitor’s shape in terms of pixel change and found the highest pixel to calculate the zone of the face. Then, it provided this data information to Max/MSP running on the Macintosh computer to control the positions of robotic motors to achieve cameras tracking visitors. Meanwhile, the Max/MSP was sending MIDI signals to trigger the digital sampler and Amiga computers; once both of them received MIDI signals, they gave the corresponding sound of gunshots and the graphics of crosshairs. The Amiga-generated graphics were overlaid with the image captured from the color cameras through a Genlock device and were displayed in real time on the embedded CRT monitors. This installation had two identical setups; figure 3 shows a simplified version.
To assess the risk for components, I adapted the method from the European research project Inside Installation (Brokerhof et al. 2011), which I learned from the ZKM’s digital conservation team. Figure 4 lists all components of the work to estimate the expected loss and how much would be possible to recover. The reason for listing these tangible components is because the work relies on them to function properly; without them, the work would lose its integrity. Furthermore, the advantage of the presentation in figure 4 is that one can visually examine the balance between the loss and recoverability during the decision-making process for what to conserve.
There are three elements shown in figure 4: Probability (red), Consequence (orange), and Recovery (green). Each element is given a score from 0 to 5, with 5 being the maximum score. For instance, at first glance, the custom-made SCSI interface is at the highest risk due to zero possibility for recovery and device failure probability at 5. This specific device was made by David Rokeby; currently, there are no spares or schematics of it at the ZKM for restoration or reproduction (fig. 5). The second risky component is the Amiga Genlock device: the probability is low, but the score of consequence is 5. This means that once the device fails, the overlaid image cannot be achieved. Furthermore, the recovery only received a score of 1 because this device is designed especially for Amiga computers and the NTSC standard for analog videos. However, the PAL standard has been used instead in most EU countries. Therefore, the same device can rarely be found in Europe. Here, I have pointed out some potential risks from the diagram to demonstrate the assessment of this method, and later it will be revisited to discuss conservation strategies.
The second case study is Particle Noise created by Carsten Nicolai for his solo exhibition in London in 2013. Nicolai is often shifting unnoticeable phenomena on a perceptible level through his installations with assistance from scientific devices. For instance, he used the principle of the Geiger counter for this work to detect invisible radiation particles in the surrounding environment where the work was installed, and then to turn them into an audible sound (Krystof 2019). This sound installation was acquired by the Kunstsammlung Nordrhein-Westfalen after the artist’s solo exhibition in 2019. Later on, it was installed again in a different exhibition room at the Kunstsammlung Nordrhein-Westfalen (K21) to integrate their collection presentation (fig. 6).
As aforementioned, the Geiger counter—in this case, one analog and one digital Geiger counter—detected and measured the radioactive particles inside the exhibition room. The measured data was sent to the Max/MSP running on the Mac mini. When Max/MSP received these data, it triggered a sine wave generator or a radio receiver to generate sounds. Thus, these sounds were played through four loudspeakers to present a real-time soundscape to visitors. At the same time, Max/MSP visualized the data measurement process into a user interface graphic and presented it on a 4:3 ratio flat screen, which was connected to the Mac mini and mounted on top of the black display box. Figure 7 shows the whole data process and signal outputs with the physical components.
These components of Particle Noise were assessed with the same risk assessment method as Border Patrol (fig. 8). Similar to the SCSI interface, the custom-made digital Geiger counter received a score of 0 within the recovery due to no spares or detailed information. Once it fails, the sound will be changed and will not be able to function properly; therefore, the consequence score is 4. Another high-risk component is the analog Geiger counter, which is a significant component of the work because the device also serves as a sculptural component. Furthermore, this Geiger counter is an antique device, and it is hard to predict when it might fail; therefore, the probability of this device is higher than the other components. On the opposite end, the recovery is a score of 1. This is because, although the same model of this device can occasionally be found online (e.g., eBay), it is uncertain whether the device is functioning well.
Comparison/Analysis of Two Case Studies with the Conservation Strategies
Next, I applied three common conservation strategies in the practice—Storage, Migration, and Emulation (Huber 2013; Ratti 2013; Rinehart and Ippolito 2014)—to analyze and compare these two case studies. In the beginning, I was curious whether there were any similarities during the conservation practice because they were used with the same software—Max/MSP, although they were in different versions. Therefore, by analyzing these two artworks and comparing them to discover their similarities, it is my hope to either conserve them in the same way or, if they are too different, find another suitable way to conserve them.
Storage
Originally, there were four Amiga computers in use in Border Patrol, but due to bad conditions such as CMOS battery acid leaking, three hard drives were unreadable. After the inventory and emergency restoration, two functional Amigas were recovered; the components of the remaining Amiga computers were stored as spares. Furthermore, as a center with a long history in media art, ZKM has collected lots of spares for repair and replacement, except for some specific devices as aforementioned in the risk assessment section: the Genlock device, SCSI interface, and so on. The digital preservation copies—disk images—from all the obsolete data carriers (hard drives and floppy disks) were made and stored in the digital repository after the initial inventory and inspection.
Particle Noise was exhibited in different countries before it was acquired by the Kunstsammlung Nordrhein-Westfalen; thus, it had several iteration differences, such as the arrangement and speaker amount. When the artist delivered the work, eight speakers were provided. The current iteration is with four speakers; the rest of them therefore are stored as spares. The spare preparation for the other physical components, such as analog and digital Geiger counters, will be confirmed with the artist (fig. 9). After Particle Noise was installed for the K21 collection exhibition in 2021, the updated software version—due to the iteration setup—was provided by the artist’s assistant, which included the Max/MSP project file, related multimedia files, JavaScript, and so forth. The whole package has been stored in the museum’s digital repository, and the checksums were added for the integrity verification. To preserve the proper setting on the computer, and prevent failure, it is recommended to create a disk image of the SSD.
Migration
Some parts of Border Patrol’s hardware could be easier to migrate than others. For instance, the artist could accept replacing CRT monitors with newer LCD monitors; however, this replacement will cause the loss of the work’s authenticity, specifically with the image quality and historicity 4:3 format. Furthermore, some hardware connectors are no longer in use due to the obsolete analog video technology; hence, the interconnection between hardware must be taken into account when migrating them. As I pointed out in the Border Patrol’s risk assessment, there is no solution to migrate or even reproduce the custom-made SCSI interface. Besides the hardware, as the disk images were made, contents could be transferred to run on more reliable up-to-date carriers. Moreover, the third version of Max/MSP project file could be opened in the current version “Max 8”; however, it showed a bunch of “no such object” messages, which means that a lot of objects (functions) are either custom made or are not supported anymore (fig. 10). Thus, Max/MSP of the work will need to be recompiled with up-to-date objects to achieve the initial functions. In short, when ZKM proceeds to migrate the whole installation, it will involve several significant changes in this artwork; then, it might be considered a reconstruction or reinterpretation.
However, Particle Noise uses the current version of Max/MSP, and the operating system of the Mac mini (macOS Big Sur) is still maintained by Apple Inc.; hence, the whole system could still function for three to five years. However, the Mac mini was produced in 2014, and soon it will not be able to match the compatibility of the newer macOS requirement and will then become obsolete. Due to Max/MSP being commercial software, a new license key will need to be purchased when the current version migrates to a newer version. Thus, it is necessary to discuss with the artist whether and how to migrate the work in the future.
Emulation
Comparing these two case studies, Border Patrol can be considered a more hardware-based artwork. One main reason is that two artists created and designed custom-made hardware and software to achieve their goal by stretching the technology at the time because there were no such technologies—like face tracking—they could have used. That is also the reason that this installation needed to connect with various hardware to function properly. In addition, the function of analyzing images was based on the SCSI interface; this created an operational dependency on this device. Moreover, the SCSI connection is now obsolete. How to connect the SCSI interface with an up-to-date computer that runs the emulator and functions well is an issue. Another big issue is that two different kinds of computers, Macintosh and Amiga, were used to serve for the whole installation. It will require at least two different emulators to emulate each computer; thus, a potential problem arises: How can these emulators communicate with each other just like hardware, or is this even possible?
By contrast, Particle Noise requires only one emulator to achieve its functionality from Max/MSP. Once the disk image of the Mac mini’s SSD is made, it will offer an opportunity to emulate the whole software environment of the work. However, the license of macOS is limited to products from Apple Inc., which means that the disk image cannot be transferred to other non-Apple hardware. Furthermore, tests are needed to determine whether the transferred disk image could directly run on newer computers without any connection issues between other devices.
Further development depends on the two museums’ decisions. For instance, ZKM will follow their strategy to make a media archaeological reconstruction with original hardware and software on one side of Border Patrol. This will allow visitors to appreciate and experience the work with its original essence. Meanwhile, ZKM will also create an updated version for the other side; to achieve this goal, the solution would be to integrate the whole system—hardware and software—into a software run on one computer to avoid the combination of different emulators and the hardware-based interface. However, Kunstsammlung Nordrhein-Westfalen has monitored Particle Noise and the technologies it uses while it is still exhibited in-house. The next step for the museum is to interview the artist, Carsten Nicolai, and his assistant to discuss conservation issues. After this whole analysis process, I understood that although they used the same software, Max/MSP, they were in different spectrums due to the rapid development of technology. Thus, the conservation strategy and concept must be different for each piece.
To conclude, the case study on Border Patrol demonstrates that without the documentation, it will be hard to understand how it functions properly and how to install/deinstall the piece without issues. Therefore, it would be wise to compile documentation while the work is being installed or during the test setup before the museum’s acquisition phase; this installation process would provide an opportunity to examine the artwork, and to bring artists and their teams together into the discussion about concerns of reinstallation and conservation.
Findings and Further Discussion
While I examined the Max/MSP patchers from these two case studies, I made four discoveries (fig. 11). First, the current operating and maintenance company, Cycling ’74, only offers the oldest version up to Max 4.1. Due to the publishing rights transfer, the earlier versions from Opcode require a pack of installation floppy disks to be installed and used. Second, the Max/MSP project file extension can be different. For example, the files from Border Patrol are “.NY” and “.korea” (named after different places where the work was exhibited), but Particle Noise is “.maxpat”; the latter is easy to identify at first glance. The reason is that the old Macintosh OS allowed users to change file extensions as they wanted. Third, one Max/MSP project file (patcher, the graphic environment) might contain a lot of subpatchers (like the case Border Patrol), other source code such as JavaScript (like Particle Noise), or even connections to other software.
Last but not least, the up-to-date Max/MSP offers more user-friendly guidance. For instance, it will give a hint where one should connect two blocks, or use a particular color dotted line to identify different signal flows. Furthermore, the help function could demonstrate how specific objects work within a Max/MSP project file to assist those examining it.
These discoveries have raised my curiosity further to investigate a method to document the Max/MSP patcher environment of artwork so that the documentation can be enhanced for Max/MSP-based works of art. A way to achieve this method might be similar to source code analysis for other software-based works, or a way to follow signal processes to detect the behavior of works. Hopefully, these efforts could benefit those who are searching for a way to document the Max/MSP-based works of art and contribute to the time-based media conservation field.
ACKNOWLEDGMENTS
I would like to thank the following individuals for their mentorship during this research:
Dr. Morgane Stricot, media and digital art conservator, head of digital conservation, ZKM | Center for Art and Media Karlsruhe; Matthieu Vlaminck, media and digital art conservator, ZKM | Center for Art and Media Karlsruhe; Dr. Nina Quabeck, head of conservation, Kunstsammlung Nordrhein-Wastfalen; Rea Grammatikopoulou, time-based media conservator, Kunstsammlung Nordrhein-Wastfalen; Professor Johannes Gfeller, former head of master program, Conservation of New Media and Digital Information, State Academy of Fine Arts Stuttgart; and Professor Nadja Wallaszkovits, head of master program, Conservation of New Media and Digital Information, State Academy of Fine Arts Stuttgart.
REFERENCES
Brokerhof, Agnes W., Tatja Scholte, Bart Ankersmit, Gaby Wijers, and Simone Vermaat. 2011. “Installation Art Subjected to Risk Assessment: Jeffrey Shaw’s Revolution as Case Study.” In Inside Installations: Theory and Practice in the Care of Complex Artworks. Edited by Tatja Scholte and Glenn Wharton. Amsterdam: Amsterdam University Press. 91–101.
Farbowitz, Jonathan. 2018. “Archiving Computer-Based Artworks.” Electronic Media Review 5. Available at https://resources.culturalheritage.org/emg-review/volume-5-2017-2018/farbowitz/ (accessed 06/15/23).
Favreau, Emmanuel, Michel Fingerhut, Olivier Koechlin, Patrick Potacsek, Miller S. Puckette, and Robert Rowe. 1986. “Software Developments for the 4X Real-Time System.” Proceedings of the 1986 International Computer Music Conference (ICMC ’86). The Hague, Netherlands. 372–373.
Heiss, Daniel, Matthieu Vlaminck, and Morgane Stricot. 2021. “Media Archaeological Reconstruction of Media and Digital Artworks: Practical Case-Studies.” In Technological Arts Preservation, edited byCemal Yılmaz, Osman Serhat Karamanm, and Selçuk Artut. Istanbul: Sabanci University Sakip Sabanci Museum. 188–235.
Huber, Hans Dieter. 2013. “From New Media to Old Media: Ambiguous Concepts, Complex Problems, and Open Questions.” In Digital Art Conservation: Theory and Practice—The Project Digital Art Conservation, edited by Bernhard Serexhe. Vienna: AMBRA V. 137–44.
Krystof, Doris. 2019. “The Exhibition as a Film Set, Sound Installation, Wunderkammer, Audiobook, and Experimental Laboratory.” In Carsten Nicolai: Parallax Symmetry, edited by Susanne Gaensheimer and Doris Krystof. Leipzig, Germany: Spector Books. 39–44.
Puckette, Miller. 1988. “The Patcher.” Proceedings of the 1988 International Computer Music Conference (ICMC ’88). Cologne, Germany. 420–429.
Puckette, Miller. 2002. “Max at Seventeen.” Computer Music Journal 26 (4): 31–43.
Ratti, Iolanda. 2013. “The Specificity of the Video Installation.” In Ephemeral Monuments: History and Conservation of Installation Art, edited by Barbara Ferriani and Marina Pugliese. Los Angeles: Getty Conservation Institute. 130–61.
Rinehart, Richard, and Jon Ippolito. 2014. Re-collection: Art, New Media, and Social Memory. Cambridge, MA: MIT Press.
AUTHOR
Tzu-Chuan Lin
Postgraduate student
State Academy of Fine Arts Stuttgart
Stuttgart, Germany
chuan.sma@gmail.com