Claudia Roeck
The Electronic Media Review, Volume Five: 2017-2018
ABSTRACT
This paper focuses on the graphics pipeline of Horizons (2008), a software-based artwork by Dutch media artist Geert Mul. The graphics pipeline is a chain of software and hardware tools a computer needs to render graphics efficiently. It can be very specific for video games or software-based artworks that make use of intensive, real-time graphics rendering and has an impact on the preservation strategy.
Horizons is dependent on a specific video card, which is a problem for standard central processing unit emulation approaches. This paper describes a method to access the video card through paravirtualization without creating a dependency on a specific video card. This enables enough performance to run Horizons smoothly. While the sound quality was not satisfying, it was a proof of concept for the video graphics.
INTRODUCTION
This investigation is part of broader research whose goal is to compare preservation strategies for software-based art. However, this paper will zoom in the emulation of the graphics rendering of a software-based artwork without comparing all of the possible preservation strategies. It will investigate how and whether an artwork with intensive graphics rendering can be emulated by means of the case study artwork Horizons (2008). This investigation is based on two papers about emulation (Rechert, Ensom, and Falcao 2016) and virtualization (Falcao, Ashe, and Jones 2014).
Computer-rendered real-time graphics are quite common in software-based art. Artists may use video game software to produce video games for their simulations or interactive animations, such as for Sow Farm (2009) by John Gerrard (b. 1974) or Olympia (2016) by David Claerbout (b. 1969). Other artists and their collaborators produce the software themselves, as did Geert Mul (b. 1965) and his programmer Carlo Prelz for Horizons (2008).
Horizons (2008) by Geert Mul has a typical setup for a computer-based artwork. The artwork receives user input from a sensor; its dedicated computer generates a video by processing image sources and the sensor input and then outputs the video on video projectors. When analyzing the work, it turned out that its graphics rendering was video card dependent, which means that changing the video card makes the work dysfunctional. Emulators of personal computers usually do not emulate specific video cards. Nevertheless, as emulation is a promissing preservation strategy in the long term, it was a goal of this research to find an emulation or virtualization solution in spite of the video card dependency. Owing to the classical technical setup of the artwork, it should be possible to generalize the findings of this research for artworks with a corresponding setup.
GENERAL DESCRIPTION OF HORIZONS
The artwork Horizons by the Dutch artist Geert Mul is an interactive multimedia installation that was commissioned by the Museum Boijmans Van Beuningen in Rotterdam, the Netherlands in 2008. The work consists of reproductions of landscape paintings from the Boijmans Van Beuningen collection in an interactive installation. The horizons of these digitized landscape paintings are all aligned to the same height. A computer program generates a video of the digitized paintings, which is projected to the wall by three digital video projectors (fig. 1). Visitors walking around the space cause the image to split where they stand, revealing a new painting underneath. This interaction is enabled by a tracking sensor that sends the visitor’s movement data to the computer. The sound pans seamless between right and left speaker depending on visitor movement and increases with the number of visitors. For the artist it is important that the movements in the video are smooth.1 The quality of the sound is also important to Geert Mul, although the sound ressembles a pink noise.
GRAPHICS RENDERING OF HORIZONS
In a real-time generated video, as it is the case for Horizons, the video is not only playing back but it is generated on the fly. Real-time means that there is an immediate response to an event, such as the tracking sensor input. Real-time rendering is typical for video games but also for interactive media artworks. It is computation intensive and must be fast. The central processing unit (CPU) of the computer is usually too slow for that. The video card is much better suited for video rendering, as it is optimized for this purpose. Furthermore, specific software that supports graphics rendering is usually installed. This so-called graphics chain boosts rendering performance but can be a challenge for preservation, as the video card and graphics software often depend on each other.
Horizons is programmed in C and Ruby and installed within a Debian 7 operating system (Horizons version 2013). The source code analysis showed that the programmer of Horizons used intermediate hardware abstraction layers, such as SDL and OpenGL, in order to generalize the underlying hardware (fig. 2). These hardware abstraction layers are the precondition to make emulation possible. Nevertheless, the model of the video card was hard-coded in one of the Ruby programs and in the configuration file of the display manager.
GPU VIRTUALIZATION
The original goal was not to make any changes to the software and the software environment of Horizons. This was tested with a CPU emulation that is completely independent from the hardware [2]. The emulator used for this emulation is called “qemu” and the computer was a Thinkpad X1 Carbon (Intel Core i7-6500U 2.50 GHz and 8GB RAM). However, it turned out that video in the emulation was stuttering. Not only did the emulation consume computing power but it was also not able to access the video card, which was further reducing rendering capacity.
As a next step, CPU virtualization was tested. It enables direct access to the CPU; therefore, it is faster. This was carried out with the same tool, qemu, but with a different command. CPU virtualization was not effective enough to play back the video smoothly. The CPU virtualization also has the disadvantage that this virtualization runs only on computers with the same type of CPU. Nevertheless, it is still independent from the computer hardware to a certain extent.
To accelerate the rendering, it would be helpful to have a method to access the video card. It would be even better if this method did not depend on a specific video card. Video card access can be achieved either by a GPU pass-through or by paravirtualization (GPU = Graphics Processing Unit = video card). These two methods are described in Suzuki et al. (2014) and Dowty and Sugerman (2009).
GPU pass-throughs are common in cloud-computing environments.With a GPU pass-through, the original vendor driver of the video card is installed in the guest system that, however, ties the emulated instance again to a specific video card. This is the reason why this procedure was not tested.
With the GPU paravirtualization, a generic virtual GPU (virtual video card) is installed within the guest system and the adaptation to the host’s GPU hardware is delegated to the emulator implementation (fig. 3). This functionality was recently added to qemu, the emulator used for Horizons. Due to its independence from the video card, this solution was preferred and tested.
Fig. 3. Principle of paravirtualization
On Linux systems, this virtual video driver has been available since kernel version 4.4. As the Linux kernel of Horizons was 2.6, the kernel had to be updated. This was carried out by updating the whole operating system from Debian 7 to Debian 9, which can be seen as a migration of Horizons (fig. 4).
Fig. 4. Paravirtualization of Horizons (2008) by first migrating (upgrading) the piece to a current operating system version and then emulating the migrated version.
The paravirtualization yielded satisfying results regarding the rendering of the video. The video patterns moved smoothly. When directly comparing it with Horizons on the artist’s computer, it became evident that the paravirtualization ran slightly faster. However, without direct comparison, it was not noticeable. The sound was an issue. The audio rendering within the virtualization did not work well enough. There was a crackling in the pink noise that was not intended and is not audible on the artist’s computer. A test recording of the sound within the emulation provided good quality. Although testing was performed on different audio device drivers, the problem could not be solved.
CONCLUSIONS
The paravirtualization tests yielded promising results regarding video rendering, while the audio rendering was not satisfying. However, when applying this strategy, one has to be aware of its risks and benefits. Paravirtualization is still hardware (CPU type) dependent and can cause changes in the artwork owing to the necessity to upgrade the operating system kernel. The emulator and its libraries for video card access and virtual video card driver in the guest system need to be aligned. Therefore, this strategy can be successful in the midterm as long as the new hardware does not have enough speed advantage to enable a full-system emulation. Furthermore, virtualization allows the identification of virtualization effects and hardware dependencies and serves as a sandbox for migration.
ACKNOWLEDGMENTS
This research was funded as part of New Approaches in the Conservation of Contemporary Art (NACCA, www.nacca.eu), a Marie Skłodowska-Curie Innovative Training Network of the European Union. I thank my PhD supervisors, Dr. Julia Noordegraaf, University of Amsterdam, and Dr. Klaus Rechert, University of Freiburg. LIMA Amsterdam (www.li-ma.nl) provided the infrastructure and secondment placement for this research; Geert Mul made his artwork available, including hardware and software; and Teal Gaure provided programming support.
ENDNOTES
1. Oral communication with Geert Mul during the installation of Shan Shui between October 28 and November 4, 2016.
2. The encoding of the video card model in the software had to be replaced by an automated recognition of the video card.
REFERENCES
Dowty, M., and J. Sugerman. 2009. GPU virtualization on VMware’s hosted I/O architecture. SIGOPS Operating Systems Review 43 (3): 73.
Falcao, P., A. Ashe, and B. Jones. 2014. Virtualisation as a Tool for the Conservation of
Software-Based Artworks. https://pdfs.semanticscholar.org/38a1/ef01599accfcda2f17767fc299fed98880bb.pdf (accessed 07/09/2019).
Rechert, K., T. Ensom, and P. Falcao. 2016. Introduction to an emulation-based preservation strategy for software-based artworks. www.tate.org.uk/research/publications/emulation-based-preservation-strategy-for-software-based-artworks (accessed 07/11/2019).
Suzuki, Y., S. Kato, H. Yamada, and K. Kono. 2014. GPUvm: Why Not Virtualizing GPUs at the Hypervisor? 2014 USENIX Annual Technical Conference. www.usenix.org/conference/atc14/technical-sessions/presentation/suzuki (accessed 07/11/2019).
CONTACT
Claudia Roeck
Time-based Media Conservator and PhD candidate
University of Amsterdam, NL
claudia.roeck@crockodile.ch