{"id":1641,"date":"2019-08-23T14:11:06","date_gmt":"2019-08-23T18:11:06","guid":{"rendered":"http:\/\/resources.culturalheritage.org\/emg-review\/?page_id=1641"},"modified":"2019-08-23T14:11:06","modified_gmt":"2019-08-23T18:11:06","slug":"roeck","status":"publish","type":"page","link":"https:\/\/resources.culturalheritage.org\/emg-review\/volume-5-2017-2018\/roeck\/","title":{"rendered":"Emulating Horizons (2008) by Geert Mul: The Challenges of Intensive Graphics Rendering"},"content":{"rendered":"\n

Claudia Roeck
The Electronic Media Review, Volume Five: 2017-2018<\/em><\/p>\n\n\n\n

ABSTRACT<\/h2>\n\n\n\n

This paper focuses on the\ngraphics pipeline of Horizons<\/em> (2008),\na software-based artwork by Dutch media artist Geert Mul. The graphics pipeline\nis a chain of software and hardware tools a computer needs to render graphics efficiently. It can be very specific for video games or\nsoftware-based artworks that make use of intensive, real-time graphics\nrendering and has an impact on the preservation strategy.<\/p>\n\n\n\n

Horizons<\/em> is dependent on a specific video\ncard, which is a problem for standard central processing\nunit emulation approaches.\nThis paper describes a method to access the video card through\nparavirtualization without creating a dependency on a specific video card. This\nenables enough performance to run Horizons<\/em>\nsmoothly. While the sound quality was not\nsatisfying,\nit was a proof of concept for the video graphics.<\/p>\n\n\n\n

INTRODUCTION<\/h2>\n\n\n\n

This investigation is part of broader research whose\ngoal is to compare preservation strategies for software-based art. However,\nthis paper will zoom in the emulation of the graphics rendering of a\nsoftware-based artwork without comparing all of the possible preservation\nstrategies. It will investigate how and whether an artwork with intensive\ngraphics rendering can be emulated by means of the case study artwork Horizons<\/em> (2008). This investigation is\nbased on two papers about emulation (Rechert, Ensom, and Falcao 2016) and\nvirtualization (Falcao, Ashe, and Jones 2014). <\/p>\n\n\n\n

Computer-rendered real-time graphics are quite common in software-based art.\nArtists may use video game software to produce video games for their\nsimulations or interactive animations, such as for Sow Farm<\/em> (2009) by John Gerrard (b. 1974) or Olympia<\/em> (2016) by David Claerbout (b. 1969). Other artists and\ntheir collaborators produce the software themselves, as did Geert Mul (b. 1965)\nand his programmer Carlo Prelz for Horizons<\/em>\n(2008). <\/p>\n\n\n\n

Horizons<\/em> (2008) by Geert Mul has\na typical setup for a computer-based artwork. The\nartwork receives user input from a sensor; its dedicated computer generates a\nvideo by processing image sources and the sensor input and\nthen outputs the video on video projectors. When analyzing the work, it turned out that its graphics rendering\nwas video card dependent, which means that changing the video card makes the\nwork dysfunctional. Emulators of\npersonal computers usually do not emulate specific video cards. Nevertheless, as emulation is a promissing\npreservation strategy in the long term, it was a goal of this research to find\nan emulation or virtualization solution in spite of the video card dependency.\nOwing to the classical technical setup of the artwork, it should be\npossible to generalize the findings of this research for artworks with a\ncorresponding setup.<\/p>\n\n\n\n

GENERAL\nDESCRIPTION OF HORIZONS<\/em><\/h2>\n\n\n\n

The artwork Horizons<\/em> by the Dutch artist Geert Mul is an interactive multimedia installation that was commissioned by the Museum Boijmans Van Beuningen in Rotterdam, the Netherlands in 2008. The work consists of reproductions of landscape paintings from the Boijmans Van Beuningen collection in an interactive installation. The horizons of these digitized landscape paintings are all aligned to the same height. A computer program generates a video of the digitized paintings, which is projected to the wall by three digital video projectors (fig. 1). Visitors walking around the space cause the image to split where they stand, revealing a new painting underneath. This interaction is enabled by a tracking sensor that sends the visitor\u2019s movement data to the computer. The sound pans seamless between right and left speaker depending on visitor movement and increases with the number of visitors. For the artist it is important that the movements in the video are smooth.1<\/sup> The quality of the sound is also important to Geert Mul, although the sound ressembles a pink noise. <\/p>\n\n\n\n

\"Fig
Fig 1. Wiring diagram of Horizons<\/em> (2008)
<\/figcaption><\/figure>\n\n\n\n

GRAPHICS\nRENDERING OF HORIZONS<\/em><\/h2>\n\n\n\n

In a real-time generated video, as it is the case for Horizons<\/em>, the video is not only playing\nback but it is generated on the fly. Real-time<\/em>\nmeans that there is an immediate response to an event, such as the tracking\nsensor input. Real-time rendering is typical for video games but also for interactive media artworks. It is\ncomputation intensive and must be fast. The central\nprocessing unit (CPU) of the computer is usually too slow for that. The\nvideo card is much better suited for video rendering, as it is optimized for this\npurpose. Furthermore, specific software\nthat supports graphics rendering is usually installed. This so-called graphi<\/em>c<\/em>s chain<\/em> boosts rendering performance but can be a challenge\nfor preservation, as the video card and graphics software often depend on each other.<\/p>\n\n\n\n

Horizons<\/em> is programmed in C and Ruby and installed within a Debian 7 operating system (Horizons<\/em> version 2013). The source code analysis showed that the programmer of Horizons<\/em> used intermediate hardware abstraction layers, such as SDL and OpenGL, in order to generalize the underlying hardware (fig. 2). These hardware abstraction layers are the precondition to make emulation possible. Nevertheless, the model of the video card was hard-coded in one of the Ruby programs and in the configuration file of the display manager.<\/p>\n\n\n\n

\"
Fig. 2. Hardware abstraction of Horizons(2008)<\/em>
<\/figcaption><\/figure>\n\n\n\n

GPU\nVIRTUALIZATION<\/h2>\n\n\n\n

The\noriginal goal was not to make any changes to the software and the software\nenvironment of Horizons<\/em>. This was\ntested with a CPU emulation that is completely independent from the hardware\n[2]. The emulator used for this emulation is called \u201cqemu\u201d and the computer was\na Thinkpad X1 Carbon (Intel Core i7-6500U 2.50 GHz and 8GB RAM). However, it\nturned out that video in the emulation was stuttering. Not only did the\nemulation consume computing power but it was also not able to access the video\ncard, which was further reducing rendering capacity. <\/p>\n\n\n\n

As\na next step, CPU virtualization was tested. It enables direct access to the\nCPU; therefore, it is faster. This was carried out with the same tool, qemu,\nbut with a different command. CPU virtualization was not effective enough to\nplay back the video smoothly. The CPU virtualization also has the disadvantage\nthat this virtualization runs only on computers with the same type of CPU.\nNevertheless, it is still independent from the computer hardware to a certain\nextent.<\/p>\n\n\n\n

To\naccelerate the rendering, it would be helpful to have a method to access the\nvideo card. It would be even better if this method did not depend on a specific\nvideo card. Video card access can be achieved either by a GPU pass-through or\nby paravirtualization (GPU = Graphics Processing Unit = video card). These two\nmethods are described in Suzuki et al.\n(2014)\nand Dowty and Sugerman (2009). <\/p>\n\n\n\n

GPU\npass-throughs are common in cloud-computing environments.With a GPU\npass-through, the original vendor driver of the video card is installed in the guest system that, however, ties\nthe emulated instance again to a specific video card. This is the reason why this procedure was not tested. <\/p>\n\n\n\n

With the GPU paravirtualization, a generic virtual GPU (virtual video card) is installed within the guest system and the adaptation to the host\u2019s GPU hardware is delegated to the emulator implementation (fig. 3). This functionality was recently added to qemu, the emulator used for Horizons<\/em>. Due to its independence from the video card, this solution was preferred and tested.<\/p>\n\n\n\n

\"Fig.

Fig. 3. Principle of paravirtualization<\/em>
<\/figcaption><\/figure>\n\n\n\n

On Linux systems, this virtual video driver has been available since kernel version 4.4. As the Linux kernel of Horizons<\/em> was 2.6, the kernel had to be updated. This was carried out by updating the whole operating system from Debian 7 to Debian 9, which can be seen as a migration of Horizons<\/em> (fig. 4). <\/p>\n\n\n\n

\"Fig.

Fig. 4. Paravirtualization of Horizons (2008) by first migrating (upgrading) the piece to a current operating system version and then emulating the migrated version. <\/em> <\/figcaption><\/figure>\n\n\n\n

The paravirtualization yielded satisfying results\nregarding the rendering of the video. The video patterns moved smoothly. When\ndirectly comparing it with Horizons<\/em>\non the artist\u2019s computer, it became evident that the paravirtualization ran\nslightly faster. However, without direct comparison, it was not noticeable. The\nsound was an issue. The audio rendering within the virtualization did not work\nwell enough. There was a crackling in the pink noise that was not intended and\nis not audible on the artist\u2019s computer. A test recording of the sound within\nthe emulation provided good quality. Although testing was performed on\ndifferent audio device drivers, the problem could not be solved.<\/p>\n\n\n\n

CONCLUSIONS<\/h2>\n\n\n\n

The paravirtualization tests yielded promising results regarding video\nrendering, while the audio rendering was not satisfying. However, when applying\nthis strategy, one has to be aware of its risks and benefits.\nParavirtualization is still hardware (CPU type) dependent and can cause changes\nin the artwork owing to the necessity to upgrade the operating system kernel.\nThe emulator and its libraries for video card access and virtual video card\ndriver in the guest system need to be aligned. Therefore, this strategy can be\nsuccessful in the midterm as long as the new hardware does not have enough\nspeed advantage to enable a full-system emulation. Furthermore, virtualization\nallows the identification of virtualization effects and hardware dependencies\nand serves as a sandbox for migration.<\/p>\n\n\n\n

ACKNOWLEDGMENTS<\/h2>\n\n\n\n

This research was funded as part of New Approaches in the Conservation\nof Contemporary Art (NACCA, www.nacca.eu), a Marie Sk\u0142odowska-Curie Innovative\nTraining Network of the European Union. I thank my PhD supervisors, Dr. Julia\nNoordegraaf, University of Amsterdam, and Dr. Klaus Rechert, University of\nFreiburg. LIMA Amsterdam (www.li-ma.nl) provided the infrastructure and\nsecondment placement for this research; Geert Mul made his artwork available,\nincluding hardware and software; and Teal Gaure provided programming support.<\/p>\n\n\n\n

ENDNOTES<\/h2>\n\n\n\n

1. Oral communication with Geert Mul during the installation of Shan Shui<\/em> between October 28 and November 4, 2016.<\/p>\n\n\n\n

2. The encoding of the video card model in the software had to be replaced by an automated recognition of the video card.<\/p>\n\n\n\n

REFERENCES<\/h2>\n\n\n\n

Dowty,\nM., and J. Sugerman. 2009. GPU virtualization on VMware’s hosted I\/O\narchitecture. SIGOPS Operating Systems Review 43 (3):\n73. <\/p>\n\n\n\n

Falcao, P., A. Ashe, and B.\nJones. 2014. Virtualisation as a Tool for the Conservation of<\/a><\/p>\n\n\n\n

Software-Based\nArtworks. https:\/\/pdfs.semanticscholar.org\/38a1\/ef01599accfcda2f17767fc299fed98880bb.pdf<\/a>\n(accessed 07\/09\/2019). <\/p>\n\n\n\n

Rechert, K., T. Ensom, and P. Falcao. <\/a>2016.\nIntroduction to an emulation-based preservation strategy for software-based\nartworks. www.tate.org.uk\/research\/publications\/emulation-based-preservation-strategy-for-software-based-artworks<\/a> (accessed\n07\/11\/2019).<\/p>\n\n\n\n

Suzuki, Y., S. Kato, H. Yamada,\nand K. Kono. 2014. GPUvm: Why Not Virtualizing GPUs at the Hypervisor? <\/a>2014\nUSENIX Annual Technical Conference. www.usenix.org\/conference\/atc14\/technical-sessions\/presentation\/suzuki<\/a> (accessed 07\/11\/2019).<\/p>\n\n\n\n

CONTACT<\/h2>\n\n\n\n

Claudia Roeck
Time-based Media Conservator and PhD candidate
University of Amsterdam, NL
claudia.roeck@crockodile.ch<\/p>\n","protected":false},"excerpt":{"rendered":"

Claudia Roeck The Electronic Media Review, Volume Five: 2017-2018 ABSTRACT This paper focuses on the graphics pipeline of Horizons (2008), a software-based artwork by Dutch media artist Geert Mul. The graphics pipeline is a chain of software and hardware tools a computer needs to render graphics efficiently. It can be very specific for video games … Continue reading “Emulating Horizons (2008) by Geert Mul: The Challenges of Intensive Graphics Rendering”<\/span><\/a><\/p>\n","protected":false},"author":37,"featured_media":0,"parent":618,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-1641","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/resources.culturalheritage.org\/emg-review\/wp-json\/wp\/v2\/pages\/1641","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/resources.culturalheritage.org\/emg-review\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/resources.culturalheritage.org\/emg-review\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/resources.culturalheritage.org\/emg-review\/wp-json\/wp\/v2\/users\/37"}],"replies":[{"embeddable":true,"href":"https:\/\/resources.culturalheritage.org\/emg-review\/wp-json\/wp\/v2\/comments?post=1641"}],"version-history":[{"count":0,"href":"https:\/\/resources.culturalheritage.org\/emg-review\/wp-json\/wp\/v2\/pages\/1641\/revisions"}],"up":[{"embeddable":true,"href":"https:\/\/resources.culturalheritage.org\/emg-review\/wp-json\/wp\/v2\/pages\/618"}],"wp:attachment":[{"href":"https:\/\/resources.culturalheritage.org\/emg-review\/wp-json\/wp\/v2\/media?parent=1641"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}