{"id":920,"date":"2019-04-28T16:42:37","date_gmt":"2019-04-28T20:42:37","guid":{"rendered":"http:\/\/resources.culturalheritage.org\/emg-review\/?page_id=920"},"modified":"2019-04-28T16:42:37","modified_gmt":"2019-04-28T20:42:37","slug":"bunz","status":"publish","type":"page","link":"https:\/\/resources.culturalheritage.org\/emg-review\/volume-4-2015-2016\/bunz\/","title":{"rendered":"How Sustainable is File-based Video Art? Exploring the Foundations for Best Practice Development"},"content":{"rendered":"\n

Sophie Bunz, Brian Castriota, Flaminia Fortunato
The Electronic Media Review, Volume Four: 2015-2016<\/em><\/p>\n\n\n\n

ABSTRACT<\/h2>\n\n\n\n

The acquisition of file-based video artworks into collecting institutions, charged with ensuring their long-term viability and accessibility, presents conservators and collection caretakers with many new challenges. This paper explores issues observed in daily practice at the Time-Based Media Conservation Laboratory of the Solomon R. Guggenheim Museum and is the product of a research consortium that was formed as part of a collaboration between the Guggenheim Conservation Department and the Master\u2019s degree program in Conservation at the Bern University of the Arts, Switzerland. The authors of the paper employed a research methodology that included literature review, practical tests, and interviews with internationally-recognized experts engaged with codec development, software engineering, archiving, and digital video preservation. This study highlights specific areas of consensus around the many factors affecting a video file\u2019s sustainability and playback consistency; considers some of the preservation options currently available, including normalization; and offers suggestions around the development of a basis for best practice in the acquisition and conservation of born-digital, file-based video artworks.<\/p>\n\n\n\n

INTRODUCTION<\/h2>\n\n\n\n

This research focuses on the sustainability of video file formats frequently employed by artists and acquired by museums, as well as some of the current options available for ensuring the long-term accessibility and playback consistency of born-digital video artworks. Carried out between 2015 and 2016, it investigates the root causes of some of the playback inconsistencies observed in daily practice at the Solomon R. Guggenheim\u2019s Conservation Department through a combination of practical tests and interviews with experts in the field of codec development, archiving, and preservation. The primary goal of this project was to inform the Guggenheim\u2019s guidelines for artists\u2019 deliverables for the acquisition of born-digital file-based video artworks. Conducted as a collaboration between the Solomon R. Guggenheim\u2019s Conservation Department and the Bern University of the Arts Master\u2019s program in Conservation and Restoration of Modern Materials and Media, this research expands on the findings presented by Joanna Phillips and Agathe Jarczyk to AIC\u2019s Electronic Media Group in 2014 in their talk \u201cLife After Tape: Collecting Digital Video Art.\u201d <\/p>\n\n\n\n

THE SITUATION IN MUSEUM PRACTICE <\/h2>\n\n\n\n

Historically, video artists frequently collaborated with\npost-production houses to produce tape masters that conformed to\nindustry-standardized specifications. Masters and exhibition copies were\ntherefore delivered to museums on a finite range of standardized tape formats\nand optical disks for exhibition purposes. Today\u2014since the shift to file-based\nproduction and delivery\u2014video artworks are now commonly produced by artists and\ntheir studios and acquired by museums as digital files,<\/em> produced in a variety of codecs, containers, resolutions, frame\nrates, and compression factors, selected by artists for either deliberate or\nincidental reasons. <\/p>\n\n\n\n

THE ARTISTIC DIMENSION<\/h2>\n\n\n\n

While museums and archives have overlapping interests\u2014particularly\nin maintaining both the quality and integrity of their assets, as well as their\nlong-term access\u2014the museum is a unique stakeholder due to the high value and\nartistic nature of these assets. These files are conferred with special status\nwithin the museum, valued as documents of the artist\u2019s creative process and a reference for how\nthe artwork\nwas experienced at the point of its creation. The\nmuseum is therefore particularly concerned with sustaining access to all of the\nintended and incidental audiovisual properties of the artist\u2019s masters and\nexhibition copies, properties that may be determined by file format, encoding\nsettings, as well as the original component libraries, playback software and\ndisplay environment. <\/p>\n\n\n\n

THE SITUATION AT THE SOLOMON R. GUGGENHEIM MUSEUM<\/h2>\n\n\n\n

Born-digital video artworks created in the last ten to fifteen\nyears have been produced in a variety of file formats, which differ not only in\ncodec and container, but also in bit sample depth, chroma subsampling patterns,\nresolution, and pixel aspect ratios. Because many artists have employed various\nversions of Apple\u2019s editing software Final Cut Pro to create their works, a\nlarge number of master files that the Guggenheim has received and now cares for\nare encoded in a version or\nflavor of Apple ProRes, Apple\u2019s family of intermediary editing codecs.\nThese include ProRes 422 HQ, ProRes 422 LT, and ProRes 4444. These are\ngenerally wrapped in the MOV Apple QuickTime\ncontainer. In the case of works recorded and edited in DV, HDV, or DVCPRO HD,\nthe Guggenheim cares for master files in these codecs, either in raw form (with\na .dv extension) or wrapped into a MOV (QuickTime Movie) container. The Guggenheim also receives compressed\nexhibition copies from artists, frequently as H.264 in a MOV or MP4\n(MPEG-4) wrapper.\nSome artists also provide normalized preservation masters\u2014these are in\nmany cases video files that have been transcoded to 10-bit uncompressed 422, also known by its FourCC\nidentifier as v210. These files are wrapped with either a MOV or AVI (Audio\nVideo Interleave) wrapper depending on whether they were produced in or for a Mac or Windows environment. <\/p>\n\n\n\n

SUSTAINABILITY <\/h2>\n\n\n\n

In the last decade, several institutions have published guidelines\nand frameworks outlining the factors that may characterize a file format\u2019s\nsustainability. These include the documents published by the U.S. Library of\nCongress (2005), the National Library of the Netherlands (2008), the National\nArchives in the UK (2008), and the Federal Agencies Digitization Guidelines\nInitiative (2014). These institutions and initiatives highlighted several\noverarching and interrelated sustainability factors, briefly summarized:<\/p>\n\n\n\n

The adoption <\/strong>or ubiquity<\/strong> of a format\u2014its well-established\nand widespread use by a critical mass of users\u2014is\nbelieved to enhance a format\u2019s sustainability\nbecause it is more likely to be guaranteed\nbroad and long-lasting support from software developers.<\/p>\n\n\n\n

Formats with external dependencies<\/strong> on specific\nkinds of hardware are generally thought to be less sustainable in the long\nterm. For file-based video, playback depends on\ndisplay hardware, playback or viewing software, as well as the presence of\nspecific decoders. These may all be characterized as external dependencies.<\/p>\n\n\n\n

A format\u2019s disclosure <\/strong>or openness<\/strong> refers to \u201cthe degree to\nwhich complete specifications and tools for validating technical integrity exist\nand are freely accessible\u201d (NDIIPP 2005). Some specifications are completely\nopen and free, some may be obtained with a fee, and some are proprietary and\nnot well-documented in the public domain.<\/p>\n\n\n\n

Interoperability <\/strong>or functionality<\/strong> are frequently discussed in the context of\nsustainability. Some formats may have greater interoperability if they are more\nwidely adopted and their decoders are broadly distributed and implemented, and if they contain certain\nself-descriptive metadata.<\/p>\n\n\n\n

In terms of self-documentation<\/strong>, it is thought that\nfiles that contain self-descriptive metadata will be easier to render correctly\nin the long-term, and will be easier to manage and monitor for integrity. Additionally, streams in\ncontainers with chunk- or frame-level fixity, such as MKV (Matroska)\nor MXF (Material Exchange Format) may be more\nsustainable by allowing for more granular pinpointing of digital change (Rice\n2012).    <\/p>\n\n\n\n

Robustness <\/strong>refers to a format\u2019s\nresilience against corruption. Uncompressed codecs are more resilient against\nperceptible loss due to bit flips than compressed codecs like H.264.  <\/p>\n\n\n\n

Transparency <\/strong>refers to the degree to which\na format is open to direct analysis with basic tools, including human\nreadability using a text-only editor.<\/p>\n\n\n\n

NORMALIZATION <\/h2>\n\n\n\n

Because none of the aforementioned production formats in the\nGuggenheim\u2019s collection satisfy all of these criteria for sustainability, there\nis an argument that normalization may enhance the sustainability of these\nworks. Within the digital archiving world, normalization is understood as the\nstandardization of an existing collection of diverse file types to a single\nformat, thought to provide the best compromise of sustainability factors. In\nmuseum practice, the term normalization is used to describe the action of\ntranscoding the artist\u2019s production format\u2014the \u201cnative master\u201d\u2014in order to\ncreate a second, supplementary backup file in a format that may be more\nsustainable in the long-term. This may be carried out either on the artist\u2019s\nside or within the museum. A normalized file, in theory, may provide a) greater\nresilience against corruption if, for example, a more robust uncompressed codec\nis employed, b) greater operability and playback consistency if a more\nwidely-implemented and\/or self-documenting format is selected, and c) an\nadditional means of access to the audiovisual content should the production\nformat of the original file become obsolete and inaccessible, if a\nnon-proprietary format with good disclosure or openness is chosen. <\/p>\n\n\n\n

PROBLEMS IN PRACTICE <\/h2>\n\n\n\n

A number of key research questions were raised by the problems and\ninconsistencies encountered in daily practice at the Guggenheim. On many\noccasions we observed a discernable audiovisual discrepancy between an artist\u2019s\nnative master file and their provided preservation file. The research carried\nout by Phillips and Jarczyk revealed the shifts in color, luminance, and\ndisplay size that can occur when a file is transcoded or normalized to\nparticular formats. This led us to ask the following: what workflows and\nformats yield acceptable results in terms of maintaining the audiovisual\nproperties of a source file, as well as the playback consistency of its\ntranscodes? In what circumstances does normalization produce a more sustainable\nversion of the work, and what criteria are met? And in what circumstances does\nnormalization not lead to better sustainability, and why not?<\/p>\n\n\n\n

We also encountered a number of files with insufficient\nself-documentation due to inconsistent or incomplete metadata, specifically\nlacking metadata related to the files\u2019 color matrix information. Was this\ndue to the limitations of the metadata extraction tools being employed or was this a symptom of the files themselves? In the case of the\nlater, was this a consequence of the software\nused to create the file or the limits of the codec or container?<\/p>\n\n\n\n

A third problem area we identified pertained to the inconsistent file playback on software players. We observed the same video file render differently in different technical environments, that is, on different machines, in different platforms, and with different software players (see fig. 1). What were the root causes of these shifts and discrepancies, and how do they relate to a) the viewing software, b) the component library or decoders present, c) the file\u2019s self-descriptive metadata (or the lack thereof), d) the codec, and e) the container?<\/p>\n\n\n\n

\"Fig.
Fig. 1: Vectorscope images (Final Cut Pro 7) of an artist-provided ProRes 422 LT MOV master file (left) and its v210 MOV transcode produced with FFmpeg (v3.0.3) (right) showing shifts in chrominance near the skin tone line. <\/figcaption><\/figure>\n\n\n\n

Lastly, we identified the lack of commonly implemented practices\nfor monitoring and quality control for file-based video artworks as a major\nrisk to their sustainability. There are currently no consistent practices\nwithin museum conservation for viewing and assessing files; each museum uses\ndifferent workflows and tools. What metadata extraction tools and viewing software\nare most appropriate for objectively evaluating and characterizing files? Are\nthere other analytical tools that should become part of the time-based media\nconservator\u2019s tool kit? What do they each uniquely provide, and what are the\nlimitations that collection caretakers should be aware of?<\/p>\n\n\n\n

INTERVIEWS WITH SPECIALISTS<\/h2>\n\n\n\n

These issues and questions prompted us to carry out interviews\nwith experts and specialists in the field of codec development, software\nengineering, and digital archiving. Our interviewees included: <\/p>\n\n\n\n

-Eric Hoffert, Technologist and Co-founder of QuickTime<\/p>\n\n\n\n

-Reto Kromer, AV\nConservator and Restoration Scientist, owner of AV Preservation by reto.ch<\/p>\n\n\n\n

-J\u00e9r\u00f4me Martinez, Digital Media Analysis Specialist,\nfounder and president of MediaArea <\/p>\n\n\n\n

-Erik Piil, Digital\nArchivist, Associate Conservator at the Kramlich Collection and New Art Trust<\/p>\n\n\n\n

-Julien Reichel,\nTechnology architect, part of the JVT for the development of H.264 <\/p>\n\n\n\n

-Dave Rice, AV archivist\nand technologist, moving image archivist at CUNY television <\/p>\n\n\n\n

These interviews, conducted over the course of twelve months,\nallowed us to draw several important conclusions that guided our practical\ntests. <\/p>\n\n\n\n

There was a general consensus that care must be taken in\nnormalization to maintain the source file\u2019s sample bit depth and chroma\nsubsampling patterns or shifts in chroma or luminance may arise in the\ntranscoded file (Rice 2015). Specifically, for formats like DV and HDV\u2014with an\n8-bit sample depth and that employ 4:2:0 or 4:1:1 subsampling patterns (depending\non whether they are PAL or NTSC respectively)\u2014the target format must be capable\nof maintaining these features or color shifts will occur. Likewise, because\nApple ProRes 4444 does not employ any chroma subsampling, loss will invariably\noccur if this format is transcoded to a 4:2:2 format like v210. <\/p>\n\n\n\n

Some formats like ProRes, DV, and HDV contain self-descriptive\nmetadata that may be lost in a transcoding process. DV and HDV, for example,\nare compound encodings and thus contain video, audio, as well as data related\nto timecode, captioning, and camera or tape read metadata (Rice 2015 b).\nProRes MOV files contain color matrix metadata\nwhich may be lost when transcoding to certain formats, and these may be\nintegral to audiovisual and\/or playback consistency (Rice 2015b).<\/p>\n\n\n\n

All interviewees also shared the view that no one codec-container\ncombination is currently suitable for the diversity of all file-based video\nthat exists, nor satisfies every criterion of sustainability defined above.\nWhile some of the uncompressed codecs meet certain criteria such as robustness\nand openness, more obscure uncompressed formats that support 4:1:1 and 4:4:4\nsubsampling patterns are poorly implemented, and also key, self-descriptive\nmetadata may be lost in transcoding.<\/p>\n\n\n\n

Moreover, the dependency of a file on specific decoders in the\nsystem\u2019s component libraries emerged as a major limiting factor affecting a\nfile\u2019s playback consistency and sustainability. In order for files to be read\nand displayed correctly, the computer on which the file is being viewed must\nhave the necessary decoders installed. If these change or are missing, a file\nmay not display or may be interpreted incorrectly. <\/p>\n\n\n\n

TRANSCODING & PLAYBACK TESTS<\/h2>\n\n\n\n

Based on these insights, we sought to investigate what essential features and metadata are maintained or lost when transcoding\nbetween different codecs, wrappers, and with different tools, as well as the\neffects these variables have on color shifts and playback consistency. For our\ntests we chose ProRes 422 LT transcoded to 10-bit uncompressed 4:2:2 as a case\nstudy. <\/p>\n\n\n\n

TEST 1: MAINTAINING AUDIOVISUAL\nCONSISTENCY WITH DIFFERENT TRANSCODING SOFTWARES<\/h2>\n\n\n\n

In our first test, our primary objective was to determine which\ntranscoding tools maintain audiovisual consistency between the original file\nand its transcode, that is, produce a transcode without any color shifts. An Apple ProRes LT 422 MOV file consisting of ten seconds\nof SMPTE color bars was transcoded to v210 in a MOV wrapper using Final Cut Pro 6, MPEG\nStreamclip, Adobe Premiere Pro, QuickTime Pro 7 and FFmpeg (v3.0.3).<\/p>\n\n\n\n

Command\nused in FFmpeg for transcoding to uncompressed 4:2:2: <\/p>\n\n\n\n

 $ffmpeg -i input -c:v v210 -c:a copy\noutput.[mov]<\/pre>\n\n\n\n
To quantitatively evaluate any color shifts that occur through transcoding, we compared the RGB values of the left-most gray color bar in each file, measured using Apple\u2019s Digital Color Meter Utility (v4.4) in Display native values<\/em> mode (fig. 2). <\/p>\n\n\n\n
\"Fig.
Fig. 2: From left to right: ProRes 422 LT MOV source file, v210 MOV transcodes created with Final Cut Pro 6, MPEG Streamclip, Adobe Premiere Pro, QuickTime Pro 7, and FFmpeg (v3.0.3) viewed in QuickTime Player 7. <\/figcaption><\/figure>\n\n\n\n
When viewed in QuickTime Player 7 all transcodes exhibited RGB value shifts, except for the transcode\nproduced in Final Cut Pro 6; in the source file, the RGB values of the\nleft-most color bar measured 135, 135, 135 and this was only maintained in the\nFinal Cut Pro 6 transcode while the others read 123, 123, 123. Moreover, in\naddition to RGB value shifts, the transcode created with QuickTime Pro 7\ndisplayed at a slightly scaled size, although the file itself remained at 1920\nby 1080 pixels. The same results that occurred in QuickTime Player 7 were observed in QuickTime Player 10. When viewed in VLC\nMedia Player (v2.0.9), the left-most gray color\nbar measured 122, 123, 122 in the original, as well as each of the transcodes (see table 1, Environment 1).<\/p>\n\n\n\n
TEST 2: MAINTAINING AUDIOVISUAL CONSISTENCY IN DIFFERENT\nCONTAINERS<\/h2>\n\n\n\n
In our second test we investigated how different containers maintain audiovisual consistency between the original file and the v210 transcodes. Files were again viewed in QuickTime Player 7, QuickTime Player 10, and VLC Media Player (v2.0.9). We compared the original source file with a v210 MOV transcode produced in Final Cut Pro 7, as well as three transcodes produced with FFmpeg (v3.0.3) wrapped in MOV, AVI, and MKV (fig. 3).<\/p>\n\n\n\n
\"Fig.
Fig. 3: Comparison of the impact of different containers on RGB value shifts during normalization, from left to right: ProRes 422 LT MOV source file, v210 MOV (FFmpeg), v210 AVI (FFmpeg), v210 MKV (FFmpeg), v210 MOV (Final Cut Pro 6) viewed in QuickTime Player 7. <\/figcaption><\/figure>\n\n\n\n
FFmpeg\ncommand used in test 2 for transcoding to uncompressed 4:2:2: <\/p>\n\n\n\n
 $ffmpeg -i input -c:v v210 -c:a copy\noutput.[mov\/avi\/mkv]<\/pre>\n\n\n\n
When viewed in QuickTime Player 7 and 10, <\/strong>neither the FFmpeg transcodes wrapped in AVI and MKV, nor the MOV produced with Final Cut Pro 7 exhibited RGB value shifts. A shift was measured in the FFmpeg-produced v210 MOV (fig. 3). In VLC, we observed no RGB value shifts between the original file and any of the transcodes, but the RGB values were different from those of the original file when viewed in QuickTime Player 7 and 10 (see table 1, environment 1).<\/p>\n\n\n\n
\"Table.
Table. 1: Comparison of the measured RGB values of the source file and various transcodes across four different technical environments, viewed in QuickTime Player 7, QuickTime Player 10, and VLC. <\/figcaption><\/figure>\n\n\n\n
In an effort to establish whether a specific workflow and container\/codec combination yields a more sustainable normalized file with adequate and correct self-documentation, we also examined and compared the metadata of each of these files. Metadata were inspected with MediaInfo (command line tool and GUI v0.7.74), and Invisor (v3.7). The v210 MOV produced with Final Cut Pro 7 was the only file that retained metadata related to color matrix coefficients, transfer characteristics, as well as the date encoded and date tagged as seen in figure 4. <\/p>\n\n\n\n
\"Fig.
Fig. 4: Metadata comparison of five files using Invisor (3.7), from left to right: ProRes 422 LT MOV, v210 MOV (FFmpeg), v210 AVI (FFmpeg), v210 MKV (FFmpeg), v210 MOV (Final Cut Pro 7). <\/figcaption><\/figure>\n\n\n\n
Surprisingly, this information was not present for the MOV, AVI and MKV v210 transcodes created with FFmpeg, although no shifts in color were observed in the AVI and MKV file. Further analysis of the v210 MOV transcodes with Atom Inspector revealed that the \u2018colr\u2019 atom was written only in the Final Cut Pro 7 transcode and not in the FFmpeg transcode (see fig. 6). <\/p>\n\n\n\n
TEST 3: IMPACT OF THE \u2018COLR\u2019 ATOM ON MAINTAINING AUDIOVISUAL CONSISTENCY <\/h2>\n\n\n\n
In the previous test we\nmeasured RGB value shifts for the v210 MOV transcode created with FFmpeg and we\nalso observed that it was missing its \u2018colr\u2019 atom. The \u2018colr\u2019 atom is a chunk\nof data specific to QuickTime formats that, in the case of video, includes the\ncolor parameter type (\u2018nclc\u2019) and three indexes consisting of \u201ca table of\nprimaries, a table of transfer function coefficients, and a table of matrixes\u201d\n(Apple Inc. 2015). This atom allows for the video file to be rendered in the\nappropriate color space, which, for HD video, is defined by the ITU-R\nRecommendation BT.709. As Dave Rice (2015) has noted, \u201cwith the absence of\n\u2018nclc\u2019 data, QuickTime applications defer to use SMPTE-C\/BT.601 to convert YUV\nto RGB, which would give incorrect colors if the intention were to use EBU PAL\nor Rec. 709.\u201d<\/p>\n\n\n\n
In this test we employed\nFFmpeg\u2019s experimental -movflags +write_colr option\u2014a recently implemented flag that copies\nthe \u2018colr\u2019 atom of the source file\u2014to determine whether the RGB value shifts\nwere a result of the QuickTime Player\u2019s incorrect interpretation of the color\nspace due to this missing \u2018nclc\u2019 tag. As\nDave Rice explained in our interview: “the ‘nclc’ atom is only occasionally required\nwithin the QuickTime container. Sometimes the corresponding information is not\nprovided or known. Also the QuickTime muxer of FFmpeg only recently added\nsupport for writing the ‘nclc’ atom (see the \u2013write_colr option).” (Rice,\nwritten interview with authors, November, 2015).<\/p>\n\n\n\n
The subsequent transcode produced with the addition of the -movflags +write_colr option into the command yielded a file that maintained the information contained in the \u2018colr\u2019 atom, visible both in Invisor (fig. 5) and Atom Inspector (v.1.0.3) (fig. 6). Moreover, no RGB value shifts were detected in QuickTime Player 7 and 10, confirming that the RGB value shifts measured previously were a result of QuickTime\u2019s incorrect interpretation of the file in the absence of this information. Additionally, we found the -map_metadata g option necessary to preserve the metadata related to the \u2018encoded date\u2019 and \u2018tagged date\u2019 fields. Using following FFmpeg command: <\/p>\n\n\n\n
$ffmpeg -i input -c:v v210 -c:a copy -movflags +write_colr -map_metadata g output.mov<\/pre>\n\n\n\n
\"Fig.
Fig. 5: (1) ProRes 422 LT MOV, (2) v210 MOV using our previous command, and (3) v210 MOV with -movflags +write_colr option and -map_metadata g option. 
<\/figcaption><\/figure>\n\n\n\n
\"Fig.
Fig. 6: Atom Inspector showing \u2018colr\u2019 atom information: (1) ProRes 422 LT MOV, (2) v210 MOV using our previous command, and (3) v210 MOV with -movflags +write_colr option and -map_metadata g option.  <\/figcaption><\/figure>\n\n\n\n
TEST 4: PLAYBACK CONSISTENCY ON\nCOMPUTERS WITH DIFFERENT OPERATING SYSTEMS, SOFTWARE VERSIONS, AND COMPONENT\nLIBRARIES <\/h2>\n\n\n\n
In our fourth test we introduced another variable: the impact of\ndiffering technical environments on audiovisual and playback consistency (see table 1). Test 1 and Test 2, carried\nout on a Mac running OS X 10.7.5 (Lion), were repeated on three other\ncomputers: one running OS X 10.7.5 (Lion), and two others running Mac OS X\n10.10.5 (Yosemite). QuickTime 7 version 7.6.6 was installed on all four computers;\nEnvironments 1 and 2 were running QuickTime 10 version 10.1 while Environments\n3 and 4 were running version 10.4. Three different versions of VLC were tested,\nup to and including the latest build, version 3.0.0.<\/p>\n\n\n\n
In order to investigate how the presence\nof certain decoders (or the lack thereof) affected playback consistency, we\nexamined the component library on each computer (in\nthe folder ~\/Library\/QuickTime) and employed the command qt_thing, developed by David Van Brink as part of the qt_tools suite, to extract a list of the\nQuickTime components installed on each machine (Van Brink 2018).<\/p>\n\n\n\n
The four technical environments tested were different from each other in the number of library components present in each OS (see table 1). However, only Environment 1 was uniquely different from the other three in terms of the number of 10-bit uncompressed components installed; the qt_thing command showed that there were eight v210 components present on Environment 1\u2014two encoders and six decoders\u2014compared to the four uncompressed components present on the other technical environments (see fig. 7).<\/p>\n\n\n\n
\"\nFig.
Fig. 7: qt_thing lists eight v210 components installed on Environment 1.  <\/figcaption><\/figure>\n\n\n\n
When the seven video files were played back, the following\nobservations were made: in Environment 2, RGB value shifts were measured for\nthe FFmpeg v210 AVI file when viewed in QuickTime Player 7, which were not\nobserved when viewed in the other three environments. In Environment 2, no RGB\nvalue shifts were measured for the transcodes produced with Final Cut Pro 6 or\nFinal Cut Pro 7 when the files were played back in QuickTime Player 7 and 10.\nFinally, interoperability issues arose when the transcode wrapped in MKV was\nplayed back in QuickTime Player 7 and 10.<\/p>\n\n\n\n
When played back in QuickTime Player 10, in both Environments 3\nand 4, we measured RGB value shifts in the test files transcoded with FFmpeg\n(both in MOV and AVI containers) and with Final Cut Pro 6. These shifts did not\nappear\nwhen the files were played back in QuickTime Player 7. No RGB value shifts\noccurred for the transcode produced in Final Cut Pro 7 when viewed both in\nQuickTime Player 7 and 10. Across all environments, in the original file and\nall transcodes, the RGB values of the left-most gray color bar measured\nconsistently 122, 123, 122 when viewed in VLC.<\/p>\n\n\n\n
INTERPRETATION OF FINDINGS<\/h2>\n\n\n\n
Metadata information related to the BT.709 specification for color\nprimaries, matrix coefficients, and transfer characteristics together with\nencoded and tagged date were preserved in the transcode created with Final Cut\nPro 7 and no RGB value shifts were measured in QuickTime Player 7. The same\nconsistency was maintained in QuickTime Player 7 for the v210 MOV file produced\nwith FFmpeg using the -movflags +write_colr -map_metadata g options. <\/p>\n\n\n\n
In general, audiovisual and\nplayback inconsistencies were observed more so in QuickTime Player 10 than in QuickTime\nPlayer 7. The consistent RGB value measurements in VLC for the ProRes file and\nall v210 transcodes in all tested environments and software versions\ndiffered from those measured in QuickTime Player 7 and 10. The values observed\nin VLC are similar to those observed in the\nQuickTime Players when the \u2018colr\u2019 atom was missing, suggesting that the shifts\nmay be due to the incorrect interpretation of these versions of VLC, which seem to default\nto the BT.601 color matrix specification. <\/p>\n\n\n\n
Our tests confirmed that the amount of self-descriptive metadata\nmaintained in a transcoding is very much dependent on the codec and container\nof both the source and target files. Moreover, both the software and the\nsettings or options employed play a key role in retaining metadata, when\nexporting from an editing timeline, transcoding, or rewrapping. In the case of\nProRes LT 422, our tests revealed a correlation between audiovisual consistency\nand the retention of metadata related to color primaries, matrix coefficients,\nand transfer characteristics contained in the \u2018colr\u2019 atom; the v210 MOV transcodes produced with Final Cut Pro 7 and FFmpeg (using\nthe -movflags +write_colr -map_metadata g option)\nretained this metadata and no RGB value shifts were measured in QuickTime Player\n7.<\/p>\n\n\n\n
When the transcodes were viewed in QuickTime Player 7 and 10, we\nobserved no discrepancy in RGB values across identical technical environments\n(with the same OS) when the same decoders (v210) and software versions were\npresent, excluding the v210\nAVI transcode. We were unable to determine the root\ncauses of the RGB value shifts observed for the v210 AVI transcode played back on Environment 2 in QuickTime 7 and\nmore broadly in QuickTime 10 across the environments. This observation could be\nlinked to Dave Rice\u2019s findings (2015) that even if two computers have the same OS and QuickTime\nversion, playback inconsistencies may arise when the component libraries differ.<\/p>\n\n\n\n
Our tests also confirmed that chroma subsampling pattern and bit\ndepth are specific to codecs and must be maintained in transcoding or color\nshifts may arise; in separate tests, not presented here, we found that color\nshifts occur when H.264, DV, HDV, and ProRes 4444 files are transcoded to 4:2:2\ncodecs like v210. We found that v210 in a MOV wrapper may however be a suitable preservation format for ProRes 422\ncodecs if a particular workflow is followed. Having both good disclosure and\nresilience, v210 satisfies several sustainability criteria that ProRes does\nnot. We found that v210 in a MOV container can also maintain the same level of\nself-documentation, i.e. the self-descriptive metadata that existed in the\noriginal file. However, it should be stressed that 10-bit uncompressed 4:2:2 is\nnot suitable for transcoding all kind of video files, particularly those\nformats that do not employ chroma subsampling\nlike ProRes 4444, or have embedded timecode\ndata like DV, DVCPRO HD, and HDV. <\/p>\n\n\n\n
Overall, our practical tests demonstrated how viewing software,\nsoftware versions, operating systems, component libraries, and a file\u2019s\nmetadata can each play a role in the correct and consistent rendering of a\nvideo file. <\/p>\n\n\n\n
NORMALIZATION AND BEYOND<\/h2>\n\n\n\n
Due in large part to the complexity and diversity of file-based\nvideo that has been created in the last twenty years, there is still no single\npreservation format capable of satisfying every criterion of sustainability. In\nmany cases normalization entails a compromise. That said, this may change in\nthe coming years if resources are focused on developing and implementing\nformats that satisfy more of these criteria. Bearing in mind that certain\nformats cannot be normalized to a well-implemented codec without manipulating\nchroma subsampling and bit depth, and thus introducing color shifts, further efforts should be\nundertaken to explore viable solutions for their preservation. In particular,\nfurther research should investigate the sustainability of DVCPRO HD (8-bit\n4:2:0) and HDV (8-bit 4:2:0\/4:1:1), two relatively-short-lived HD formats that\nemploy anamorphic pixels, comparing both the changes and advantages that\nnormalization to square-pixel formats entails. <\/p>\n\n\n\n
We believe close attention\nshould be paid to the Advanced Media Workflow Association\u2019s efforts to develop\nthe AS-07 specification for MXF (AMWA 2016), which will allow for native encoded\nDV bitstreams or MPEG transport streams and their metadata to be preserved and\nmapped to the MXF wrapper. Additionally, the open source container Matroska,\ndeveloped by Steve Lhomme, seems to be promising in its capacity for storing\nself-descriptive metadata and frame-level fixity data. The lossless, open\nsource codec FFV1 should also be investigated in\nfuture tests. <\/p>\n\n\n\n
DOCUMENTING AND COLLECTING COMPONENT LIBRARIES<\/h2>\n\n\n\n
As long as the consistent viewing experience of file-based video artworks relies on external dependencies such as particular display hardwares, viewing softwares, and decoders, these works remain vulnerable to changes that may be perceived as losses. Like with many time-based media artworks, the preservation of one element in the system such as the file does not necessarily result in the preservation of the work as a whole (note 1).
<\/p>\n\n\n\n
 Given the importance of decoders in maintaining access and playback consistency of video files, further research should explore the viability of documenting and collecting component libraries from artists as a possible preservation strategy for museums. As demonstrated in our tests, the qt_thing command in qt_tools is useful for documenting QuickTime components present on a Macintosh system, however, this tool has not been updated since 2008, and this may cause problems in the future for its use and adoption. <\/p>\n\n\n\n
The Pericles Extraction Tool\u2014an open-source tool based on Java\nsoftware under development within the PERICLES Project\u2014will hopefully offer a\nway to capture and document the components in use when playing back a video\nfile in a specific playback environment in the future (Corubolo et al. 2014). Such a tool could\npotentially allow for the comprehensive documentation of the native production\nand viewing environment of a file-based video artwork at the point of\nacquisition. Further development should allow users to determine a) where\nindividual components are located within the system, b) the software versions\nthey correspond to, c) the date they were installed, and d) the decoder in use\nduring playback of a video file. Running such a tool at the point of acquisition\nwould of course require greater collaboration between collection caretakers and\nartists, but this remains an admirable goal. <\/p>\n\n\n\n
TOOLS FOR VIEWING AND ASSESSING FILES<\/h2>\n\n\n\n
The DigitalColor Meter Utility proved useful in identifying\naudiovisual inconsistencies between source files and rewrapped or transcoded\nfiles. In the course of our research, we were also able to identify several\nother software tools useful in the examination and technical analysis of\nfile-based video. The open source metadata extraction tools developed by the\nMediaArea team \u2013 including MediaInfo and MediaConch \u2013 were enormously helpful\nin our testing, and they have emerged as essential analytical tools for\ncaretakers of file-based video. Invisor\u2014a GUI tool built on MediaInfo\u2014allows for\nside-by-side metadata comparison of up to twenty files. In our research it\nproved to be an extremely useful tool for comparing metadata between files and\ntheir derivatives. The\nsoftware has been updated recently (May 19, 2016)\nto Version 3.7 with the latest MediaInfo library. <\/p>\n\n\n\n
To date, no standard viewing software has been adopted for\nevaluating file-based video and this remains a problem for the field. Within\nthe scope of our tests we were unable to determine whether VLC is currently an\nappropriate tool to objectively assess and characterize the audiovisual\nproperties of video files.\nBut according to Felix Paul K\u00fchne at VideoLAN (E-mail correspondence, August 7,\n2016), there are efforts underway to implement \u2018colr\u2019 support into the latest\nversion of VLC (v.3.0.0). <\/p>\n\n\n\n
Given the proprietary nature of the QuickTime Players and Apple\u2019s announcement\nin 2016 that it will drop support for them in Windows due to security issues\n(US-CERT 2016), there is an urgent need for\na non-proprietary, widely-adopted viewing software that correctly renders a\nvideo file across a range of technical environments. Assuming it is provided\nwith regular and long-term maintenance, such a tool would allow caretakers to\nreliably characterize digital video files entering collections and would also\nallow access to the variety of native video file formats in museum collections\naround the world for many years to come. <\/p>\n\n\n\n
CONCLUSION<\/h2>\n\n\n\n
During the course of this research it became clear to us that\nthere were no simple or easy answers to our questions pertaining to the\nlong-term sustainability of file-based video artworks. This is due in large\npart to the complex and ever-evolving digital landscape into which these works\nwere born. The threat to their long-term sustainability posed by the\nunannounced redesign and drop of support for obsolete formats is real. These\nrisks are compounded by a lack of implemented practices and digital tools for\ncharacterizing and evaluating digital video artworks and their native viewing\nenvironment. That said, many efforts are underway to address these issues. <\/p>\n\n\n\n
Those of us in the museum world caring for file-based video assets\nmust therefore make our interests and concerns known by working more closely\nwith the archival community in developing their open source projects like\nFFmpeg, VideoLAN\u2019s VLC player, and MediaArea\u2019s metadata extraction and quality\ncontrol tools; participating in meetings of standards bodies such as the\nSociety for Motion Picture and Television Engineers (SMPTE) and the Motion\nPicture Experts Group (MPEG); and by reaching\nout to industries and software developers like Apple and others. Rather than\nrelying solely on the efforts of those in adjacent fields and industries to\ndevelop tools and protocols for us to borrow, we should be actively engaging\nand collaborating with these experts and specialists. Only then will we be able\nto establish and maintain our own best practices of care around file-based\nvideo art.<\/p>\n\n\n\n
ACKNOWLEDGEMENTS<\/h2>\n\n\n\n
We are very grateful to our supervisors Agathe Jarczyk and Joanna\nPhillips for providing guidance and feedback at each stage of our research, and\nfor facilitating this collaboration. We also thank our interviewees\u2014Eric\nHoffert, Reto Kromer, Jerome\nMartinez, Erik Piil, Julien Reichel, and Dave\nRice\u2014for their critical insights and participation in our research. Finally, we\nare grateful for the generous support of the\nBern University of the Arts, the Solomon R. Guggenheim Museum, the Samuel H.\nKress Foundation, and the European Union\u2019s Horizon 2020 Actions under\nthe Marie\nSk\u0142odowska-Curie\nGrant no. 642892.<\/p>\n\n\n\n
NOTES<\/h2>\n\n\n\n
  1.  As Dave Rice (2015) indicates, \u201cSustaining video presentations through emulation requires maintaining a player and all of its dependencies. For instance, if a creator determines that a video is intended for presentation through QuickTime Pro 7 this may mean preserving QuickTime Pro 7 as an application along with its underlying set of components as well as an operating system that can support QuickTime Pro 7\u2019s underlying 32 bit QtKit framework. Often video players rely on system codec libraries or audio-visual frameworks working behind the scenes, so it can be challenging to determine what exactly is necessary to document emulation sufficiently.\u201d<\/li><\/ol>\n\n\n\n
    REFERENCES <\/h2>\n\n\n\n
    The Advanced Media Workflow Association (AMWA). 2016. MXF for Archive & Preservation: AS-07. http:\/\/amwa.tv\/projects\/2016-07-06(AMWA%20AS-07%20handout,%20A4).pdf<\/a> (accessed 08\/18\/16). <\/p>\n\n\n\n
    Apple Inc. 2002. QuickTime File Format. Cupertino: Apple Computer Inc.  https:\/\/developer.apple.com\/standards\/qtff-2001.pdf<\/a> (accessed 08\/18\/16).<\/p>\n\n\n\n
    Apple Inc. 2009. Final Cut Pro 7 Professional Formats and Workflows. Cupertino: Apple Computer Inc. http:\/\/operationalportal.com\/wp-content\/uploads\/Final-Cut-Pro-7-Professional-Formats-and-Workflows-en.pdf<\/a> (accessed 08\/18\/16). <\/p>\n\n\n\n
    Apple Inc. 2012. QuickTime File Format Specification. Cupertino: Apple Computer Inc. https:\/\/wikileaks.org\/sony\/docs\/05\/docs\/Apple\/qtff.pdf<\/a> (accessed 08\/18\/16).<\/p>\n\n\n\n
    Apple Inc. 2014. Apple\nProRes: White Paper. Cupertino: Apple Computer Inc. www.apple.com\/final-cutpro\/docs\/Apple_ProRes_White_Paper.pdf<\/a> (accessed 08\/18\/16).<\/p>\n\n\n\n
    Apple Inc. 2015. Media Data\nAtom Types. https:\/\/developer.apple.com\/library\/mac\/documentation\/QuickTime\/QTFF\/QTFFChap3\/qtff3.html<\/a> (accessed 08\/18\/16).<\/p>\n\n\n\n
    Blood, G. 2011. Refining Conversion Contract Specifications: Determining Suitable Digital Video Formats for Medium-term Storage. www.digitizationguidelines.gov\/audio-visual\/documents\/IntrmMastVidFormatRecs_20111001.pdf<\/a> (accessed 08\/18\/16).<\/p>\n\n\n\n
    Cherna, T., P. Hoddie, and C. Pirazzi. 1999. Technical Note TN2162: Uncompressed Y\u2032CbCr Video in QuickTime Files. https:\/\/developer.apple.com\/library\/mac\/technotes\/tn2162\/_index.html<\/a> (accessed 08\/18\/16).<\/p>\n\n\n\n
    Corubolo, F., A. Eggers, A.\nHasan, M. Hedges, S. Waddington, and J. Ludwig. 2014.A pragmatic approach to significant environment information\ncollection to support object reuse. Proceedings\nof the 11th International Conference on Digital Preservation (iPres14)<\/em>.\nState Library of Victoria.<\/p>\n\n\n\n
    Dance Heritage Coalition, Inc. and Media Matters, LLC. 2004. Digital Video Preservation Reformatting Project: A Report. www.danceheritage.org\/digitalvideopreservation.pdf<\/a> (accessed 08\/18\/16).<\/p>\n\n\n\n
    Escobedo, K. 2011. Digital Video Preservation: Identifying Containers and Codecs. https:\/\/siarchives.si.edu\/blog\/digital-video-preservation-identifying-containers-and-codecs<\/a> (accessed 08\/18\/16).<\/p>\n\n\n\n
    Escobedo, K. 2013. Digital Video Preservation: Continuing the Conversation. https:\/\/siarchives.si.edu\/blog\/digital-video-preservation-continuing-conversation <\/a>(accessed 08\/18\/16).<\/p>\n\n\n\n
    Federal Agencies Digitization Guidelines Initiative (FADGI). 2014.Digital File Formats for Video Tape Reformatting. www.digitizationguidelines.gov\/guidelines\/video_reformatting_compare.html<\/a> (accessed 08\/18\/16).<\/p>\n\n\n\n
    Fleischhauer, C. and FADGI. 2011. Preservation Video File Format Issues and Considerations: What Have We Encountered as We Drafted MXF AS-AP? www.digitizationguidelines.gov\/guidelines\/FADGI_MXF_ASAP_Issues_20110815.pdf<\/a> (accessed 08\/18\/16).<\/p>\n\n\n\n
    Fleischhauer, C. 2011. Format Considerations in Audio-Visual Preservation Reformatting: Snapshots from the Federal Agencies Digitization Guidelines Initiative. Information Quarterly<\/em> 22 (2, 2010): 35-40. www.digitizationguidelines.gov\/audio-visual\/documents\/IP_Fleischhauer_AudioVisual_Reformatting_isqv22no2.pdf<\/a> (accessed 08\/18\/16).<\/p>\n\n\n\n
    Goldsmith, B. 2013.\nDigitizing Video for Long-Term Preservation: An RFP Guide and Template.\nPreservation and Conservation Department, New York University Libraries. www.scart.be\/?q=en\/content\/digitizing-video-long-term-preservation-rfp-guide-and-template<\/a> (accessed 08\/18\/16).<\/p>\n\n\n\n
    Jarczyk, A. and J. Phillips.\n2014. Life after Tape: Collecting Digital Video Art. Talk delivered at the\nElectronic Media Group Session of AIC\u2019s\n42nd Annual Meeting \u201cConscientious Conservation: Sustainable Choices in\nCollection Care\u201d<\/em>, San Francisco.<\/p>\n\n\n\n
    Lacinak, C. 2010. A Primer on Codecs for Moving Image and Sound Archives. www.avpreserve.com\/wp-content\/uploads\/2010\/04\/AVPS_Codec_Primer.pdf<\/a> (accessed 08\/18\/16).<\/p>\n\n\n\n
    Library of Congress. 2013. Sustainability Factors. www.digitalpreservation.gov\/formats\/sustain\/sustain.shtml<\/a> (accessed 08\/18\/16).<\/p>\n\n\n\n
    Lorrain, E. 2014. A short guide to choosing a digital format for video archiving masters. SCART by PACKED vzw. www.scart.be\/?q=en\/content\/short-guide-choosing-digital-format-video-archiving-masters<\/a> (accessed 08\/18\/16). <\/p>\n\n\n\n
    Matroska\n(2018). Matroska Media Container. https:\/\/www.matroska.org\/<\/a> (accessed 04\/07\/19).<\/p>\n\n\n\n
    MediaArea (2019). MediaInfo. https:\/\/mediaarea.net\/de\/MediaInfo<\/a> (accessed 04\/07\/19).<\/p>\n\n\n\n
    The National Archives. 2008. Selecting File Formats for Long-Term\nPreservation.www.nationalarchives.gov.uk\/documents\/selecting-file-formats.pdf<\/a> (accessed 08\/18\/16).<\/p>\n\n\n\n
    National Library of the Netherlands. 2008.Evaluating File Formats for Long-Term Preservation. www.kb.nl\/sites\/default\/files\/docs\/KB_file_format_evaluation_method_27022008.pdf<\/a> (accessed 08\/18\/16).<\/p>\n\n\n\n
    Sacredote, A. and L. Sorensen. 2009.Codec Comparison for the digital Preservation for Analog Video. Proceedings of the Electronic Media Group Session, AIC 35th Annual Meeting<\/em> May 15-20, 2009, Los Angeles. The Electronic Media Review<\/em> (1, 012). Washington D. C.: AIC. 59-70. www.bavc.org\/sites\/default\/files\/resource\/codecs_sacerdotesorensen.pdf<\/a> (accessed 08\/18\/16).<\/p>\n\n\n\n
    Poynton, C. 2012. Digital Video and HD: Algorithms and\nInterfaces<\/em>. Waltham: Elsevier.<\/p>\n\n\n\n
    Pozdeev,\nM. 2019. Invisor. https:\/\/www.invisorapp.com\/<\/a> (accessed 07\/07\/19).<\/p>\n\n\n\n
    Rice, D. 2012. Reconsidering\nthe Checksum for Audiovisual Preservation: Detecting digital change in\naudiovisual data with decoders and checksums. IASA Journal<\/em> 39.<\/p>\n\n\n\n
    Rice, D. 2015. Sustaining\nConsistent Video Presentation. www.tate.org.uk\/research\/publications\/sustaining-consistent-video-presentation<\/a> (accessed 08\/18\/16).<\/p>\n\n\n\n
    Rice, D, 2015b. Interview by Bunz, Sophie, Castriota, Brian & Fortunato, Flaminia. Research Towards the Preservation of Digital Video Artworks: File Sustainability and Risk Factors<\/em>. Written Interview, November 2015.<\/p>\n\n\n\n
    Richardson, I. 2010. The H.264 Advanced Video Compression\nStandard.<\/em>\nChichester: Wiley.<\/p>\n\n\n\n
    United States Library of Congress and National Digital Information Infrastructure and\nPreservation Program (NDIIPP). 2005. Sustainability of Digital Formats:\nPlanning for Library of Congress Collections. www.digitalpreservation.gov\/formats\/index.shtml<\/a> (accessed 08\/18\/16).<\/p>\n\n\n\n
    US-CERT.\n2016. Alert (TA16-105A): Apple Ends Support for QuickTime for Windows; New\nVulnerabilities Announced. https:\/\/www.us-cert.gov\/ncas\/alerts\/TA16-105A<\/a> (accessed 08\/18\/16).<\/p>\n\n\n\n
    Van\nBrink, D. 2018. QT_TOOLS. http:\/\/dvb.omino.com\/sw\/qt_tools\/<\/a> (accessed 01\/25\/19).<\/p>\n\n\n\n

    Sophie Bunz, MA Student
    Conservation-Restoration Modern Materials and Media
    Bern University of the Arts 
    bunzsophie@gmail.com<\/a><\/p>\n\n\n\n
    Brian Castriota
    Samuel H. Kress Fellow in Time-Based Media Conservation
    Solomon R. Guggenheim Museum
    bcastriota@gmail.com<\/a><\/p>\n\n\n\n
    Flaminia Fortunato, MA Student
    Conservation-Restoration Modern Materials and Media
    Bern University of the Arts
    flaminia.fortunato7@gmail.com<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
    Sophie Bunz, Brian Castriota, Flaminia FortunatoThe Electronic Media Review, Volume Four: 2015-2016 ABSTRACT The acquisition of file-based video artworks into collecting institutions, charged with ensuring their long-term viability and accessibility, presents conservators and collection caretakers with many new challenges. This paper explores issues observed in daily practice at the Time-Based Media Conservation Laboratory of the … Continue reading “How Sustainable is File-based Video Art? Exploring the Foundations for Best Practice Development”<\/span><\/a><\/p>\n","protected":false},"author":37,"featured_media":0,"parent":616,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-920","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/resources.culturalheritage.org\/emg-review\/wp-json\/wp\/v2\/pages\/920","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/resources.culturalheritage.org\/emg-review\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/resources.culturalheritage.org\/emg-review\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/resources.culturalheritage.org\/emg-review\/wp-json\/wp\/v2\/users\/37"}],"replies":[{"embeddable":true,"href":"https:\/\/resources.culturalheritage.org\/emg-review\/wp-json\/wp\/v2\/comments?post=920"}],"version-history":[{"count":0,"href":"https:\/\/resources.culturalheritage.org\/emg-review\/wp-json\/wp\/v2\/pages\/920\/revisions"}],"up":[{"embeddable":true,"href":"https:\/\/resources.culturalheritage.org\/emg-review\/wp-json\/wp\/v2\/pages\/616"}],"wp:attachment":[{"href":"https:\/\/resources.culturalheritage.org\/emg-review\/wp-json\/wp\/v2\/media?parent=920"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}