GDFR Wiki	Main Page | About | Help | FAQ | Special pages | Log in
	Printable version | Disclaimers | Privacy policy

Activity 3: Gathering of functional and non-functional requirements for a community format registry

From GDFR Wiki

Contents 1 Activity 3: Gathering of functional and non-functional requirements for a community format registry 1.1 Existing GDFR Use Cases and Functional Requirements: 1.2 2008/9 Format Registry Use Cases and Requirements: 1.2.1 Introduction 1.2.2 CFR Functional [FR] and Non-Functional [NFR] Requirements 1.2.3 Other Requirements 1.2.4 Use Cases 1.2.5 National Library of New Zealand Thoughts

Activity 3: Gathering of functional and non-functional requirements for a community format registry

---

Lead: Stephen Abrams (CDL), Steve Knight (NLNZ) Additional Participants: Open to all

Deadline: January 15, 2009

Subtask 1: The lead group will look at the GDFR wiki page that has been set up for this activity and will determine if any changes to the wiki page are needed to support this activity. Any wiki page changes can be made directly by this group or can be requested from Andrea Goethals. This group will determine how best to solicit participation, for example through targeted invitations or more openly through an email list.

Subtask 2: Participants will compile a set of functional and non-functional requirements for the format registry. At a minimum participants should document the name of their institution and rate the importance of the requirement to their institution.

Deliverables: Completed compilation of functional and non-functional requirements for the format registry on the GDFR wiki.

---

Existing GDFR Use Cases and Functional Requirements:

2003
The original use cases for GDFR were submitted in 2003 by the Bibliothèque nationale de France, Harvard University, JISC, JSTOR, MIT, NYU, OCLC, UK National Archives, and the University of Pennysylvania. The full text of these use cases is on this website at http://gdfr.info/docs.html#usecases_2003.

A list of services that GDFR should support was compiled by Mackenzie Smith (MIT) based on the above use cases. This service model is documented in: "Global Format Registry Service Architecture". MS; March 11, 2003 which is on this website at http://gdfr.info/docs.html#service_model

2006-7
OCLC developed a set of use cases for GDFR in 2006-7. They are contained within this document: "GDFR Analysis Model, v. 2.0; October 1, 2007 which is on this website at http://gdfr.info/docs.html#analysis_model. This document is the GDFR functional requirements specification.

---

2008/9 Format Registry Use Cases and Requirements:

This section will be used to compose a complete set of functional and non-functional requirements / use cases for a format registry.

Introduction

An understanding of digital formats is fundamental to the effective long-term curation and preservation of digital assets. To date, two large-scale efforts – the PRONOM project in the UK, and the GDFR (Global Digital Format Registry) project in the US – have been focused on the need to collect, manage, and preserve significant representation information about digital formats. However, the stakeholder communities addressed by these projects have expressed concern over the necessity to provide intellectual, technical, administrative, and financial support for two independent, and undoubtedly duplicative, registries. An informal group of international library, archive, and preservation institutions has agreed to support the efforts of an ad hoc Format Registry Working Group (FRWG) to determine the best way forward in establishing a single common Community Format Registry (CFR).

CFR Functional [FR] and Non-Functional [NFR] Requirements

Note: In the following requirements the terms “will”, “must”, “shall”, and their grammatical cognates indicate a requirement; the terms “should”, “ought”, and their cognates indicate a recommendation; and the terms “can”, “may”, and their cognates indicate an option.

The requirements are presented in a hierarchical fashion, proceeding from the general to the specific in order to permit evaluation and acceptance/rejection at an arbitrary level of granularity.

Requirement priorities can be rated on the following scale: Very important, Important, Average, Slightly important, Not important.

0. Definition. A format is a class of digital object whose members all share a common set of syntactic and semantic rules controlling the mapping from an abstract information model [ISO 14721] to serialized byte streams, and in many useful instances, back again from serialized bytes to an abstract model. [NFR]

1. The CFR will manage a controlled namespace for the unambiguous persistent public identification of digital formats. [FR]

• 1.1 CFR identifiers are intended primarily for machine actionability. [NFR]

• 1.2 To the fullest extent that is consistent with meeting other imperative requirements, including but not limited to § 1.1, CFR identifiers should be amenable for human purposes and resilient against transcription error. [NFR]

2. The CFR will support the binding of various typed representation information [ISO 14721] to format identifiers. [FR]

• 2.1 This representation information should be capable of expressing the descriptive, administrative, syntactic, semantic, and behavioral properties of formats pertinent to curation and preservation analysis, decision making, and activities. [FR]

• • 2.1.1 Descriptive representation information will include: a canonical CFR identifier, as defined by § 1. [FR]

• • 2.1.2 Descriptive representation information should include: [FR]

• • • 2.1.2.1 An arbitrary number of namespaced identifiers publicly associated with the format, such as MIME type, PUID, GDFR identifier, Apple UTI, Library of Congress FDD identifier, standard identifier (ANSI, ECMA, FNOR, ISO, ITU, NISO, etc.), IETF RFC identifier, IANA identifier, W3C recommendation identifier, etc.
    • 2.1.2.2 An arbitrary number of common names publicly associated with the format.
    • 2.1.2.3 A format version identifier, as issued by the legitimate maintenance agency
    • 2.1.2.4 A short discursive description of the salient properties of the format.
    • 2.1.2.5 A format classification providing a means to indicate a format's (using terminology drawn from the GDFR Format Classification):

• • • • 2.1.2.5.1 Genre
      • 2.1.2.5.2 Role
      • 2.1.2.5.3 Composition
      • 2.1.2.5.4 Encoding form
      • 2.1.2.5.6 Constraint
      • 2.1.2.5.7 Basis
      • 2.1.2.5.8 Domain
      • 2.1.2.5.9 Transformative nature

• • • 2.1.2.6 An arbitrary number of typed relationships to other formats, including (using terminology drawn from the GDFR Format Model and Relationships):

• • • • 2.1.2.6.1 Affinity
      • 2.1.2.6.2 Containment
      • 2.1.2.6.3 Definition (Average: BnF)
      • 2.1.2.6.4 Extension
      • 2.1.2.6.5 Modification
      • 2.1.2.6.6 Requisition
      • 2.1.2.6.7 Restriction
      • 2.1.2.6.8 Semantic equivalence
      • 2.1.2.6.9 Syntactic equivalent
      • 2.1.2.6.10 Version

• • • 2.1.2.7 An arbitrary number of informative notes.

• • 2.1.3 Administrative representation information should include: [FR]

• • • 2.1.3.1 The corporate or individual agent(s) that created the format.
    • 2.1.3.2 The corporate or individual agent(s) that hold the intellectual property rights to the format
    • 2.1.3.3 The corporate or individual agent(s) responsible for format maintenance
    • 2.1.3.4 The format creation or release date
    • 2.1.3.5 The format withdrawal date
    • 2.1.3.6 An indication of the IPR status of the format.
    • 2.1.3.7 An arbitrary number of informative notes documenting format administrative properties.

• • 2.1.4 Syntactic representation information should include: [FR]

• • • 2.1.4.1 Arbitrary number of typed external signatures
    • 2.1.4.2 Arbitrary number of internal signatures
    • 2.1.4.3 An indication of byte ordering: big-endian, little-endian, either, both, or unknown
    • 2.1.4.4 An arbitrary number of format grammars expressed in some formal notation, such as ABNF, BSDL, DFDL, EAST, XCEL, etc.
    • 2.1.4.5 An arbitrary number of example files. (Important: FCLA; Average: Harvard, Portico)
    • 2.1.4.6 An arbitrary number of informative notes documenting format syntactic properties.

• • 2.1.5 Semantic representation information should include: [FR]

• • • 2.1.5.1 An arbitrary number of specification documents. (Very important: FCLA, Harvard, Portico)
    • 2.1.5.2 An arbitrary number of format assessments expressed in some formal notation, such as Library of Congress FDD, etc. (Average: BnF)
    • 2.1.5.3 An arbitrary number of informative notes documenting format semantic properties.

• • 2.1.6 Behavioral representation information should include: [FR]

• • • 2.1.6.1 An arbitrary number of format software dependencies

• • • • 2.1.6.1.1 Software can be described in terms of: [FR]

• • • • • 2.1.6.1.1.1 Name
        • 2.1.6.1.1.2 Version
        • 2.1.6.1.1.3 Description
        • 2.1.6.1.1.4 Creating agent(s)
        • 2.1.6.1.1.5 Maintenance agent(s)
        • 2.1.6.1.1.6 Release date
        • 2.1.6.1.1.7 Withdrawal date
        • 2.1.6.1.1.8 An arbitrary number of specification documents
        • 2.1.6.1.1.9 An arbitrary number of hardware dependencies
        • 2.1.6.1.1.10 An arbitrary number of files
        • 2.1.6.1.1.11 An arbitrary number of format defined characteristics
        • 2.1.6.1.1.12 An indication of the format-specific function, such as creation, transformation, rendering, etc.
        • 2.1.6.1.1.13 An indication of the IPR status, such as copyright, license, etc.
        • 2.1.6.1.1.14 An arbitrary number of informative notes documenting the software.
        • 2.1.6.1.1.15 An arbitrary number of informative notes documenting community experience with the software.
      • 2.1.6.1.2 Dependent software may have its own hardware and media dependencies.

• • • 2.1.6.2 An arbitrary number of format hardware dependencies.

• • • • 2.1.6.2.1 Hardware can be described in terms of: [FR]

• • • • • 2.1.6.2.1.1 Name
        • 2.1.6.2.1.2 Version
        • 2.1.6.2.1.3 Description
        • 2.1.6.2.1.4 Creating agent(s)
        • 2.1.6.2.1.5 Maintenance agent(s)
        • 2.1.6.2.1.6 Release date
        • 2.1.6.2.1.7 Withdrawal date
        • 2.1.6.2.1.8 An arbitrary number of specification documents
        • 2.1.6.2.1.9 Hardware type, such as CPU, dongle, media drive, sound card, video card, etc.
        • 2.1.6.2.1.10 An indication of the IPR status of the hardware.
        • 2.1.6.2.1.11 An arbitrary number of informative notes documenting the hardware.

• • • • 2.1.6.2.2 Dependent hardware may have its own software and media dependencies.

• • • 2.1.6.3 An arbitrary number of format media dependencies

• • • • 2.1.6.3.1 Media can be described in terms of: [FR]

• • • • • 2.1.6.3.1.1 Name
        • 2.1.6.3.1.2 Version
        • 2.1.6.3.1.3 Description
        • 2.1.6.3.1.4 Creating agent(s)
        • 2.1.6.3.1.5 Maintenance agent(s)
        • 2.1.6.3.1.6 Release date
        • 2.1.6.3.1.7 Withdrawal date
        • 2.1.6.3.1.8 An arbitrary number of specification documents
        • 2.1.6.3.1.9 Media type, such as flexible magnetic disk, rigid magnetic disk, optical disk, magnetic tape cartridge, magnetic tape reel, etc.
        • 2.1.6.3.1.10 Other significant media characteristics such as access type (e.g. read/write, read-only, etc.), capacity, read/write speed, coercity, etc.
        • 2.1.6.3.1.11 An indication of the IPR status of the media.
        • 2.1.6.3.1.12 An arbitrary number of informative notes documenting the media.

• • • • 2.1.6.3.2 Dependent media may have its software and hardware dependencies.

• • • 2.1.6.4 An arbitrary number of informative notes documenting format behavioral properties.

• • 2.1.7 Agents can be either corporate or individual. [NFR]

• • • 2.1.7.1 All agents can be described in terms of: [FR]

• • • • 2.1.7.1 Name
      • 2.1.7.2 Agent type: corporate or individual
      • 2.1.7.3 Postal address
      • 2.1.7.4 Telephone number
      • 2.1.7.5 Fax number
      • 2.1.7.6 Email address
      • 2.1.7.7 URI
      • 2.1.7.8 An arbitrary number of informative notes documenting agent properties.

• • • 2.1.7.2 Individual agents can also be described in terms of: [FR]

• • • • 2.1.7.2.1 Job title
      • 2.1.7.2.2 Corporate affiliation.

• • 2.1.8 Documents can be described in terms of: [FR]

• • • 2.1.8.1 Title
    • 2.1.8.2 Edition
    • 2.1.8.3 Authoring agent(s)
    • 2.1.8.4 Publishing agent(s)
    • 2.1.8.5 Date of publication
    • 2.1.8.6 An arbitrary number of formal identifiers, such as DDC, ICC, IEC, IETF BCP, IETF RFC, IETF STD, ISBN, ISO, ITU, LCCN, OCLC number, SICI, etc.
    • 2.1.8.7 Document language
    • 2.1.8.8 Document type
    • 2.1.8.9 An indication of the IPR status of the document.
    • 2.1.8.10 An arbitrary number of informative notes documenting document properties.
    • 2.1.8.11 An arbitrary number of files that contain manifestations of the document content.

• • 2.1.9 Files can be described in terms of: [FR]

• • • 2.1.9.1 Name
    • 2.1.9.2 File type, such as data, executable, object code, source code, etc.
    • 2.1.9.3 An arbitrary number of typed message digest values, such as CRC-32, MD5, SHA-1, SHA-256, etc.
    • 2.1.9.4 An indication of the IPR status of the file.
    • 2.1.9.5 Agent(s) who hold a copy of the file.

• • • • 2.1.9.5.1 Local holdings are described in terms of:

• • • • • 2.1.9.5.1.1 A locally-meaningful identifier.
        • '2.1.9.5.1.2[ An indication of public accessibility to the file.

• • • 2.1.9.6 An arbitrary number of informative notes documenting file properties.

• • 2.1.10 IPR status can be described in terms of: [FR]

• • • 2.1.10.1 The agent holding the rights.
    • 2.1.10.2 The effective date of the rights claim.
    • 2.1.10.3 The expiry date of the rights claim.
    • 2.1.10.4 The legal jurisdiction in which the rights claim is made.
    • 2.1.10.5 The type of rights claim, such as copyright, patent, trade secret, etc.
    • 2.1.10.6 License terms of use for the items covered under the rights claim.
    • 2.1.10.7 An arbitrary number of informative notes documenting the rights claim.

• 2.2 All representation information will be tagged with provenance information sufficient to provide a complete audit trail of changes over time. [FR] (Very important: DNB, FCLA, Harvard, Portico)[NLNZ]

• • 2.2.1 Representation information provenance will be described in terms of: a date/time stamp; and the change-initiating agent. [FR]

• • 2.2.2 For changes to existing representation information, provenance can also be described in terms of information sufficient to retrieve the previous value unambiguously. [NFR]

• 2.3 All representation information will be tagged to indicate the level of centrally-coordinated review by the CFR governing authority or its designees. [FR] (Very important: DNB)

• 2.4 Some representation information may be suppressed from public view for a limited embargo period in order to protect proprietary, trade secret, or other legally-encumbered information. [FR]

• • 2.4.1 The embargo period will be determined by the CFR governing authority. [NFR]

3. The CFR will follow evolving best practices for the secure, sustainable management of format representation information. [NFR]

• 3.1 CFR representation information should be replicated at one or more geographically dispersed locations, or CFR nodes. [FR]

• • 3.1.1 The network of CFR nodes can be used to provide high-availability access to managed representation information. [NFR] (Very important: DNB, FCLA, Important: Harvard)

• • 3.1.2 CFR replication will occur periodically to synchronize the holdings of representation information in the various nodes. [FR]

• • 3.1.3 Access to a CFR node can occur without that node having network connectivity to the other nodes of the network. [FR] (Very important: BnF; Slightly important: Harvard)

• • 3.1.4 A CFR node may locally-manage format representation information that is not replicated to other nodes. [FR] (Very important: Portico; Important: BnF; Not sure: FCLA; Not important: Harvard)

• 3.2 CFR nodes will provide end-user interfaces for discovery and delivery of managed representation information for human and automated agents. [FR] (Very important: DNB, Harvard)

---

Other Requirements

Please indicate below each requirement the importance of it to your institution using these words (in decreasing order of importance):
Very Important, Important, Average, Slightly Important, Not Important

• Store sample files for each format created by different applications (e.g. a PDF produced by Adobe PDF Library 4.8)
  • Average [Harvard] [Portico]
  • Important [FCLA]
  • Important [NLNZ]
• Store format specifications
  • Very Important [Harvard] [Portico][FCLA][NLNZ]
• Possibility to have a local mirror of the registry, that can be used offline.
  • Very Important [Bibliothèque nationale de France] [Portico][FCLA][NLNZ]
  • Slightly Important [Harvard]
• Ability to manage in the local mirror of the registry local information that is not supposed to be shared with other mirrors. There are two goals for this requirement: manage in the local mirror information that the institution has no right to share (information on software for example), manage information on local preferences (format preferences, local policies…).
  • Very Important [Portico][NLNZ]
  • Important [Bibliothèque nationale de France]
  • Not Sure [FCLA] Will we ever have the case where request for new format entry to GDFR is denied?
  • Not Important [Harvard]
• For local mirrors: ability to express that the type of a file is “local profile”. A “local profile” is a file used by a specific institution to describe its local policies and to assess compliancy of ingested files with these policies. For example, a TEI profile defined by a specific institution to express how a XML file shall be encoded according to institution needs. There are two goals for this requirement: manage this information in the local mirror of the registry, share this information with other interested institutions.
  • Important [Bibliothèque nationale de France][NLNZ]
• For local mirrors: ability to express value assessment and priorities for using formats. For example: prefer .rtf than .doc, prefer TIFF than JPEG.
  • Average [Bibliothèque nationale de France]
• Ability to express that the genre of a format is “metadata” (e.g. Dublin Core, MIX…)
  • Average [Bibliothèque nationale de France]
• Ability to express that the type of a file is “metadata”
  • Average [Bibliothèque nationale de France]
• Ability to express that the genre of a format is “stylesheet” (e.g. CSS, XSLT…)
  • Average [Bibliothèque nationale de France]
• Ability to express that the type of a file is “stylesheet”
  • Average [Bibliothèque nationale de France]
• Ability to express a “description” relationship between two formats, that is the fact that files in a metadata format can describe files in another format. For example, files in TextMD format can describe files in ALTO format. For example, files in MIX format can describe files in TIFF format.
  • Average [Bibliothèque nationale de France]
  • Important [NLNZ]
• Storage of format representation information of as many different formats as possible
  • Very Important [Deutsche Nationalbibliothek][Harvard] [Portico][FCLA][NLNZ]
• Distributed storage, discovery, and delivery of representation information, providing high availability at any time from any place
  • Very Important [Deutsche Nationalbibliothek][FCLA][NLNZ]
  • Important [Harvard] - the high availability is important to us, not necessarily the distributed part
• End-user service interfaces for discovery and delivery of representation information to human and automated agents (standardized web service interfaces)
  • Very Important [Deutsche Nationalbibliothek][Harvard]
• Storage of representation information on the base of a common well defined and transparent documented data model including at the minimum the data listed in the GDFR Analysis Model, v2.0, section 3.1.2
  • Very Important [Deutsche Nationalbibliothek][FCLA]
• Keeping full change history of all properties stored, versioning of records
  • Very Important [Deutsche Nationalbibliothek][Harvard][Portico][FCLA][NLNZ]
• Centrally-organized collection of format representation information
  • Very Important [Deutsche Nationalbibliothek]
• Centrally-organized editing, review and auditing of representation information
  • Very Important [Deutsche Nationalbibliothek]
• Guaranteed public access to the representation information
  • Very Important [Deutsche Nationalbibliothek][FCLA][NLNZ]
• Centrally-organized but cooperative accounted for auditing of the format registry system itself
  • Very Important [Deutsche Nationalbibliothek]
• Support for the preservation strategies migration and emulation by connecting to a software repository and additional information about view paths
  • Very Important [Deutsche Nationalbibliothek][NLNZ]
  • Average [FCLA]
  • Slightly Important[Harvard]
• Support for preservation of web archives by additional information about web browsers as specific rendering environments
  • Very Important [Deutsche Nationalbibliothek][NLNZ]
  • Average [FCLA]

Use Cases

---

Use Case Source: Bibliothèque nationale de France

• Very Important [Bibliothèque nationale de France]

Use Case ID: PRES_1

Description: Assemble Information Representation metadata

Summary: The goal of this use case id to allow a preservation expert to add information representation on formats (using an external format registry) so that this information is kept in perpetuity and could be referenced in the SPAR system itself.

Actors: Preservation expert, Format registry, Ingest module

Assumptions: The information representation metadata comes in the form of a SIP. The data-object is made of the description of the information representation.

Pre-conditions: Request stored in historical database.

Primary Functional Path:

1. Actor authenticates with the system.
2. Elaboration of a SIP with the information coming from the format registry
3. Ingest of this SIP in the system (ING_2).
4. Receipt of the external identifier of this information representation for latter use (policy, process, ...).

Primary Result: AIP of the information representation metadata

Post-conditions:

Exceptional paths:

1. The SIP is not valid according to the QA process: a non-conforming receipt is sent to the actor.
2. Failure in the storage transaction: a failure receipt is sent to the actor and a trap is sent to the administration.

---

Use Case Source: Bibliothèque nationale de France

• Very Important [Bibliothèque nationale de France]

Use Case ID: PRES_4

Description: Harvest Information Representation metadata

Summary: This use case allows the collect in a regular basis of information representation metadata from a format registry.

Actors: Format registry, Ingest module

Assumptions: Frequency of the collect

Pre-conditions: The AIP of the information representation already exists.

Primary Functional Path:

1. Connect to the format registry
2. Collect the new information representation metadata from the format registry.
3. Lookup for the corresponding AIP
4. For each new format with a corresponding modified,
5. Build a SIP
6. Submit the updated SIP (ING_2)

Primary Result: Information representation AIP updated

Post-conditions:

Exceptional paths:

1. SIP refused: send a rap to the administration.

---

Use Case Source: Bibliothèque nationale de France

• Very Important [Bibliothèque nationale de France]

Use Case ID: ADM_4

Description: Assemble process metadata

Summary: The process metadata are stored in the archive as AIP; so they are imported as any other SIP. The goal of this use case is to allow a IT staff to add information on available processes so that this information can be made persistent and can be referred to in the SPAR system itself.

Actors: IT staff, Ingest module

Assumptions: The process metadata comes in SIP format. The data object can be the specification of the process or even the source code of the program that realizes this process.

Pre-conditions: Request is stored in historical database.

Primary Functional Path:

1. Actors authenticates with the system.
2. Receive the process metadata in SIP format.
3. Import this SIP in the system (ING_2)
4. Send receipt of the import to the actor: the receipt contains the external identifier of the process for a latter reference or use (transformation packages, AIP generation, DIP generation, …).
5. Store record of the operation in historical database

Primary Result: AIP of a particular process

Post-conditions:

Exceptional paths:

1. The SIP is not valid according to the QA for a process: a non-conforming receipt is delivered to the actor.
2. Failure in the storage transaction: a failure receipt is delivered and a trap is sent to the Administration

---

National Library of New Zealand Thoughts

Our understanding of what the CFR will need to do is informed by our current thinking on the concept of format obsolescence and how to analyse the risk of obsoleteness coming to pass. At the National Library of New Zealand, we have written some documents on this. One is an informal Risk discussion paper that outlines some of our current world view.

Some highlights of the document:

1. NLNZ is obliged to take in content regardless of its conformance to format standards (i.e. if it fails DROID and JHOVE, it still has to come in).
2. Our initial risk assessment is whether we as an institution can render it or not. That is, is the format associated with an application within our tech profile?
3. Characteristics within a format (that is, certain compression methods, colourspaces, etc) can prohibit an application from rendering the content.
4. To achieve this, there should be a Format Library, an Application Library and a Characteristics Library, all interrelated, telling us what we can and cannot render successfully.
5. Formats, Applications and Characteristics will all have 'sustainability information' associated with them -- is there a vendor support end date, how stable is it, etc. This information is used to assess a) when we should move content from a certain format; and b) what format/application/characteristic profile we could move the content into.

Comment on section above:

Point 0 above: "Definition: A format is a class of digital object whose members all share a common set of syntactic and semantic rules controlling from an abstract information model [ISO 14721] to serialized byte streams, and in many useful instances, back again from serialized bytes to an abstract model. [NFR]"

This statement describes formats (and the files that represent their encodings) from the view of the format originator or creator. It does not appear to account for the fact that application developers (file encoders) interpret these syntactic and semantic rules and frequently either wittingly or unwittingly, alter them to suit their own purposes when writing byte streams.

So, to a repository preservation manager, while the definition of the format in its idealized state is of course important, equally important are the quirks regarding how different applications actually encode files. We've got a few RTFs that don't fully comply with the spec for the format, but we can happily open them in a number of applications. They are not versions of the format, rather badly encoded files where a number of applications are "forgiving" enough to still open them properly. It seems that we would want to record this information somewhere like the GDFR and gather together the community's knowledge in this area as well.

This relates to the NLNZ validation routine and our institutional requirement that we accept files even if they don't conform with a format standard. From our point of view it would be desirable to have JHOVE(2) capture what technical metadata it can even if a file does not meet the requirements around "validity" or "well-formedness".

CFR Scope

In terms of CFR requirements, the actual scope has to be defined first (apologies if this has already been defined in previous meetings). How much/little of the list below is envisaged as being within the remit of the CFR? Will it:

1. store information that can be used to help identify how files are encoded (their format & version)?
2. store information that can be used to understand the ‘significant properties’ of objects (we prefer the term ‘characteristics’)?
3. describe the creating application of the content (and the related computing environment (with any hardware or software dependencies))?
4. list applications that can render formats and their variations, taking account of characteristics that can hinder this?
5. supply risk analysis of formats/applications/characteristics?
6. describe appropriate (and inappropriate) preservation strategies?
7. hold information to be used in the evaluation of preservation actions?

First and foremost we see the primary objective as being point 1. The unequivocal description of an object in terms of its format and version is the ideal goal. To get to any of the other parts (risk analysis, preservation strategies) this information has to be complete, reliable and absolutely comprehensive.

Our point of view is that all major characteristics of an object must be known before any sort of action can be taken on it (for example, a TIFF image has CIELab encoding -- run this blindly through a TIFF-JPEG2000 converter and you get a lovely bright green result). While we understand that such depth of detail (point 2) may be a step too far for the CFR, trustworthy information at the format and version level will help us fill in half the picture (no mean feat). [We are working on this characteristics aspect with great vigour.]

In addition, points 3&4 could be achievable too. This means we would have a baseline of rendering information for the content (even ignoring for the moment the relationship with characteristics of the objects that could stop it being rendered 'properly').

In terms of risk, we're nervous about community consensus on this. Current models seems to favour numeric analysis with certain results then triggering planning or actions. We do not see the underlying information being comprehensive enough, and nor do we see consistent interpretation of this information into numerals (for example, how exactly do you put 'Market Share' into a scale of 1-5?) At the moment, we believe that this should be tackled on an institutional level. This is not to say that we do not think this information should be shared/discussed/argued. It should be and as widely as possible. However, the CFR may not be the place to do that while such analysis is sill in its infancy.

Point 6 is the community dream: towards full automation of mature, tried and tested preservation actions. Could work on this part stifle the more immediate concerns of point 1? If possible, placeholders could be made in the CFR structure for this.

Point 7 is related to characteristics (significant properties) and could be a step too far at this point with this level of effort. We are working on this though (as are others).

View source | Discuss this page | Page history | What links here | Related changes Main Page | About GDFR Wiki | Find: This page has been accessed 3,812 times. This page was last modified 02:20, 16 January 2009.