--- storage/meta/meta-format.tex 2003/07/23 20:34:53 1.5 +++ storage/meta/meta-format.tex 2010/01/28 17:39:47 1.20 @@ -7,16 +7,16 @@ %\usepackage{courier} % create in-text links black (with PDF) -%\usepackage[colorlinks=true,linkcolor=black]{hyperref} +\usepackage[colorlinks=true,linkcolor=black]{hyperref} % Format URLs nicely (without PDF) -\usepackage{url} +%\usepackage{url} \title{A simple metadata format for resource bundles} \author{Robert Casties, Dirk Wintergrün, Hans-Christoph Liess} -\date{V0.3 of \today} +\date{V1.3.4 of 24.7.2008} \begin{document} @@ -32,8 +32,22 @@ File and directory names should not cont in filenames are only the alphanumeric set a-z, A-Z, 0-9, hyphen ``-'', underscore ``\_'' and dot ``.''. -File and directory paths in the metadata file use the conventional -Unix file separator slash ``/''. +Files and directories with names that contain illegal characters must +be transformed to allowed names. A proposition for a simple +transformation rule is + +\begin{itemize} +\item whitespace characters (e.g. blank, tab, cr, lf) are replaced by + hyphens ``-'' + +\item other illegal characters are replaced by underscores ``\_''. +\end{itemize} + +This rule does not provide a reversible mapping to the original +illegal file name and it does not provide a collision-free mapping, +i.e. two different illegal file names might be mapped to the same +allowed file name. Additional precautions for these cases must be +taken. \section{Metadata files} @@ -45,20 +59,20 @@ files are possible: \begin{itemize} \item a file named \texttt{index.meta} in a directory. -\item a file named like the data file it describes with an +\item a file with the same name as the data file it describes and an additional extension \texttt{.meta}. For example metadata for the - file \texttt{0001.tif} would be in a file \texttt{0001.tif.meta}. + file \texttt{p0001.tif} would be in a file \texttt{p0001.tif.meta}. \end{itemize} The resource directory must contain an \texttt{index.meta} file with -information about the resource as a whole. Other directories can -contain \texttt{index.meta} files. +information about the resource as a whole. Subdirectories can +contain additional \texttt{index.meta} files. Additional information about single data files that are part of the resource can either be put in \texttt{file} tags in the \texttt{index.meta} file or in separate \emph{filename}\texttt{.meta} files for each data file. Information from the directory level file is -inherited at the file level. +inherited at the file level when it is not overwritten. \section{Resource format} @@ -72,13 +86,28 @@ supplied by the provider of the resource automatic scripts later in the process, these elements must be present in the final file. -The outer container element is \texttt{resource}. Sub-types (``ECHO'', -``MPIWG'') can be specified if necessary with a \texttt{type} -parameter. Its sub-elements are: +File and directory paths in the metadata file use the conventional +Unix file separator slash ``/''. + +The outer container element is \texttt{resource}. It has the following +\textbf{attributes}: \begin{description} -\item[description] An informal textual description of the - resource -- optional. +\item[type] sub-type of resource (e.g. ``ECHO'', ``MPIWG'') -- + optional. + +\item[version] version number of metadata format (currently 1.2) -- + required. +\end{description} + +\noindent The allowed \textbf{elements} inside \texttt{resource} are: + +\begin{description} +\item[description] An informal textual description of the resource -- + optional\footnote{At least one description of the resource's content + is required. The description can be an informal + \texttt{description} element or a descriptive element (like + \texttt{bib}) in a \texttt{meta} container.}. \item[name] The filename of the resource (name of the directory this file is contained in) -- required. @@ -94,14 +123,35 @@ parameter. Its sub-elements are: \item[archive-path] The full path to the resource directory inside the whole archive collection, including the resource directory -- deduced. + +\item[archive-id] The ID for this document in the archive -- + optional. \item[derived-from] Container for the description of the original resource if this resource is a modified version of another resource -- optional. \begin{description} + \item[archive-id] The ID of the original resource + -- required (or archive-path). + \item[archive-path] The full path to the original resource - --required. + -- required (or archive-id). + + \item[description] An informal textual description of the relation + of this resource to the original resource -- optional. + \end{description} + +\item[used-by] Container for the description of modified resources + if this resource is the source of another resource + -- optional. + + \begin{description} + \item[archive-id] The ID of the derived resource + -- required (or archive-path). + + \item[archive-path] The full path to the derived resource + -- required (or archive-id). \item[description] An informal textual description of the relation of this resource to the original resource -- optional. @@ -112,21 +162,30 @@ parameter. Its sub-elements are: -- optional. \begin{description} + \item[archive-id] The ID of the linked resource + -- required (or archive-path). + \item[archive-path] The full path to the linked resource - --required. + -- required (or archive-id). \item[description] An informal textual description of the relation of this resource to the linked resource -- optional. \end{description} -\item[content-type] The content type of this resource -- required.\\ - The content type enables the choice of tools to manipulate and - display the resource. There should be a common list of content - types. For digital documents (books, manuscripts) this would be - "scanned document", for other image data "scanned - images".\footnote{The criterion for documents is a ordered - succession of image files (pages) and equal image size and - resolution throughout the images of a resource.} +\item[media-type] \label{tag-media-type} The main media type of this + resource -- required.\\ The main media type can be overridden by + \texttt{media-type}s in subdirectories. Possible types are + \begin{itemize} + \item \texttt{image} + + \item \texttt{text} + + \item \texttt{audio} + + \item \texttt{video} + + \item \texttt{data} for other type of data + \end{itemize} \item[meta] Additional metadata information about the resource -- optional.\\ For a description of additional metadata see below. @@ -142,6 +201,12 @@ parameter. Its sub-elements are: \item[name] The name of the subdirectory -- required. + \item[original-name] A text string associated with the directory as + original name -- optional. (E.g. if the data in this directory + came from an external source and had a name that had to be changed + according to section~\ref{sec:file-directory-names} but it should + be possible to reference the original name.) + \item[path] The directory path of this subdirectory relative to the resource's root directory (excluding the directory itself) -- required (may be empty or omitted if the directory is a direct @@ -161,18 +226,25 @@ parameter. Its sub-elements are: \item[name] The name of the file -- required. + \item[original-name] A text string associated with the file as + original name -- optional. (e.g. if this file came from an + external source and had a name that had to be changed according to + section~\ref{sec:file-directory-names} it is possible + to preserve the original name.) + \item[path] The directory path of this file relative to the resource's root directory (excluding the file itself) -- required (may be empty or omitted if the file is in the resource's root directory). + + \item[date] The file's modification or creation date\footnote{The + preferred time and date format is ``YYYY/MM/DD HH:MM:SS''}, + whichever is more recent -- optional. \item[modification-date] The file's modification date -- optional. \item[creation-date] The file's creation date -- optional. - - \item[date] The file's creation date if is has not been modified -- - optional. - + \item[size] The file size -- deduced. \item[mime-type] The file's mime-type -- optional. @@ -191,7 +263,48 @@ parameter. Its sub-elements are: \label{sec:additional-metadata} All elements with \texttt{meta} tags can contain an arbitrary number -of additional metadata elements. +of the following additional metadata elements. + +\subsection{Workflow state} +\label{sec:workflow-state} + +All additional metadata elements can have a \texttt{workflow-state} +\textbf{attribute}. This attribute reflects the state of the +corresponding metadata element. The possible values for the +\texttt{workflow-state} attribute are +\begin{itemize} +\item \texttt{preliminary} this information is preliminary. It must + be checked in further workflow steps. + +\item \texttt{inwork} + +\item \texttt{final} +\end{itemize} + +workflow states other than \texttt{preliminary} are part of the +workflow handling of the respective projects. + +Metadata elements can appear multiple times with different +\texttt{workflow-state} attributes. This enables metadata versioning. + + + +\subsection{Content type} +\label{sec:content-type} + +\begin{description} +\item[content-type] \label{tag-content-type} The content type of this + resource -- required.\\ + The content type enables the choice of tools to manipulate and + display the resource. There should be a common list of content + types. For digital documents (books, manuscripts) this would be + "scanned document", for other image data "scanned + images".\footnote{The criterion for documents is a ordered + succession of image files (pages) and equal image size and + resolution throughout the images of a resource.} +\end{description} + + \subsection{Language} \label{sec:lang} @@ -219,17 +332,49 @@ on the page \subsection{Collection context} \label{sec:collection-context} -The context of a resource as part of a collection or part of a project can be -specified in the \texttt{context} element. All elements in the -container can appear multiple times. +The context of a resource as part of a collection or part of a project +can be specified in the \texttt{context} element. The context element +can appear multiple times if the resource is part of multiple +collections or projects. \begin{description} \item[context] information on collection or project context. \begin{description} - \item[link] URL to additional context information. + \item[link] URL to additional context information -- optional. - \item[name] Textual description of project or collection. + \item[name] Textual description of project or collection -- optional. + + \item[meta-datalink] description of external sources of canonical meta + information -- optional + \begin{description} + \item[db] \textbf{attribute} to identify different sets of meta data + links to the same resource -- optional + + \item[object] \textbf{attribute} to identify different objects or + parts of the same resource -- optional + + \item[label] textual label for the link -- optional + + \item[url] URL to present to the client -- optional + + \item[metadata-url] URL to an external server to be queried -- optional + \end{description} + + \item[meta-baselink] description of external server for canonical meta + information -- optional + \begin{description} + \item[db] \textbf{attribute} to identify different sets of meta data + links to the same resource -- optional + + \item[label] textual label for the link -- optional + + \item[url] URL to present to the client -- optional + + \item[metadata-url] URL to an external server to be queried -- + required (the parameter \texttt{object=} with an object id has + to be appended to this URL) + \end{description} \end{description} \end{description} @@ -272,6 +417,8 @@ the commonly used program ``EndNote''. published in multiple volumes. \item [translator] Name of the translator. \item [isbn-issn] + \item[call-number] Call number in holding library + \item[holding-library] Holding library \end{description} \end{description} @@ -300,6 +447,8 @@ the commonly used program ``EndNote''. published in multiple volumes. \item [translator] Name of the translator \item [isbn-issn] + \item[call-number] Call number in holding library + \item[holding-library] Holding library \end{description} \end{description} @@ -328,6 +477,8 @@ the commonly used program ``EndNote''. \item [number-of-volumes] Number of volumes, if the the book is published as multiple volumes. \item [isbn-issn] + \item[call-number] Call number in holding library + \item[holding-library] Holding library \end{description} \end{description} @@ -353,6 +504,26 @@ the commonly used program ``EndNote''. \item [number-of-volumes] Number of volumes, if the the book is published as multiple volumes. \item [isbn-issn] + \item[call-number] Call number in holding library + \item[holding-library] Holding library + \end{description} +\end{description} + +\subsubsection{Journal Volume} + +\begin{description} +\item [bib type="journal-volume"] a volume of a scientific journal. + \begin{description} + \item [title] Name of the journal. + \item [editor] The editor of the journal. + \item [publisher] Name of the publishing company. + \item [city] City where the journal is published. + \item [year] The year of publication. + \item [volume] Volume number. + \item [numer-of-pages] Number of pages of the volume. + \item [isbn-issn] + \item[call-number] Call number in holding library + \item[holding-library] Holding library \end{description} \end{description} @@ -370,6 +541,8 @@ the commonly used program ``EndNote''. \item [pages] Number of pages of the article. \item [alternate-journal] Alternate Journal \item [isbn-issn] + \item[call-number] Call number in holding library + \item[holding-library] Holding library \end{description} \end{description} @@ -386,6 +559,8 @@ the commonly used program ``EndNote''. \item [issue-number] Number of the issue the article is part of. \item [pages Number] of pages of the article. \item [date] Date when the article appeared. + \item[call-number] Call number in holding library + \item[holding-library] Holding library \end{description} \end{description} @@ -401,6 +576,8 @@ the commonly used program ``EndNote''. \item [pages] Number of pages of the article. \item [issue-date] Date of the issue the article is part of. \item [city] City of the newspaper. + \item[call-number] Call number in holding library + \item[holding-library] Holding library \end{description} \end{description} @@ -419,6 +596,8 @@ the commonly used program ``EndNote''. \item [University] Name of the university where the thesis was handed in. \item [isbn-issn] + \item[call-number] Call number in holding library + \item[holding-library] Holding library \end{description} \end{description} @@ -436,6 +615,8 @@ the commonly used program ``EndNote''. \item [institution] Institution where the report was produced. \item [type] Type of report. \item [report-number] Report number. + \item[call-number] Call number in holding library + \item[holding-library] Holding library \end{description} \end{description} @@ -457,6 +638,34 @@ the commonly used program ``EndNote''. annotations etc. \item [description] This can be any kind of description. \item [keywords] Keywords related to the manuscript. + \item[call-number] Call number in holding library + \item[holding-library] Holding library + \end{description} +\end{description} + + +\subsubsection{Correspondence} + +\begin{description} +\item [bib type="correspondence"] a piece of correspondence e.g. letter, telegram, in the following called ``letter'' + + \begin{description} + \item[type] The type of correspondence, e.g. ``letter'', ``postcard'', ``telegram'', ``letter draft'' + \item [author] The author/sender of the letter. + \item [recipient] The recipient of the letter. + \item [date] normalised date of the letter. + \item [date-range-end] end of range of uncertain dating -- optional. + \item [date-original] the date in its original form as noted on the letter -- optional. + \item [place] place where the letter was written/sent. + \item [title] Title of the letter -- optional. + \item[incipit] The opening phrase of the letter -- optional. + \item[excipit] The closing phrase of the letter -- optional. + \item [pages] Number of pages of the manuscript. + \item [signature] Canonical signature/call number of the manuscript. + \item [description] This can be any kind of description. + \item [keywords] Keywords related to the manuscript. + \item[call-number] Call number in the current holding library + \item[holding-library] current holding library \end{description} \end{description} @@ -522,7 +731,7 @@ appear multiple times. \end{description} -\subsection{Information on the document structure (table of contents)} +\subsection{Document structure (table of contents)} \label{sec:toc} Information on the structure of a document like the division into @@ -605,7 +814,7 @@ tags. %%\url{http://pythia.mpiwg-berlin.mpg.de/toolserver/TS_lise} -\subsection{Information on scanned images} +\subsection{Digital images} \label{sec:inform-scann-imag} Image files representing scanned images can have an \texttt{img} @@ -619,16 +828,17 @@ Required is one of three possible sets o \item[img] digital image information. \begin{description} - \item[original-size-x] The width of the original image. The unit of - measure can be contained as parameter \texttt{unit}, the default - is meter ``m''. The width to be considered is the total width of - the scanned area. + \item[original-size-x] The width of the original + image -- required. \\ + The unit of measure can be contained as parameter \texttt{unit}, + the default is meter ``m''. The width to be considered is the + total width of the scanned area. - \item[original-size-y] The height of the original image. + \item[original-size-y] The height of the original image -- required. - \item[original-pixel-x] The width of the hi-res scan in pixels. + \item[original-pixel-x] The width of the hi-res scan in pixels -- deduced. - \item[original-pixel-y] The height of the hi-res scan in pixels. + \item[original-pixel-y] The height of the hi-res scan in pixels -- deduced. \end{description} \end{description} @@ -639,10 +849,14 @@ or \begin{description} \item[original-dpi-x] The resolution of the hi-res scan in its width - in pixels per inch. + in pixels per inch -- required. \item[original-dpi-y] The resolution of the hi-res scan in its height - in pixels per inch. + in pixels per inch -- required. + + \item[original-pixel-x] The width of the hi-res scan in pixels -- deduced. + + \item[original-pixel-y] The height of the hi-res scan in pixels -- deduced. \end{description} \end{description} @@ -653,18 +867,281 @@ or \begin{description} \item[original-dpi] The resolution of the hi-res scan in pixels per - inch if the resolutions in width and height are the same. + inch if the resolutions in width and height are the same -- required. + + \item[original-pixel-x] The width of the hi-res scan in pixels -- deduced. + + \item[original-pixel-y] The height of the hi-res scan in pixels -- deduced. + \end{description} +\end{description} + + + +\subsection{Digital image acquisition} +\label{sec:inform-about-image} + +A description of the technology used in the process of producing a +digital image. + +\begin{description} +\item[image-acquisition] description of the image production process + \begin{description} + \item[device] acquisition device (e.g. ``flatbed scanner'') + + \item[image-type] type and color-depth of the image -- required (e.g. ``RGB 24 + bit'') + + \item[production-comment] additional textual information about the + production process + \end{description} +\end{description} + + + +\subsection{Full text with images} +\label{sec:full-text-with} + +Full text in a XML format should be specified with a +\texttt{content-type}\footnote{see section~\ref{tag-content-type} +on page\pageref{tag-content-type}} ``fulltext''. + +The relation between the full text and optional images of +whole pages or parts of pages must be specified in a +\texttt{texttool} container. + +\begin{description} +\item[texttool] representation of full text with images + + \begin{description} + \item[text] the file name of the full text file (with path + inside document directory) + + \item[text-url-path] a characteristic part of the URL with which the + full text can be retrieved (the form and content of this element + is dependent on the specific text retrieval mechanism) + + \item[image] the directory name of the directory containig the + page image files (with path inside document directory) + + \item[xslt] the file name of an additional XSL transformation + file + + \item[pagebreak] the name of the element that indicates page breaks + (default ``pb'') + \end{description} +\end{description} + + + +\subsection{Copyright and access conditions} +\label{sec:access-conditions} + +If the access to a resource is bound to conditions for technical or legal +reasons then the conditions can be put in a \texttt{access-conditions} +container. Other usage conditions like copyright can also be +documented in this container. + +\begin{description} +\item[access-conditions] legal and technical conditions for access to + this resource + + \begin{description} + \item[attribution] The name or institution this resource should be + attributed to when it's publicly presented + + \begin{description} + \item[name] a name (free text) + + \item[url] a URL (with an optional \texttt{label} attribute to show + as text) + + \item[description] more information (free text, e.g. holding + library call number) + \end{description} + + \item[copyright] the copyright holder and it's conditions + \begin{description} + \item[owner] the name of the copyright holder + \begin{description} + \item[name] a name (free text) + + \item[url] a URL (with an optional \texttt{label} attribute to show + as text) + \end{description} + + \item[date] the date when the copyright was issued + + \item[duration] the duration of the copyright term (if known) + + \item[description] free-text field for special or additional + conditions + \end{description} + + + \item[publish-metadata] metadata about this resource can be made + freely available when this tag is present (otherwise metadata has + the same access conditions as the rest of the resource). Access to + the resource itself is regulated separately by the \texttt{access} + element. + + \item[access] conditions of access to this resource. Different + access types are specified by a \texttt{type} attribute: + \begin{description} + \item[type=group] access restricted to the members of this named + group. The method to identify a user belonging to a named group + is not specified in this document. + \begin{description} + \item[name] name of the group. + + \item[only-before] the access condition is only valid before the + given date (format: ``YYYY/MM/DD''). + + \item[only-after] the access condition is only valid after the + given date (format: ``YYYY/MM/DD''). + \end{description} + + \item[type=institution] access restricted to the members of this + institution. The method to identify a user to belong to the + institution is not specified in this document. + \begin{description} + \item[name] name of the group. + + \item[only-before] the access condition is only valid before the + given date (format: ``YYYY/MM/DD''). + + \item[only-after] the access condition is only valid after the + given date (format: ``YYYY/MM/DD''). + \end{description} + + + \item[type=subnet] access restricted to all computers with an + IP-address in this subnet. + \begin{description} + \item[range] subnet range defined in + truncated-quad (e.g. ``141.14''), network-netmask + (e.g. ``141.14.0.0/255.255.0.0''), or network-range + (e.g. ``141.14.0.0/16'') notation. + + \item[only-before] the access condition is only valid before the + given date (format: ``YYYY/MM/DD''). + + \item[only-after] the access condition is only valid after the + given date (format: ``YYYY/MM/DD''). + \end{description} + + + \item[type=scientific] access to this resource should be restricted to + scientific work + \begin{description} + \item[only-before] the access condition is only valid before the + given date (format: ``YYYY/MM/DD''). + + \item[only-after] the access condition is only valid after the + given date (format: ``YYYY/MM/DD''). + \end{description} + + + \item[type=free] access to this resource is not restricted + \begin{description} + \item[only-before] the access condition is only valid before the + given date (format: ``YYYY/MM/DD''). + + \item[only-after] the access condition is only valid after the + given date (format: ``YYYY/MM/DD''). + \end{description} + + + \item[type=special] if none of the above conditions seems appropriate, + a free-form text can be specified here. + \begin{description} + \item[description] description of special access conditions. + + \item[only-before] the access condition is only valid before the + given date (format: ``YYYY/MM/DD''). + + \item[only-after] the access condition is only valid after the + given date (format: ``YYYY/MM/DD''). + \end{description} + + \end{description} + \end{description} +\end{description} + +\noindent +It should be noted that control over access to the resource has to be +provided by additional technical measures. Access conditions in the +metadata file only state that conditions \emph{should} be observed, it +is not implied that they \emph{are} necessarily observed, as the +enforcement of conditions depends on additional measures. + + + +\subsection{Acquisition of raw-data} +\label{sec:acqu-inform} + +Information about the acquisition source for raw data resources can be +provided in an \texttt{acquisition} container. + +\begin{description} +\item[acquisition] the acquisition source of this resource -- required + for raw data. + \begin{description} + \item[provider] where this resource came from -- required + \begin{description} + \item[name] free-text name of the provider (institution or + individual) + + \item[address] address of the provider + + \item[contact] contact person at the provider (i.e. name and email) + + \item[url] URL related to the provider + + \item[provider-id] id of the provider (internally used) -- deduced + \end{description} + + \item[date] date of acquisition -- required + + \item[description] free-text description of the acquisition source or + additional information + \end{description} +\end{description} + + + +\subsection{Documentary Films} +\label{sec:documentary-films} + +Documentary films can be described using a \texttt{film-acquisition} +container. + +\begin{description} +\item[film-acquisition] description of a (documentary) film -- + required for documentary film + \begin{description} + \item[recording] specification of the recording process + \begin{description} + \item[author] the person or persons doing the recording + + \item[date] the date or time span when the film was recorded + + \item[location] the place where the film was recorded + + \item[device] recording device used (e.g. ``Sony CP-DV8 Camcorder'') + + \item[format] format of the recorded film -- required (e.g. ``DV + 720x524 25fps interlaced'') + \end{description} + + \item[description] free-form description of the recording and the + content of the film \end{description} \end{description} +(More information about the digitization step could be added in a +\texttt{digitization} tag similar to the \texttt{recording} tag.) -\subsection{Access restrictions} -\label{sec:access-restrictions} -If the access to a resource is restricted for technical or legal -reasons then the restrictions can be put in a -\texttt{access-restrictions} container. The format of the information -inside the container has to be further specified. \section{Sample metadata files for ECHO resources} @@ -674,7 +1151,7 @@ scanned document. \begin{small} \begin{verbatim} - + Fleck, 1980 fleck.1980 University of Bern @@ -715,7 +1192,7 @@ architectural drawing. \begin{small} \begin{verbatim} - + Bibliotheca Hertziana scanned images