wiki:mpdl2.0-design

Version 10 (modified by jwillenborg, 13 years ago) (diff)

--

MPDL 2.0 backend system: software design

The MPDL backend software is redesigned so that as much software as possible is separated from the content management system and could be used independently from it. We offer two MPDL librares:

  • XML technology (available as a web archive file "mpiwg-mpdl-xml.war")
  • language technology (available as a web archive file "mpiwg-mpdl-lt.war")

Dependant from the CMS we offer the MPDL CMS library:

  • CMS technology (available as a web archive file "mpiwg-mpdl-cms.war")

XML technology

The XML technology web archive ""mpiwg-mpdl-xml.war"" consists of:

  • Java code: mpiwg-mpdl-xml.jar
    • de.mpg.mpiwg.berlin.mpdl.xml.transform
    • de.mpg.mpiwg.berlin.mpdl.xml.xquery
    • de.mpg.mpiwg.berlin.mpdl.xml.pdf
    • de.mpg.mpiwg.berlin.mpdl.servlets.xml
  • external Java libraries
    • saxon9-s9api.jar
    • ...
  • web application configuration file: web.xml

Following HTTP servlets are available:

XSL transformation

  • TransformServlet
    • URL: /mpdl/transform
    • Request parameters:
      • srcUrl
        • source URL of XML document
      • xslUrl
        • URL of XSL document
      • parameters (optional)
        • list of parameters, e.g. "yourParam1=val1 yourParam2=val2"
      • outputProperties (optional)
        • list of output properties, e.g. "method=xhtml indent=no"
    • Response output:
      • transformed document (HTML, XML, etc.)
  • GetFragmentServlet
    • URL: /mpdl/getFragment
    • Request parameters:
      • srcUrl
        • source URL of XML document
      • ms1Name
        • first milestone name, e.g. "pb"
      • ms1Position
        • first milestone position, e.g. 1
      • ms2Name
        • second milestone name, e.g. "pb"
      • ms2Position
        • second milestone position, e.g. 2
    • Response output:
      • XML fragment

PDF rendering

  • RenderServlet
    • URL: /mpdl/render
    • Request parameters:
      • srcUrl
        • source URL of XML document
      • xslUrl
        • URL of XSL document
      • parameters (optional)
        • list of parameters, e.g. "yourParam1=val1 yourParam2=val2"
    • Response output:
      • rendered document (PDF)

XPath/XQuery

  • !XPathServlet
    • URL: /mpdl/xpath
    • Request parameters:
      • srcUrl
        • source URL of XML document
      • xpath
        • xpath source code
    • Response output:
      • XPath result for that document
  • !XQueryServlet
    • URL: /mpdl/xquery
    • Request parameters:
      • srcUrl
        • source URL of XML document
      • xquery
        • xquery source code
    • Response output:
      • XQuery result for that document

Language technology

The language technology web archive ""mpiwg-mpdl-lt.war"" consists of:

  • data (data files, Java BerkelyDB data)
    • morphology data (Perseus, CELEX, Lexique with languages: ara, eng, fre, ger, gre, ita, lat, nld, zho)
    • dictionary data (dictionaries: autenrieth, baretti, bonitz, cooper, florio, lewis-short, lidell-scott-jones, salmone, webster)
  • Java code: mpiwg-mpdl-lt.jar
    • de.mpg.mpiwg.berlin.mpdl.lt.*
    • de.mpg.mpiwg.berlin.mpdl.servlets.lt
  • external Java libraries
    • berkeley-db-3.3.82.jar
    • ...
  • web application configuration file: web.xml

Following HTTP servlets are available:

Morphology

  • TokenizeServlet
    • URL: /mpdl/tokenize
    • Request parameters:
      • srcUrl
        • source URL of fulltext
          • unstructured text
          • XML fragment/document
      • language (if available use xml:lang in XML document else this language value)
        • ISO 639-3 specifier
      • normalization (optional; default: without normalization)
        • "reg", "norm", "reg norm"
      • dictionary (optional; default: with dictionary)
        • "yes", "no"
      • stopElements (optional, default: empty)
        • elements which should not be analyzed and enriched (e.g. "lb")
      • outputFormat (optional; default: "xml")
        • "xml", "wordList"
    • Response output:
      • xml
        • document enriched by element <w>
          • e.g. <s><w lang="deu" reg="dies" norm="dies" forms="dies, dieser, dieses, diesen" lemmas="dieser" dictionaries="dwds">Dies</w> <w lang="deu" reg="ist" norm="ist" forms="bin, bist, ist, seid, sind, sein, war, warst, wart" lemmas="sein" dictionaries="dwds">ist</w> <w lang="deu" reg="ein" norm="ein" forms="ein, eines, einer" lemmas="ein" dictionaries="dwds">ein</w> <w lang="deu" reg="satz" norm="satz" forms="satz, sätze, satzes" lemmas="satz" dictionaries="dwds">Satz</w></s>
      • wordList
        • word tokens (separated by Blank)
        • word tokens (XML)
  • LemmaServlet
    • URL: /mpdl/getLemmas
    • Request parameters:
      • forms
        • one word form (string)
        • list of word forms (XML)
      • language
        • ISO 639-3 specifier
      • normalization (optional; default: without normalization)
        • "reg", "norm", "reg norm"
    • Response output:
      • lemmas
        • one lemma
        • list of lemmas (XML)
  • FormServlet
    • URL: /mpdl/getForms
    • Request parameters:
      • lemmas
        • one lemma (string)
        • list of lemmas (XML)
      • language
        • ISO 639-3 specifier
      • normalization (optional; default: without normalization)
        • "reg", "norm", "reg norm"
    • Response output:
      • forms
        • list forms (XML)

Dictionary

  • WordServlet
    • URL: /mpdl/getDictionaryEntries
    • Request parameters:
      • forms or lemmas or range
        • one form or lemma (string)
        • list of forms or lemmas (XML)
        • range
          • entries beginning with, e.g. "a*"
          • entries from position x to y: e.g. "200-300"
      • inputType (optional; default: "form")
        • "form", "lemma"
      • dictionary (optional; default: all dictionaries)
        • dictionary name, e.g. "webster"
      • language (optional, default: all languages)
        • ISO 639-3 specifier
      • outputType (optional)
        • "full", "compact"
      • outputFormat (optional)
        • "xml", "html"
      • normalization (optional; default: without normalization)
        • "reg", "norm", "reg norm"
    • Response output:
      • dictionary entries
        • dictionary entries (XML/HTML format)
          • if result is big then it is devided into result pages
          • with external links

Text functions

  • NormalizeServlet
    • URL: /mpdl/normalize
    • Request parameters:
      • srcUrl
        • source URL of XML fragment/document
      • method
        • method of normalization (e.g. "reg", "norm", "reg norm")
      • type
        • type of normalization (e.g. "display", "dictionary", "search")
    • Response output:
      • normalized XML fragment/document
  • TranscodeServlet
    • URL: /mpdl/transcode
    • Request parameters:
      • text
        • text to be transcoded (string)
      • srcEncoding
        • source encoding (e.g. "betacode", "buckwalter", "unicode")
      • destEncoding
        • destination encoding (e.g. "betacode", "buckwalter", "unicode")
    • Response output:
      • transcoded text

CMS technology

The CMS technology web archive ""mpiwg-mpdl-cms.war"" consists of:

  • Java code: mpiwg-mpdl-cms.jar
    • de.mpg.mpiwg.berlin.mpdl.cms.*
    • de.mpg.mpiwg.berlin.mpdl.servlets.cms
  • external Java libraries
    • mpiwg-mpdl-xml.jar
    • mpiwg-mpdl-lt.jar
    • ...
  • web application configuration file: web.xml

Following HTTP servlets are available:

  • GetDocumentServlet
    • URL: /mpdl/getDoc
    • Request parameters:
      • docId
        • document identifier (e.g. "/echo/la/benedetti_1585.xml")
    • Response output:
      • document
  • DocumentOperationServlet
    • URL: /mpdl/documentOperation
    • Request parameters:
      • operation
        • "create", "update", "delete"
      • srcUrl
      • destFileName (optional: default: file name in URL)
        • destination file name, e.g. "benedetti_1585.xml"
      • destLanguage (optional, default: xml:lang in document or "eng")
        • destination language (ISO 639-3 specifier), e.g. "lat"
    • Response output:
      • job id of scheduled operation
  • QueryServlet
    • URL: /mpdl/query
    • Request parameters:
      • queryType (optional: default: "morphological normalized")
        • "exact", "morphological", "normalized"
      • query
        • attribute query (e.g. "author = 'Benedetti' and language = 'lat'")
        • fulltext query (e.g. "quantitas")
      • docbases (optional, default: all document bases)
        • document bases (e.g. "mpdl", "archimedes-project")
      • orderBy (optional)
        • order query result by fieldname: e.g. "author" or "score" (fulltext queries)
      • resultPageNumber (optional, default: 1)
        • query result hits: page number
      • resultPageSize (optional, default: 100)
        • query result hits: page size
    • Response output:
      • query result (XML format)
  • QueryDocumentServlet
    • URL: /mpdl/queryDoc
    • Request parameters:
      • docId
        • document identifier (e.g. "/echo/la/benedetti_1585.xml")
      • query
        • fulltext query (e.g. "quantitas")
        • morphological fulltext query (e.g. "quantitas")
      • resultPageNumber (optional, default: 1)
        • query result hits: page number
      • resultPageSize (optional, default: 100)
        • query result hits: page size
    • Response output:
      • query result (XML format)