Mercurial > hg > digilib

annotate doc/src/site/markdown/java-settings.md @ 1417:e6ad329cfac8

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
more codec docs.
author robcast
date Thu, 22 Oct 2015 17:07:52 +0200
parents 3e2f71bafcfb
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1396
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
1 # Java VM memory settings for digilib
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
2
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
3 (Robert Casties, September 2013)
1216
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
4
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
5 The Java virtual machine (Java-VM) only uses a fixed amount of memory for
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
6 its operations. When an operation needs more memory than available it aborts
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
7 with an error ("out of memory error").
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
8
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
9 digilib can need a lot of memory depending on the size and type of images.
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
10 Since digilib runs as a servlet under Tomcat its in the same VM as the Tomcat
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
11 server.
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
12
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
13 The amount of memory Tomcat (version 5.0) uses is configured by creating a
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
14 `setenv.sh` (or `setenbv.bat`) script with a line
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
15
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
16 CATALINA_OPTS="-Xmx512m"
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
17
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
18 in Tomcat's `bin` directory (giving 512MB RAM in this case).
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
19
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
20 You can check the amount of memory your digilib instance has available on the
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
21 bottom of the web page `/server/dlConfig.jsp` in your digilib instance (e.g.
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
22 <http://localhost:8080/digilib/server/dlConfig.jsp>
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
23
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
24 # Installing JAI ImageIO
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
25
1396
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
26 (Robert Casties, September 2013)
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
27
1216
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
28 In principle you should be able to install the
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
29 [Java Advanced Imaging](http://java.sun.com/javase/technologies/desktop/media/jai/) JAI-ImageIO
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
30 JAR file `jai_imageio.jar` (and native
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
31 library files if available) in the `/WEB-INF/lib/` directory of the
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
32 digilib web application as part of the default installation.
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
33
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
34 You can see if the Jai-ImageIO plugin is active by checking for the
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
35 availability of the TIFF image format under "Supported image types" on the
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
36 [`/server/dlConfig.jsp`](http://localhost:8080/digilib/server/dlConfig.jsp)
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
37 status page.
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
38
1417
e6ad329cfac8 more codec docs.
robcast
parents: 1416
diff changeset
39 Sometimes there are problems with leaking memory. Newer versions of Tomcat refuse to load
1416
3e2f71bafcfb add new codec options to doc.
robcast
parents: 1396
diff changeset
40 the libraries (see JREMemoryLeakPreventionListener) and I found that in some
3e2f71bafcfb add new codec options to doc.
robcast
parents: 1396
diff changeset
41 cases digilib stopped reading TIFF files
1216
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
42 after a period of running. In these cases it helped to install the JAI files in
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
43 Tomcats `lib/` directory or globally in the local Java JDK
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
44 installation (i.e. in the Java's 'jre/lib/ext/' directory on linux).
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
45
1417
e6ad329cfac8 more codec docs.
robcast
parents: 1416
diff changeset
46 If you really need to have the imageio-plugins JAR inside the web app, please consider
e6ad329cfac8 more codec docs.
robcast
parents: 1416
diff changeset
47 using Harald Kuhrs [IIOProviderContextListener](https://github.com/haraldk/TwelveMonkeys#deploying-the-plugins-in-a-web-app).
e6ad329cfac8 more codec docs.
robcast
parents: 1416
diff changeset
48
e6ad329cfac8 more codec docs.
robcast
parents: 1416
diff changeset
49
1396
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
50 # Codec availability and Performance
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
51
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
52 (Ubbo Veentjer, Oct 2015)
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
53
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
54 The number of image formats, which digilib may read or write, but also the performance of operating on this image formats depends on the ImageIO readers and writers available on the classpath.
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
55
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
56 Working with larger images in JPEG format we experienced a big performance difference using different implementations of the JPEG readers/writers. OpenJDK-7 for example brings rather slow JPEG codecs, OpenJDK-8 operates much quicker on JPEG images, relying on libjpeg-turbo for this file format. Also the official Oracle-JDKs may include faster native codecs.
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
57
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
58 Some drop-in replacements for the native codecs are:
1216
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
59
1396
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
60 * https://github.com/geosolutions-it/imageio-ext
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
61 * https://github.com/haraldk/TwelveMonkeys
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
62
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
63 if these jar files are availabe on the classpath, the codecs may be used by digilib. To add them the jar files could e.g. be placed in the lib directory of tomcat or addded as a dependency to the digilib maven project.
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
64
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
65 The actual codec implementation used is logged by digilib in debug mode, e.g.
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
66
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
67 1564059 [http-apr-9092-exec-4] DEBUG digilib.image.DocuImage - ImageIO: this reader: class com.twelvemonkeys.imageio.plugins.jpeg.JPEGImageReader
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
68
1416
3e2f71bafcfb add new codec options to doc.
robcast
parents: 1396
diff changeset
69 (Robert Casties, Oct 2015)
3e2f71bafcfb add new codec options to doc.
robcast
parents: 1396
diff changeset
70
1417
e6ad329cfac8 more codec docs.
robcast
parents: 1416
diff changeset
71 You can now use the TwelveMonkeys codecs instead of the default JAI-ImageIO by just [building digilib](build-maven.html) with the Maven-Parameter `imageio=12m`:
1416
3e2f71bafcfb add new codec options to doc.
robcast
parents: 1396
diff changeset
72
3e2f71bafcfb add new codec options to doc.
robcast
parents: 1396
diff changeset
73 mvn -Dimageio=12m package
3e2f71bafcfb add new codec options to doc.
robcast
parents: 1396
diff changeset
74
1396
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
75 # Codec performance
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
76
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
77 (Ubbo Veentjer, Oct 2015)
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
78
1417
e6ad329cfac8 more codec docs.
robcast
parents: 1416
diff changeset
79 In our tests comparing the performance of OpenJDK7, OpenJDK8, imageio-ext and TwelveMonkeys codecs we experienced the following numbers for decoding, encoding and scaling a 4968px*5968px JPEG file with a color profile to 50% size:
1216
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
80
1417
e6ad329cfac8 more codec docs.
robcast
parents: 1416
diff changeset
81 24801 ms - OpenJDK7
e6ad329cfac8 more codec docs.
robcast
parents: 1416
diff changeset
82 11507 ms - OpenJDK7 with com.twelvemonkeys.imageio.plugins.jpeg.JPEGImageReader
e6ad329cfac8 more codec docs.
robcast
parents: 1416
diff changeset
83 4216 ms - OpenJDK7 with imageio-ext using libjpeg-turbo
e6ad329cfac8 more codec docs.
robcast
parents: 1416
diff changeset
84 3635 ms - OpenJDK8
1396
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
85
1417
e6ad329cfac8 more codec docs.
robcast
parents: 1416
diff changeset
86 This numbers may depend on the actual implementation used, the processing power of the CPU and many other factors, to this is just meant to be a rough hint.
1396
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
87
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
88 For using imageio-ext, the native library needs to be
1417
e6ad329cfac8 more codec docs.
robcast
parents: 1416
diff changeset
89 available with the `LD_LIBRARY_PATH` environment variable (compare: https://github.com/geosolutions-it/imageio-ext/wiki/TurboJPEG-plugin), also the .jar archives need to be on the classpath.
1396
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
90
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
91 For using the TwelveMonkey Codecs we added the following jars to the tomcat lib directory, which were retrieved by maven (dependency on imageio-jpeg-3.1.2):
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
92
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
93 * common-image-3.1.2.jar
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
94 * common-lang-3.1.2.jar
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
95 * imageio-jpeg-3.1.2.jar
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
96 * common-io-3.1.2.jar
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
97 * imageio-core-3.1.2.jar
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
98 * imageio-metadata-3.1.2.jar
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
99
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
100
1416
3e2f71bafcfb add new codec options to doc.
robcast
parents: 1396
diff changeset
101 # Available image formats
1396
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
102
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
103 (Ubbo Veentjer, Oct 2015)
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
104
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
105 Digilib logs on startup which image formats are supported, e.g:
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
106
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
107 9763 [localhost-startStop-1] INFO digilib.conf.DigilibConfiguration - DocuImage supported image formats: raw, jpeg, tif, WBMP, pcx, PNM, JPG, wbmp, JPEG, PNG, jpeg 2000, tiff, BMP, JPEG2000, RAW, jpeg2000, GIF, TIF, TIFF, bmp, jpg, PCX, pnm, png, gif,
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
108
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
109 by adding e.g. TwelveMonkeys or image-io ext codecs, more codecs could become available.
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
110
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
111