Mercurial > hg > digilib

annotate doc/src/site/markdown/java-settings.md @ 1416:3e2f71bafcfb

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
add new codec options to doc.
author robcast
date Thu, 22 Oct 2015 16:42:30 +0200
parents 7a2de95a7b8e
children e6ad329cfac8
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1396
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
1 # Java VM memory settings for digilib
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
2
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
3 (Robert Casties, September 2013)
1216
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
4
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
5 The Java virtual machine (Java-VM) only uses a fixed amount of memory for
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
6 its operations. When an operation needs more memory than available it aborts
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
7 with an error ("out of memory error").
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
8
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
9 digilib can need a lot of memory depending on the size and type of images.
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
10 Since digilib runs as a servlet under Tomcat its in the same VM as the Tomcat
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
11 server.
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
12
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
13 The amount of memory Tomcat (version 5.0) uses is configured by creating a
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
14 `setenv.sh` (or `setenbv.bat`) script with a line
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
15
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
16 CATALINA_OPTS="-Xmx512m"
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
17
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
18 in Tomcat's `bin` directory (giving 512MB RAM in this case).
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
19
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
20 You can check the amount of memory your digilib instance has available on the
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
21 bottom of the web page `/server/dlConfig.jsp` in your digilib instance (e.g.
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
22 <http://localhost:8080/digilib/server/dlConfig.jsp>
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
23
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
24 # Installing JAI ImageIO
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
25
1396
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
26 (Robert Casties, September 2013)
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
27
1216
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
28 In principle you should be able to install the
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
29 [Java Advanced Imaging](http://java.sun.com/javase/technologies/desktop/media/jai/) JAI-ImageIO
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
30 JAR file `jai_imageio.jar` (and native
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
31 library files if available) in the `/WEB-INF/lib/` directory of the
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
32 digilib web application as part of the default installation.
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
33
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
34 You can see if the Jai-ImageIO plugin is active by checking for the
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
35 availability of the TIFF image format under "Supported image types" on the
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
36 [`/server/dlConfig.jsp`](http://localhost:8080/digilib/server/dlConfig.jsp)
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
37 status page.
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
38
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
39 Sometimes there are memory issues. Newer versions of Tomcat refuse to load
1416
3e2f71bafcfb add new codec options to doc.
robcast
parents: 1396
diff changeset
40 the libraries (see JREMemoryLeakPreventionListener) and I found that in some
3e2f71bafcfb add new codec options to doc.
robcast
parents: 1396
diff changeset
41 cases digilib stopped reading TIFF files
1216
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
42 after a period of running. In these cases it helped to install the JAI files in
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
43 Tomcats `lib/` directory or globally in the local Java JDK
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
44 installation (i.e. in the Java's 'jre/lib/ext/' directory on linux).
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
45
1396
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
46 # Codec availability and Performance
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
47
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
48 (Ubbo Veentjer, Oct 2015)
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
49
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
50 The number of image formats, which digilib may read or write, but also the performance of operating on this image formats depends on the ImageIO readers and writers available on the classpath.
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
51
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
52 Working with larger images in JPEG format we experienced a big performance difference using different implementations of the JPEG readers/writers. OpenJDK-7 for example brings rather slow JPEG codecs, OpenJDK-8 operates much quicker on JPEG images, relying on libjpeg-turbo for this file format. Also the official Oracle-JDKs may include faster native codecs.
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
53
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
54 Some drop-in replacements for the native codecs are:
1216
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
55
1396
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
56 * https://github.com/geosolutions-it/imageio-ext
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
57 * https://github.com/haraldk/TwelveMonkeys
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
58
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
59 if these jar files are availabe on the classpath, the codecs may be used by digilib. To add them the jar files could e.g. be placed in the lib directory of tomcat or addded as a dependency to the digilib maven project.
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
60
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
61 The actual codec implementation used is logged by digilib in debug mode, e.g.
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
62
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
63 1564059 [http-apr-9092-exec-4] DEBUG digilib.image.DocuImage - ImageIO: this reader: class com.twelvemonkeys.imageio.plugins.jpeg.JPEGImageReader
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
64
1416
3e2f71bafcfb add new codec options to doc.
robcast
parents: 1396
diff changeset
65 (Robert Casties, Oct 2015)
3e2f71bafcfb add new codec options to doc.
robcast
parents: 1396
diff changeset
66
3e2f71bafcfb add new codec options to doc.
robcast
parents: 1396
diff changeset
67 You can now use the TwelveMonkeys codecs instead of the default JAI-ImageIO by [building digilib](build-maven.html) with the Maven-Parameter `imageio=12m`:
3e2f71bafcfb add new codec options to doc.
robcast
parents: 1396
diff changeset
68
3e2f71bafcfb add new codec options to doc.
robcast
parents: 1396
diff changeset
69 mvn -Dimageio=12m package
3e2f71bafcfb add new codec options to doc.
robcast
parents: 1396
diff changeset
70
1396
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
71 # Codec performance
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
72
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
73 (Ubbo Veentjer, Oct 2015)
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
74
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
75 In our tests comparing the performance of OpenJDK7, OpenJDK8, imageio-ext and TwelveMonkeys codecs we experienced the following numbers for decoding, encoding and scaling a 4968px*5968px to 50% size:
1216
bd7dfa8b164e move new digilib doc from https://it-dev.mpiwg-berlin.mpg.de/hg/digilib-doc into main repo.
robcast
parents:
diff changeset
76
1396
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
77 24801 ms - OpenJDK7
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
78 11507 ms - OpenJDK7 with com.twelvemonkeys.imageio.plugins.jpeg.JPEGImageReader
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
79 4216 ms - OpenJDK7 with imageio-ext using libjpeg-turbo
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
80 3635 ms - OpenJDK8
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
81
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
82 This numbers may depend on the actual implementation used, the processing power of the CPU and many other factors, to this are just meant to be a rough hint.
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
83
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
84 For using imageio-ext, the native library needs to be
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
85 available with the LD_LIBRARY_PATH environment variable (compare: https://github.com/geosolutions-it/imageio-ext/wiki/TurboJPEG-plugin), also the .jar archives need to be on the classpath.
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
86
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
87 For using the TwelveMonkey Codecs we added the following jars to the tomcat lib directory, which were retrieved by maven (dependency on imageio-jpeg-3.1.2):
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
88
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
89 * common-image-3.1.2.jar
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
90 * common-lang-3.1.2.jar
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
91 * imageio-jpeg-3.1.2.jar
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
92 * common-io-3.1.2.jar
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
93 * imageio-core-3.1.2.jar
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
94 * imageio-metadata-3.1.2.jar
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
95
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
96
1416
3e2f71bafcfb add new codec options to doc.
robcast
parents: 1396
diff changeset
97 # Available image formats
1396
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
98
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
99 (Ubbo Veentjer, Oct 2015)
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
100
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
101 Digilib logs on startup which image formats are supported, e.g:
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
102
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
103 9763 [localhost-startStop-1] INFO digilib.conf.DigilibConfiguration - DocuImage supported image formats: raw, jpeg, tif, WBMP, pcx, PNM, JPG, wbmp, JPEG, PNG, jpeg 2000, tiff, BMP, JPEG2000, RAW, jpeg2000, GIF, TIF, TIFF, bmp, jpg, PCX, pnm, png, gif,
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
104
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
105 by adding e.g. TwelveMonkeys or image-io ext codecs, more codecs could become available.
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
106
7a2de95a7b8e documentation updated with Ubbo's codec tuning tips and IIIF API info.
robcast
parents: 1216
diff changeset
107