MPIWGWeb: SrvTxtUtils.py annotate

annotate SrvTxtUtils.py @ 55:12cb73494367

bibliographie funktionen

author	dwinter
date	Tue, 30 Apr 2013 16:46:32 +0200
parents	e40ff9829108
children	63184d71c014

rev	line source
0 bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	1 """Utility methods for handling XML, reading HTTP, etc"""
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	2
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	3 from App.ImageFile import ImageFile
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	4 from App.Common import rfc1123_date
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	5
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	6 import sys
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	7 import os
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	8 import stat
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	9 import urllib
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	10 import urllib2
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	11 import logging
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	12
49 e40ff9829108 improved old project inline image handling. casties parents: 16 diff changeset	13 import xml.etree.ElementTree as ET
e40ff9829108 improved old project inline image handling. casties parents: 16 diff changeset	14
e40ff9829108 improved old project inline image handling. casties parents: 16 diff changeset	15 srvTxtUtilsVersion = "1.7"
0 bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	16
16 54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	17 map_months = {'en': [u"",
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	18 u"January",
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	19 u"February",
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	20 u"March",
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	21 u"April",
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	22 u"May",
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	23 u"June",
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	24 u"July",
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	25 u"August",
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	26 u"September",
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	27 u"October",
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	28 u"November",
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	29 u"December"],
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	30 'de': [u"",
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	31 u"Januar",
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	32 u"Februar",
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	33 u"M\u00e4rz",
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	34 u"April",
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	35 u"Mai",
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	36 u"Juni",
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	37 u"Juli",
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	38 u"August",
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	39 u"September",
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	40 u"Oktober",
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	41 u"November",
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	42 u"Dezember"]}
0 bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	43
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	44 def getInt(number, default=0):
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	45 """returns always an int (0 in case of problems)"""
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	46 try:
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	47 return int(number)
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	48 except:
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	49 return int(default)
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	50
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	51 def getAt(array, idx, default=None):
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	52 """returns element idx from array or default (in case of problems)"""
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	53 try:
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	54 return array[idx]
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	55 except:
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	56 return default
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	57
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	58 def unicodify(s):
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	59 """decode str (utf-8 or latin-1 representation) into unicode object"""
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	60 if not s:
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	61 return u""
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	62 if isinstance(s, str):
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	63 try:
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	64 return s.decode('utf-8')
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	65 except:
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	66 return s.decode('latin-1')
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	67 else:
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	68 return s
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	69
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	70 def utf8ify(s):
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	71 """encode unicode object or string into byte string in utf-8 representation.
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	72 assumes string objects to be utf-8"""
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	73 if not s:
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	74 return ""
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	75 if isinstance(s, str):
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	76 return s
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	77 else:
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	78 return s.encode('utf-8')
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	79
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	80 def getText(node, recursive=0):
49 e40ff9829108 improved old project inline image handling. casties parents: 16 diff changeset	81 """returns all text content of a (etree) node and its subnodes"""
0 bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	82 if node is None:
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	83 return ''
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	84
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	85 # ElementTree:
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	86 text = node.text or ''
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	87 for e in node:
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	88 if recursive:
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	89 text += getText(e)
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	90 else:
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	91 text += e.text or ''
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	92 if e.tail:
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	93 text += e.tail
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	94
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	95 # 4Suite:
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	96 #nodelist=node.childNodes
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	97 #text = ""
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	98 #for n in nodelist:
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	99 # if n.nodeType == node.TEXT_NODE:
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	100 # text = text + n.data
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	101
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	102 return text
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	103
49 e40ff9829108 improved old project inline image handling. casties parents: 16 diff changeset	104
e40ff9829108 improved old project inline image handling. casties parents: 16 diff changeset	105 def serialize(node):
e40ff9829108 improved old project inline image handling. casties parents: 16 diff changeset	106 """returns a string containing an XML snippet of (etree) node"""
e40ff9829108 improved old project inline image handling. casties parents: 16 diff changeset	107 s = ET.tostring(node, 'UTF-8')
e40ff9829108 improved old project inline image handling. casties parents: 16 diff changeset	108 # snip off XML declaration
e40ff9829108 improved old project inline image handling. casties parents: 16 diff changeset	109 if s.startswith('<?xml'):
e40ff9829108 improved old project inline image handling. casties parents: 16 diff changeset	110 i = s.find('?>')
e40ff9829108 improved old project inline image handling. casties parents: 16 diff changeset	111 return s[i+3:]
e40ff9829108 improved old project inline image handling. casties parents: 16 diff changeset	112
e40ff9829108 improved old project inline image handling. casties parents: 16 diff changeset	113 return s
e40ff9829108 improved old project inline image handling. casties parents: 16 diff changeset	114
e40ff9829108 improved old project inline image handling. casties parents: 16 diff changeset	115
16 54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	116 def getMonthName(mon, lang):
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	117 """returns the name of the month mon in the language lang"""
54f135c1ed65 updated MPIWGFeature. casties parents: 0 diff changeset	118 return map_months[lang][mon]
0 bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	119
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	120 def getHttpData(url, data=None, num_tries=3, timeout=10, noExceptions=False):
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	121 """returns result from url+data HTTP request"""
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	122 # we do GET (by appending data to url)
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	123 if isinstance(data, str) or isinstance(data, unicode):
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	124 # if data is string then append
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	125 url = "%s?%s"%(url,data)
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	126 elif isinstance(data, dict) or isinstance(data, list) or isinstance(data, tuple):
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	127 # urlencode
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	128 url = "%s?%s"%(url,urllib.urlencode(data))
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	129
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	130 response = None
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	131 errmsg = None
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	132 for cnt in range(num_tries):
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	133 try:
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	134 logging.debug("getHttpData(#%s %ss) url=%s"%(cnt+1,timeout,url))
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	135 if sys.version_info < (2, 6):
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	136 # set timeout on socket -- ugly :-(
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	137 import socket
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	138 socket.setdefaulttimeout(float(timeout))
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	139 response = urllib2.urlopen(url)
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	140 else:
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	141 # timeout as parameter
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	142 response = urllib2.urlopen(url,timeout=float(timeout))
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	143 # check result?
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	144 break
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	145 except urllib2.HTTPError, e:
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	146 logging.error("getHttpData: HTTP error(%s): %s"%(e.code,e))
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	147 errmsg = str(e)
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	148 # stop trying
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	149 break
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	150 except urllib2.URLError, e:
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	151 logging.error("getHttpData: URLLIB error(%s): %s"%(e.reason,e))
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	152 errmsg = str(e)
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	153 # stop trying
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	154 #break
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	155
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	156 if response is not None:
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	157 data = response.read()
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	158 response.close()
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	159 return data
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	160
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	161 if noExceptions:
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	162 return None
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	163
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	164 raise IOError("ERROR fetching HTTP data from %s: %s"%(url,errmsg))
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	165 #return None
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	166
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	167
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	168 def refreshingImageFileIndexHtml(self, REQUEST, RESPONSE):
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	169 """index_html method for App.ImageFile that updates the file info for each request."""
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	170 stat_info = os.stat(self.path)
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	171 self.size = stat_info[stat.ST_SIZE]
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	172 self.lmt = float(stat_info[stat.ST_MTIME]) or time.time()
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	173 self.lmh = rfc1123_date(self.lmt)
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	174 # call original method
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	175 return ImageFile.index_html(self, REQUEST, RESPONSE)
bca61e893fcc first checkin of MPIWGWeb r2 branch from CVS into mercurial casties parents: diff changeset	176

Mercurial > hg > MPIWGWeb

annotate SrvTxtUtils.py @ 55:12cb73494367