Mercurial > hg > ChinaGisRestApi

annotate RestDbInterface.py @ 261:5b38b50052e4

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Display Polygons and Lines via chgis_id Reload button alternative changed to "R"
author fknauft
date Fri, 07 Oct 2011 15:58:07 +0200
parents 901c1f745d13
children 3a10287447b1
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
1 '''
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
2 Created on 19.5.2010
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
3
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
4 @author: casties
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
5 '''
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
6
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
7 from OFS.Folder import Folder
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
8 from Products.PageTemplates.PageTemplateFile import PageTemplateFile
60
9fdadb60529f working on authentication and authorization
casties
parents: 58
diff changeset
9 from AccessControl import getSecurityManager, Unauthorized
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
10 from Products.ZSQLExtend import ZSQLExtend
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
11 import logging
15
5e3edf980813 more work on xml import
casties
parents: 14
diff changeset
12 import re
18
060797795a4d work on json return values
casties
parents: 17
diff changeset
13 import json
19
78e70dfa7ad6 GoogleMaps related functions
fknauft
parents: 18
diff changeset
14 import time
70
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
15 import psycopg2
254
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
16 import urllib
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
17
70
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
18 # make psycopg use unicode objects
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
19 import psycopg2.extensions
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
20 psycopg2.extensions.register_type(psycopg2.extensions.UNICODE)
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
21 psycopg2.extensions.register_type(psycopg2.extensions.UNICODEARRAY)
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
22
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
23 from zope.interface import implements
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
24 from zope.publisher.interfaces import IPublishTraverse
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
25 from ZPublisher.BaseRequest import DefaultPublishTraverse
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
26
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
27
70
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
28 def unicodify(s,alternate='latin-1'):
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
29 """decode str (utf-8 or latin-1 representation) into unicode object"""
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
30 if not s:
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
31 return u""
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
32 if isinstance(s, str):
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
33 try:
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
34 return s.decode('utf-8')
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
35 except:
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
36 return s.decode(alternate)
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
37 else:
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
38 return s
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
39
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
40 def utf8ify(s):
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
41 """encode unicode object or string into byte string in utf-8 representation.
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
42 assumes string objects to be utf-8"""
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
43 if not s:
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
44 return ""
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
45 if isinstance(s, str):
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
46 return s
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
47 else:
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
48 return s.encode('utf-8')
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
49
16
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
50 def getTextFromNode(node):
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
51 """get the cdata content of a XML node"""
16
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
52 if node is None:
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
53 return ""
16
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
54
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
55 if isinstance(node, list):
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
56 nodelist = node
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
57 else:
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
58 nodelist=node.childNodes
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
59
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
60 rc = ""
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
61 for node in nodelist:
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
62 if node.nodeType == node.TEXT_NODE:
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
63 rc = rc + node.data
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
64 return rc
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
65
16
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
66 def sqlName(s,lc=True):
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
67 """returns restricted ASCII-only version of string"""
15
5e3edf980813 more work on xml import
casties
parents: 14
diff changeset
68 if s is None:
5e3edf980813 more work on xml import
casties
parents: 14
diff changeset
69 return ""
5e3edf980813 more work on xml import
casties
parents: 14
diff changeset
70
5e3edf980813 more work on xml import
casties
parents: 14
diff changeset
71 # all else -> "_"
16
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
72 s = re.sub(r'[^A-Za-z0-9_]','_',s)
15
5e3edf980813 more work on xml import
casties
parents: 14
diff changeset
73 if lc:
5e3edf980813 more work on xml import
casties
parents: 14
diff changeset
74 return s.lower()
5e3edf980813 more work on xml import
casties
parents: 14
diff changeset
75
5e3edf980813 more work on xml import
casties
parents: 14
diff changeset
76 return s
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
77
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
78
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
79 class RestDbInterface(Folder):
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
80 """Object for RESTful database queries
7
bd52d9445a41 trying to rework
casties
parents: 5
diff changeset
81 path schema: /db/{schema}/{table}/
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
82 omitting table gives a list of schemas
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
83 omitting table and schema gives a list of schemas
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
84 """
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
85 implements(IPublishTraverse)
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
86
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
87 meta_type="RESTdb"
4
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
88 manage_options=Folder.manage_options+(
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
89 {'label':'Config','action':'manage_editRestDbInterfaceForm'},
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
90 )
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
91
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
92 # management templates
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
93 manage_editRestDbInterfaceForm=PageTemplateFile('zpt/editRestDbInterface',globals())
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
94
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
95 # data templates
8
a9a49f5765c9 reworking templates and schema
casties
parents: 7
diff changeset
96 XML_index = PageTemplateFile('zpt/XML_index', globals())
a9a49f5765c9 reworking templates and schema
casties
parents: 7
diff changeset
97 XML_schema = PageTemplateFile('zpt/XML_schema', globals())
a9a49f5765c9 reworking templates and schema
casties
parents: 7
diff changeset
98 XML_schema_table = PageTemplateFile('zpt/XML_schema_table', globals())
a9a49f5765c9 reworking templates and schema
casties
parents: 7
diff changeset
99 HTML_index = PageTemplateFile('zpt/HTML_index', globals())
a9a49f5765c9 reworking templates and schema
casties
parents: 7
diff changeset
100 HTML_schema = PageTemplateFile('zpt/HTML_schema', globals())
a9a49f5765c9 reworking templates and schema
casties
parents: 7
diff changeset
101 HTML_schema_table = PageTemplateFile('zpt/HTML_schema_table', globals())
254
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
102 GIS_schema_table = PageTemplateFile('zpt/GIS_schema_table', globals())
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
103 KML_schema_table = PageTemplateFile('zpt/KML_schema_table', globals())
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
104 HTML_schema_usertables = PageTemplateFile('zpt/HTML_schema_usertables', globals())
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
105
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
106
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
107
22
1a4b56716902 NEW - # 12: create table and upload data
casties
parents: 21
diff changeset
108 JSONHTML_index = PageTemplateFile('zpt/JSONHTML_index', globals())
1a4b56716902 NEW - # 12: create table and upload data
casties
parents: 21
diff changeset
109 JSONHTML_schema = PageTemplateFile('zpt/JSONHTML_schema', globals())
1a4b56716902 NEW - # 12: create table and upload data
casties
parents: 21
diff changeset
110 JSONHTML_schema_table = PageTemplateFile('zpt/JSONHTML_schema_table', globals())
24
f3811d9a36da start adding checkuploadtable
casties
parents: 23
diff changeset
111 # JSON_* templates are scripts
254
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
112
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
113
45
ed8db63fab4f more refactoring of templates
casties
parents: 44
diff changeset
114 def JSON_index(self):
30
51db9e78bf98 working on field types
casties
parents: 29
diff changeset
115 """JSON index function"""
51db9e78bf98 working on field types
casties
parents: 29
diff changeset
116 self.REQUEST.RESPONSE.setHeader("Content-Type", "application/json")
45
ed8db63fab4f more refactoring of templates
casties
parents: 44
diff changeset
117 json.dump(self.getListOfSchemas(), self.REQUEST.RESPONSE)
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
118
45
ed8db63fab4f more refactoring of templates
casties
parents: 44
diff changeset
119 def JSON_schema(self,schema):
30
51db9e78bf98 working on field types
casties
parents: 29
diff changeset
120 """JSON index function"""
51db9e78bf98 working on field types
casties
parents: 29
diff changeset
121 self.REQUEST.RESPONSE.setHeader("Content-Type", "application/json")
45
ed8db63fab4f more refactoring of templates
casties
parents: 44
diff changeset
122 json.dump(self.getListOfTables(schema), self.REQUEST.RESPONSE)
30
51db9e78bf98 working on field types
casties
parents: 29
diff changeset
123
44
c6c47034d2a4 more refactoring of templates
casties
parents: 43
diff changeset
124 def JSON_schema_table(self,schema,table):
30
51db9e78bf98 working on field types
casties
parents: 29
diff changeset
125 """JSON index function"""
51db9e78bf98 working on field types
casties
parents: 29
diff changeset
126 self.REQUEST.RESPONSE.setHeader("Content-Type", "application/json")
44
c6c47034d2a4 more refactoring of templates
casties
parents: 43
diff changeset
127 json.dump(self.getTable(schema, table), self.REQUEST.RESPONSE)
30
51db9e78bf98 working on field types
casties
parents: 29
diff changeset
128
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
129
4
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
130 def __init__(self, id, title, connection_id=None):
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
131 """init"""
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
132 self.id = id
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
133 self.title = title
4
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
134 # database connection id
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
135 self.connection_id = connection_id
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
136 # create template folder
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
137 self.manage_addFolder('template')
44
c6c47034d2a4 more refactoring of templates
casties
parents: 43
diff changeset
138
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
139
44
c6c47034d2a4 more refactoring of templates
casties
parents: 43
diff changeset
140 def getRestDbUrl(self):
c6c47034d2a4 more refactoring of templates
casties
parents: 43
diff changeset
141 """returns url to the RestDb instance"""
c6c47034d2a4 more refactoring of templates
casties
parents: 43
diff changeset
142 return self.absolute_url()
c6c47034d2a4 more refactoring of templates
casties
parents: 43
diff changeset
143
22
1a4b56716902 NEW - # 12: create table and upload data
casties
parents: 21
diff changeset
144 def getJsonString(self,object):
1a4b56716902 NEW - # 12: create table and upload data
casties
parents: 21
diff changeset
145 """returns a JSON formatted string from object"""
1a4b56716902 NEW - # 12: create table and upload data
casties
parents: 21
diff changeset
146 return json.dumps(object)
4
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
147
17
ed997e639cfd more upload table
casties
parents: 16
diff changeset
148 def getCursor(self,autocommit=True):
4
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
149 """returns fresh DB cursor"""
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
150 conn = getattr(self,"_v_database_connection",None)
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
151 if conn is None:
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
152 # create a new connection object
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
153 try:
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
154 if self.connection_id is None:
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
155 # try to take the first existing ID
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
156 connids = SQLConnectionIDs(self)
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
157 if len(connids) > 0:
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
158 connection_id = connids[0][0]
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
159 self.connection_id = connection_id
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
160 logging.debug("connection_id: %s"%repr(connection_id))
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
161
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
162 da = getattr(self, self.connection_id)
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
163 da.connect('')
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
164 # we copy the DAs database connection
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
165 conn = da._v_database_connection
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
166 #conn._register() # register with the Zope transaction system
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
167 self._v_database_connection = conn
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
168 except Exception, e:
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
169 raise IOError("No database connection! (%s)"%str(e))
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
170
5
7539a9e69365 small fix
casties
parents: 4
diff changeset
171 cursor = conn.getcursor()
17
ed997e639cfd more upload table
casties
parents: 16
diff changeset
172 if autocommit:
ed997e639cfd more upload table
casties
parents: 16
diff changeset
173 # is there a better version to get to the connection?
ed997e639cfd more upload table
casties
parents: 16
diff changeset
174 cursor.connection.set_isolation_level(psycopg2.extensions.ISOLATION_LEVEL_AUTOCOMMIT)
ed997e639cfd more upload table
casties
parents: 16
diff changeset
175
4
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
176 return cursor
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
177
55
2f4c427dec44 wrangled kml construction into template and method
casties
parents: 53
diff changeset
178 def getFieldNameMap(self,fields):
2f4c427dec44 wrangled kml construction into template and method
casties
parents: 53
diff changeset
179 """returns a dict mapping field names to row indexes"""
2f4c427dec44 wrangled kml construction into template and method
casties
parents: 53
diff changeset
180 map = {}
2f4c427dec44 wrangled kml construction into template and method
casties
parents: 53
diff changeset
181 i = 0
2f4c427dec44 wrangled kml construction into template and method
casties
parents: 53
diff changeset
182 for f in fields:
2f4c427dec44 wrangled kml construction into template and method
casties
parents: 53
diff changeset
183 map[f[0]] = i
2f4c427dec44 wrangled kml construction into template and method
casties
parents: 53
diff changeset
184 i += 1
2f4c427dec44 wrangled kml construction into template and method
casties
parents: 53
diff changeset
185
2f4c427dec44 wrangled kml construction into template and method
casties
parents: 53
diff changeset
186 return map
2f4c427dec44 wrangled kml construction into template and method
casties
parents: 53
diff changeset
187
17
ed997e639cfd more upload table
casties
parents: 16
diff changeset
188 def executeSQL(self, query, args=None, hasResult=True, autocommit=True):
4
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
189 """execute query with args on database and return all results.
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
190 result format: {"fields":fields, "rows":data}"""
16
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
191 logging.debug("executeSQL query=%s args=%s"%(query,args))
17
ed997e639cfd more upload table
casties
parents: 16
diff changeset
192 cur = self.getCursor(autocommit=autocommit)
53
f5bfcfa97e7e more JSON store
casties
parents: 45
diff changeset
193 if args is not None:
f5bfcfa97e7e more JSON store
casties
parents: 45
diff changeset
194 # make sure args is a list
f5bfcfa97e7e more JSON store
casties
parents: 45
diff changeset
195 if isinstance(args,basestring):
f5bfcfa97e7e more JSON store
casties
parents: 45
diff changeset
196 args = (args,)
f5bfcfa97e7e more JSON store
casties
parents: 45
diff changeset
197
4
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
198 cur.execute(query, args)
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
199 # description of returned fields
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
200 fields = cur.description
17
ed997e639cfd more upload table
casties
parents: 16
diff changeset
201 if hasResult:
ed997e639cfd more upload table
casties
parents: 16
diff changeset
202 # get all data in an array
ed997e639cfd more upload table
casties
parents: 16
diff changeset
203 data = cur.fetchall()
ed997e639cfd more upload table
casties
parents: 16
diff changeset
204 cur.close()
55
2f4c427dec44 wrangled kml construction into template and method
casties
parents: 53
diff changeset
205 #logging.debug("fields: %s"%repr(fields))
58
5ed0769f5ad3 more cleanup in kml generation
casties
parents: 55
diff changeset
206 #logging.debug("rows: %s"%repr(data))
17
ed997e639cfd more upload table
casties
parents: 16
diff changeset
207 return {"fields":fields, "rows":data}
ed997e639cfd more upload table
casties
parents: 16
diff changeset
208 else:
ed997e639cfd more upload table
casties
parents: 16
diff changeset
209 cur.close()
ed997e639cfd more upload table
casties
parents: 16
diff changeset
210 return None
16
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
211
60
9fdadb60529f working on authentication and authorization
casties
parents: 58
diff changeset
212 def isAllowed(self,action,schema,table,user=None):
9fdadb60529f working on authentication and authorization
casties
parents: 58
diff changeset
213 """returns if the requested action on the table is allowed"""
9fdadb60529f working on authentication and authorization
casties
parents: 58
diff changeset
214 if user is None:
9fdadb60529f working on authentication and authorization
casties
parents: 58
diff changeset
215 user = self.REQUEST.get('AUTHENTICATED_USER',None)
9fdadb60529f working on authentication and authorization
casties
parents: 58
diff changeset
216 logging.debug("isAllowed action=%s schema=%s table=%s user=%s"%(action,schema,table,user))
9fdadb60529f working on authentication and authorization
casties
parents: 58
diff changeset
217 # no default policy!
9fdadb60529f working on authentication and authorization
casties
parents: 58
diff changeset
218 return True
9fdadb60529f working on authentication and authorization
casties
parents: 58
diff changeset
219
9fdadb60529f working on authentication and authorization
casties
parents: 58
diff changeset
220
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
221 def publishTraverse(self,request,name):
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
222 """change the traversal"""
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
223 # get stored path
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
224 path = request.get('restdb_path', [])
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
225 logging.debug("publishtraverse: name=%s restdb_path=%s"%(name,path))
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
226
13
e2c73c077533 NEW - # 12: create table and upload data
casties
parents: 12
diff changeset
227 if name in ("index_html", "PUT"):
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
228 # end of traversal
13
e2c73c077533 NEW - # 12: create table and upload data
casties
parents: 12
diff changeset
229 if request.get("method") == "POST" and request.get("action",None) == "PUT":
e2c73c077533 NEW - # 12: create table and upload data
casties
parents: 12
diff changeset
230 # fake PUT by POST with action=PUT
e2c73c077533 NEW - # 12: create table and upload data
casties
parents: 12
diff changeset
231 name = "PUT"
e2c73c077533 NEW - # 12: create table and upload data
casties
parents: 12
diff changeset
232
e2c73c077533 NEW - # 12: create table and upload data
casties
parents: 12
diff changeset
233 return getattr(self, name)
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
234 #TODO: should we check more?
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
235 else:
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
236 # traverse
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
237 if len(path) == 0:
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
238 # first segment
8
a9a49f5765c9 reworking templates and schema
casties
parents: 7
diff changeset
239 if name == 'db':
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
240 # virtual path -- continue traversing
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
241 path = [name]
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
242 request['restdb_path'] = path
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
243 else:
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
244 # try real path
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
245 tr = DefaultPublishTraverse(self, request)
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
246 ob = tr.publishTraverse(request, name)
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
247 return ob
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
248 else:
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
249 path.append(name)
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
250
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
251 # continue traversing
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
252 return self
44
c6c47034d2a4 more refactoring of templates
casties
parents: 43
diff changeset
253
c6c47034d2a4 more refactoring of templates
casties
parents: 43
diff changeset
254
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
255 def index_html(self,REQUEST,RESPONSE):
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
256 """index method"""
8
a9a49f5765c9 reworking templates and schema
casties
parents: 7
diff changeset
257 # ReST path was stored in request
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
258 path = REQUEST.get('restdb_path',[])
19
78e70dfa7ad6 GoogleMaps related functions
fknauft
parents: 18
diff changeset
259
8
a9a49f5765c9 reworking templates and schema
casties
parents: 7
diff changeset
260 # type and format are real parameter
26
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
261 resultFormat = REQUEST.get('format','HTML').upper()
27
a2e4ca3f1cff NEW - # 12: create table and upload data
casties
parents: 26
diff changeset
262 queryType = REQUEST.get('type',None)
19
78e70dfa7ad6 GoogleMaps related functions
fknauft
parents: 18
diff changeset
263
26
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
264 logging.debug("index_html path=%s resultFormat=%s queryType=%s"%(path,resultFormat,queryType))
8
a9a49f5765c9 reworking templates and schema
casties
parents: 7
diff changeset
265
26
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
266 if queryType is not None:
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
267 # non-empty queryType -- look for template
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
268 pt = getattr(self.template, "%s_%s"%(resultFormat,queryType), None)
8
a9a49f5765c9 reworking templates and schema
casties
parents: 7
diff changeset
269 if pt is not None:
26
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
270 return pt(format=resultFormat,type=queryType,path=path)
8
a9a49f5765c9 reworking templates and schema
casties
parents: 7
diff changeset
271
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
272 if len(path) == 1:
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
273 # list of schemas
61
e81d034b28a5 more permission handling and table metadata
casties
parents: 60
diff changeset
274 return self.showListOfSchemas(format=resultFormat)
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
275 elif len(path) == 2:
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
276 # list of tables
61
e81d034b28a5 more permission handling and table metadata
casties
parents: 60
diff changeset
277 return self.showListOfTables(format=resultFormat,schema=path[1])
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
278 elif len(path) == 3:
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
279 # table
25
cef1bfa821cb fixed problem with not passing request
casties
parents: 24
diff changeset
280 if REQUEST.get("method") == "POST" and REQUEST.get("create_table_file",None) is not None:
24
f3811d9a36da start adding checkuploadtable
casties
parents: 23
diff changeset
281 # POST to table to check
61
e81d034b28a5 more permission handling and table metadata
casties
parents: 60
diff changeset
282 return self.checkTable(format=resultFormat,schema=path[1],table=path[2])
24
f3811d9a36da start adding checkuploadtable
casties
parents: 23
diff changeset
283 # else show table
254
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
284 return self.showTable(format=resultFormat,schema=path[1],table=path[2],REQUEST=REQUEST, RESPONSE=RESPONSE)
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
285
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
286 # don't know what to do
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
287 return str(REQUEST)
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
288
12
0fa22c291ff1 NEW - # 12: create table and upload data
casties
parents: 9
diff changeset
289 def PUT(self, REQUEST, RESPONSE):
0fa22c291ff1 NEW - # 12: create table and upload data
casties
parents: 9
diff changeset
290 """
0fa22c291ff1 NEW - # 12: create table and upload data
casties
parents: 9
diff changeset
291 Implement WebDAV/HTTP PUT/FTP put method for this object.
0fa22c291ff1 NEW - # 12: create table and upload data
casties
parents: 9
diff changeset
292 """
13
e2c73c077533 NEW - # 12: create table and upload data
casties
parents: 12
diff changeset
293 logging.debug("RestDbInterface PUT")
e2c73c077533 NEW - # 12: create table and upload data
casties
parents: 12
diff changeset
294 #logging.debug("req=%s"%REQUEST)
12
0fa22c291ff1 NEW - # 12: create table and upload data
casties
parents: 9
diff changeset
295 #self.dav__init(REQUEST, RESPONSE)
0fa22c291ff1 NEW - # 12: create table and upload data
casties
parents: 9
diff changeset
296 #self.dav__simpleifhandler(REQUEST, RESPONSE)
13
e2c73c077533 NEW - # 12: create table and upload data
casties
parents: 12
diff changeset
297 # ReST path was stored in request
e2c73c077533 NEW - # 12: create table and upload data
casties
parents: 12
diff changeset
298 path = REQUEST.get('restdb_path',[])
e2c73c077533 NEW - # 12: create table and upload data
casties
parents: 12
diff changeset
299 if len(path) == 3:
e2c73c077533 NEW - # 12: create table and upload data
casties
parents: 12
diff changeset
300 schema = path[1]
e2c73c077533 NEW - # 12: create table and upload data
casties
parents: 12
diff changeset
301 tablename = path[2]
e2c73c077533 NEW - # 12: create table and upload data
casties
parents: 12
diff changeset
302 file = REQUEST.get("create_table_file",None)
e2c73c077533 NEW - # 12: create table and upload data
casties
parents: 12
diff changeset
303 if file is None:
e2c73c077533 NEW - # 12: create table and upload data
casties
parents: 12
diff changeset
304 RESPONSE.setStatus(400)
e2c73c077533 NEW - # 12: create table and upload data
casties
parents: 12
diff changeset
305 return
4
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
306
30
51db9e78bf98 working on field types
casties
parents: 29
diff changeset
307 fields = None
51db9e78bf98 working on field types
casties
parents: 29
diff changeset
308 fieldsStr = REQUEST.get("create_table_fields",None)
51db9e78bf98 working on field types
casties
parents: 29
diff changeset
309 logging.debug("put with schema=%s table=%s file=%s fields=%s"%(schema,tablename,file,repr(fieldsStr)))
51db9e78bf98 working on field types
casties
parents: 29
diff changeset
310 if fieldsStr is not None:
51db9e78bf98 working on field types
casties
parents: 29
diff changeset
311 # unpack fields
51db9e78bf98 working on field types
casties
parents: 29
diff changeset
312 fields = [{"name":n, "type": t} for (n,t) in [f.split(":") for f in fieldsStr.split(",")]]
31
6d2055f1b4fa more work on types
casties
parents: 30
diff changeset
313
28
9e4f9cfd1edc start adding field structure to xml upload
casties
parents: 27
diff changeset
314 ret = self.createTableFromXML(schema, tablename, file, fields)
26
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
315 # return the result as JSON
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
316 format = REQUEST.get("format","JSON")
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
317 if format == "JSON":
20
67ca17753cd5 NEW - # 12: create table and upload data
casties
parents: 18
diff changeset
318 RESPONSE.setHeader("Content-Type", "application/json")
67ca17753cd5 NEW - # 12: create table and upload data
casties
parents: 18
diff changeset
319 json.dump(ret, RESPONSE)
26
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
320
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
321 elif format == "JSONHTML":
20
67ca17753cd5 NEW - # 12: create table and upload data
casties
parents: 18
diff changeset
322 RESPONSE.setHeader("Content-Type", "text/html")
67ca17753cd5 NEW - # 12: create table and upload data
casties
parents: 18
diff changeset
323 RESPONSE.write("<html>\n<body>\n<pre>")
67ca17753cd5 NEW - # 12: create table and upload data
casties
parents: 18
diff changeset
324 json.dump(ret, RESPONSE)
67ca17753cd5 NEW - # 12: create table and upload data
casties
parents: 18
diff changeset
325 RESPONSE.write("</pre>\n</body>\n</html>")
13
e2c73c077533 NEW - # 12: create table and upload data
casties
parents: 12
diff changeset
326
e2c73c077533 NEW - # 12: create table and upload data
casties
parents: 12
diff changeset
327 else:
e2c73c077533 NEW - # 12: create table and upload data
casties
parents: 12
diff changeset
328 # 400 Bad Request
e2c73c077533 NEW - # 12: create table and upload data
casties
parents: 12
diff changeset
329 RESPONSE.setStatus(400)
e2c73c077533 NEW - # 12: create table and upload data
casties
parents: 12
diff changeset
330 return
252
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
331 def getAttributeNames(self,schema='public',table=None):
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
332 return self.executeSQL("SELECT attname FROM pg_attribute, pg_class WHERE pg_class.oid = attrelid AND attnum>0 AND relname = '%s';"%(table))
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
333
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
334 def getAttributeTypes(self,schema='public',table=None):
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
335 return self.executeSQL("SELECT field_name, gis_type FROM public.gis_table_meta_rows WHERE table_name = '%s';"%(table))
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
336
61
e81d034b28a5 more permission handling and table metadata
casties
parents: 60
diff changeset
337 def showTable(self,format='XML',schema='public',table=None,REQUEST=None,RESPONSE=None):
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
338 """returns PageTemplate with tables"""
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
339 logging.debug("showtable")
38
f6d9a3caf986 fixed errors from merging
casties
parents: 37
diff changeset
340 if REQUEST is None:
f6d9a3caf986 fixed errors from merging
casties
parents: 37
diff changeset
341 REQUEST = self.REQUEST
254
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
342 queryArgs={'doc':None,'id':None}
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
343 queryArgs['doc'] = REQUEST.get('doc')
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
344 queryArgs['id'] = REQUEST.get('id')
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
345
23
860ec92f99df added cross-site access header (for GET)
casties
parents: 22
diff changeset
346 # should be cross-site accessible
25
cef1bfa821cb fixed problem with not passing request
casties
parents: 24
diff changeset
347 if RESPONSE is None:
cef1bfa821cb fixed problem with not passing request
casties
parents: 24
diff changeset
348 RESPONSE = self.REQUEST.RESPONSE
38
f6d9a3caf986 fixed errors from merging
casties
parents: 37
diff changeset
349
23
860ec92f99df added cross-site access header (for GET)
casties
parents: 22
diff changeset
350 RESPONSE.setHeader('Access-Control-Allow-Origin', '*')
38
f6d9a3caf986 fixed errors from merging
casties
parents: 37
diff changeset
351
22
1a4b56716902 NEW - # 12: create table and upload data
casties
parents: 21
diff changeset
352 # everything else has its own template
254
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
353 pt = getattr(self.template, '%s_schema_table'%format, REQUEST)
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
354 if pt is None:
254
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
355 return "ERROR!! template %s_schema_table not found at %s"%(format, self.template )
44
c6c47034d2a4 more refactoring of templates
casties
parents: 43
diff changeset
356 #data = self.getTable(schema,table)
254
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
357 return pt(schema=schema,table=table,args=queryArgs)
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
358
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
359 def getLiveUrl(self,schema,table,useTimestamp=True,REQUEST=None):
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
360 if REQUEST is None:
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
361 REQUEST = self.REQUEST
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
362 logging.debug("getLiveUrl")
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
363 baseUrl = self.absolute_url()
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
364 timestamp = time.time()
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
365 # filter parameters in URL and add to new URL
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
366 params = [p for p in REQUEST.form.items() if p[0] not in ('format','timestamp')]
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
367 params.append(('format','KML'))
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
368 if useTimestamp:
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
369 # add timestamp so URL changes every time
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
370 params.append(('timestamp',timestamp))
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
371 paramstr = urllib.urlencode(params)
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
372 return "%s/db/%s/%s?%s"%(baseUrl,schema,table,paramstr)
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
373
901c1f745d13 GIS-Links from MPDL-Documents now served by Mappit-Server
fknauft
parents: 252
diff changeset
374
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
375
58
5ed0769f5ad3 more cleanup in kml generation
casties
parents: 55
diff changeset
376 def getTable(self,schema='public',table=None,sortBy=1,username='guest'):
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
377 """return table data"""
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
378 logging.debug("gettable")
252
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
379 attrNames=self.getAttributeNames(schema,table)
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
380 attrTypes=self.getAttributeTypes(schema,table)
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
381 attrString=""
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
382 # try:
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
383 for name in attrNames['rows']:
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
384 logging.debug("name: ", name[0])
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
385 not_added=True
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
386 if name[0] == "the_geom": #FJK: the table column is "the_geom"
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
387 attrString=attrString+"ST_AsText("+name[0]+"),"
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
388 not_added=False
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
389 break
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
390 for a_iter in attrTypes['rows']:
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
391 not_added = True
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
392 logging.debug("attrTypes.field_name: ", a_iter[0])
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
393 if a_iter[0]==name[0]:
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
394 logging.debug("attrTypes.gis_type: ", a_iter[1])
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
395 if a_iter[1] == "the_geom": #FJK: the table column is registered in gis_table_meta_rows as type "the_geom"
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
396 attrString=attrString+"ST_AsText("+name[0]+"),"
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
397 not_added=False
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
398 if not_added:
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
399 attrString=attrString+name[0]+","
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
400 attrString=str(attrString).rsplit(",",1)[0] #to remove last ","
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
401 if sortBy:
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
402 data = self.executeSQL('select %s from "%s"."%s" order by %s'%(attrString,schema,table,sortBy))
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
403 else:
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
404 data = self.executeSQL('select %s from "%s"."%s"'%(attrString,schema,table))
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
405 # except:
227
11dea0923d2f Upload of xml-tables with empty Cells possible
fknauft
parents: 87
diff changeset
406 """ table does not exist """
252
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
407 # fields=self.get
efd2469d1722 geometry-column of tables will be displayed as string
fknauft
parents: 230
diff changeset
408 # self.createEmptyTable(schema, table, fields)
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
409 return data
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
410
26
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
411 def hasTable(self,schema='public',table=None,username='guest'):
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
412 """return if table exists"""
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
413 logging.debug("hastable")
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
414 data = self.executeSQL('select 1 from information_schema.tables where table_schema=%s and table_name=%s',(schema,table))
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
415 ret = bool(data['rows'])
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
416 return ret
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
417
61
e81d034b28a5 more permission handling and table metadata
casties
parents: 60
diff changeset
418 def showListOfTables(self,format='XML',schema='public',REQUEST=None,RESPONSE=None):
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
419 """returns PageTemplate with list of tables"""
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
420 logging.debug("showlistoftables")
23
860ec92f99df added cross-site access header (for GET)
casties
parents: 22
diff changeset
421 # should be cross-site accessible
25
cef1bfa821cb fixed problem with not passing request
casties
parents: 24
diff changeset
422 if RESPONSE is None:
cef1bfa821cb fixed problem with not passing request
casties
parents: 24
diff changeset
423 RESPONSE = self.REQUEST.RESPONSE
23
860ec92f99df added cross-site access header (for GET)
casties
parents: 22
diff changeset
424 RESPONSE.setHeader('Access-Control-Allow-Origin', '*')
24
f3811d9a36da start adding checkuploadtable
casties
parents: 23
diff changeset
425
9
76ac7a721273 more rework of templates
casties
parents: 8
diff changeset
426 pt = getattr(self.template, '%s_schema'%format, None)
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
427 if pt is None:
9
76ac7a721273 more rework of templates
casties
parents: 8
diff changeset
428 return "ERROR!! template %s_schema not found"%format
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
429
45
ed8db63fab4f more refactoring of templates
casties
parents: 44
diff changeset
430 #data = self.getListOfTables(schema)
ed8db63fab4f more refactoring of templates
casties
parents: 44
diff changeset
431 return pt(schema=schema)
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
432
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
433 def getListOfTables(self,schema='public',username='guest'):
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
434 """return list of tables"""
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
435 logging.debug("getlistoftables")
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
436 # get list of fields and types of db table
61
e81d034b28a5 more permission handling and table metadata
casties
parents: 60
diff changeset
437 #qstr="""SELECT c.relname AS tablename FROM pg_catalog.pg_class c
e81d034b28a5 more permission handling and table metadata
casties
parents: 60
diff changeset
438 # LEFT JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace
e81d034b28a5 more permission handling and table metadata
casties
parents: 60
diff changeset
439 # WHERE c.relkind IN ('r','') AND n.nspname NOT IN ('pg_catalog', 'pg_toast')
e81d034b28a5 more permission handling and table metadata
casties
parents: 60
diff changeset
440 # AND pg_catalog.pg_table_is_visible(c.oid)
e81d034b28a5 more permission handling and table metadata
casties
parents: 60
diff changeset
441 # AND c.relname ORDER BY 1"""
e81d034b28a5 more permission handling and table metadata
casties
parents: 60
diff changeset
442 qstr = """SELECT table_name FROM information_schema.tables WHERE table_type = 'BASE TABLE'
e81d034b28a5 more permission handling and table metadata
casties
parents: 60
diff changeset
443 AND table_schema = %s ORDER BY 1"""
e81d034b28a5 more permission handling and table metadata
casties
parents: 60
diff changeset
444 data=self.executeSQL(qstr,(schema,))
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
445 return data
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
446
61
e81d034b28a5 more permission handling and table metadata
casties
parents: 60
diff changeset
447 def showListOfSchemas(self,format='XML',REQUEST=None,RESPONSE=None):
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
448 """returns PageTemplate with list of schemas"""
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
449 logging.debug("showlistofschemas")
23
860ec92f99df added cross-site access header (for GET)
casties
parents: 22
diff changeset
450 # should be cross-site accessible
25
cef1bfa821cb fixed problem with not passing request
casties
parents: 24
diff changeset
451 if RESPONSE is None:
cef1bfa821cb fixed problem with not passing request
casties
parents: 24
diff changeset
452 RESPONSE = self.REQUEST.RESPONSE
23
860ec92f99df added cross-site access header (for GET)
casties
parents: 22
diff changeset
453 RESPONSE.setHeader('Access-Control-Allow-Origin', '*')
24
f3811d9a36da start adding checkuploadtable
casties
parents: 23
diff changeset
454
9
76ac7a721273 more rework of templates
casties
parents: 8
diff changeset
455 pt = getattr(self.template, '%s_index'%format, None)
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
456 if pt is None:
9
76ac7a721273 more rework of templates
casties
parents: 8
diff changeset
457 return "ERROR!! template %s_index not found"%format
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
458
45
ed8db63fab4f more refactoring of templates
casties
parents: 44
diff changeset
459 #data = self.getListOfSchemas()
ed8db63fab4f more refactoring of templates
casties
parents: 44
diff changeset
460 return pt()
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
461
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
462 def getListOfSchemas(self,username='guest'):
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
463 """return list of schemas"""
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
464 logging.debug("getlistofschemas")
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
465 # TODO: really look up schemas
9
76ac7a721273 more rework of templates
casties
parents: 8
diff changeset
466 data={'fields': (('schemas',),), 'rows': [('public',),]}
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
467 return data
26
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
468
61
e81d034b28a5 more permission handling and table metadata
casties
parents: 60
diff changeset
469 def checkTable(self,format,schema,table,REQUEST=None,RESPONSE=None):
26
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
470 """check the table.
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
471 returns valid data fields and table name."""
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
472 if REQUEST is None:
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
473 REQUEST = self.REQUEST
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
474 RESPONSE = REQUEST.RESPONSE
74
6e19a9af00e6 htmlHead() included
fknauft
parents: 73
diff changeset
475
26
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
476 file = REQUEST.get("create_table_file",None)
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
477 res = self.checkTableFromXML(schema, table, file)
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
478 logging.debug("checkTable result=%s"%repr(res))
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
479 # return the result as JSON
61
e81d034b28a5 more permission handling and table metadata
casties
parents: 60
diff changeset
480 if format == "JSON":
26
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
481 RESPONSE.setHeader("Content-Type", "application/json")
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
482 json.dump(res, RESPONSE)
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
483
61
e81d034b28a5 more permission handling and table metadata
casties
parents: 60
diff changeset
484 elif format == "JSONHTML":
26
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
485 RESPONSE.setHeader("Content-Type", "text/html")
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
486 RESPONSE.write("<html>\n<body>\n<pre>")
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
487 json.dump(res, RESPONSE)
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
488 RESPONSE.write("</pre>\n</body>\n</html>")
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
489
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
490 else:
61
e81d034b28a5 more permission handling and table metadata
casties
parents: 60
diff changeset
491 return "ERROR: invalid format"
74
6e19a9af00e6 htmlHead() included
fknauft
parents: 73
diff changeset
492
26
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
493 def checkTableFromXML(self,schema,table,data,REQUEST=None,RESPONSE=None):
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
494 """check the table with the given XML data.
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
495 returns valid data fields and table name."""
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
496 logging.debug("checkTableFromXML schema=%s table=%s"%(schema,table))
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
497 # clean table name
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
498 tablename = sqlName(table)
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
499 tableExists = self.hasTable(schema, table)
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
500 if data is None:
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
501 fieldNames = []
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
502 else:
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
503 # get list of field names from upload file
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
504 fields = self.importExcelXML(schema,tablename,data,fieldsOnly=True)
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
505
26
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
506 res = {'tablename': tablename, 'table_exists': tableExists}
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
507 res['fields'] = fields
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
508 return res
19
78e70dfa7ad6 GoogleMaps related functions
fknauft
parents: 18
diff changeset
509
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
510 def createEmptyTable(self,schema,table,fields):
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
511 """create a table with the given fields
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
512 returns list of created fields"""
15
5e3edf980813 more work on xml import
casties
parents: 14
diff changeset
513 logging.debug("createEmptyTable")
19
78e70dfa7ad6 GoogleMaps related functions
fknauft
parents: 18
diff changeset
514
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
515 sqlFields = []
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
516 for f in fields:
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
517 if isinstance(f,dict):
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
518 # {name: XX, type: YY}
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
519 name = sqlName(f['name'])
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
520 type = f['type']
70
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
521 if hasattr(self, 'toSqlTypeMap'):
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
522 sqltype = self.toSqlTypeMap[type]
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
523 else:
9ec7e32e8ad3 working on maps
casties
parents: 61
diff changeset
524 sqltype = 'text'
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
525
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
526 else:
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
527 # name only
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
528 name = sqlName(f)
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
529 type = 'text'
31
6d2055f1b4fa more work on types
casties
parents: 30
diff changeset
530 sqltype = 'text'
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
531
31
6d2055f1b4fa more work on types
casties
parents: 30
diff changeset
532 sqlFields.append({'name':name, 'type':type, 'sqltype':sqltype})
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
533
60
9fdadb60529f working on authentication and authorization
casties
parents: 58
diff changeset
534 if self.hasTable(schema,table):
9fdadb60529f working on authentication and authorization
casties
parents: 58
diff changeset
535 # TODO: find owner
9fdadb60529f working on authentication and authorization
casties
parents: 58
diff changeset
536 if not self.isAllowed("update", schema, table):
9fdadb60529f working on authentication and authorization
casties
parents: 58
diff changeset
537 raise Unauthorized
9fdadb60529f working on authentication and authorization
casties
parents: 58
diff changeset
538 self.executeSQL('drop table "%s"."%s"'%(schema,table),hasResult=False)
9fdadb60529f working on authentication and authorization
casties
parents: 58
diff changeset
539 else:
9fdadb60529f working on authentication and authorization
casties
parents: 58
diff changeset
540 if not self.isAllowed("create", schema, table):
9fdadb60529f working on authentication and authorization
casties
parents: 58
diff changeset
541 raise Unauthorized
9fdadb60529f working on authentication and authorization
casties
parents: 58
diff changeset
542
43
562717546168 refactoring...
casties
parents: 41
diff changeset
543 fieldString = ", ".join(['"%s" %s'%(f['name'],f['sqltype']) for f in sqlFields])
562717546168 refactoring...
casties
parents: 41
diff changeset
544 sqlString = 'create table "%s"."%s" (%s)'%(schema,table,fieldString)
562717546168 refactoring...
casties
parents: 41
diff changeset
545 logging.debug("createemptytable: SQL=%s"%sqlString)
562717546168 refactoring...
casties
parents: 41
diff changeset
546 self.executeSQL(sqlString,hasResult=False)
562717546168 refactoring...
casties
parents: 41
diff changeset
547 self.setTableMetaTypes(schema,table,sqlFields)
562717546168 refactoring...
casties
parents: 41
diff changeset
548 return sqlFields
19
78e70dfa7ad6 GoogleMaps related functions
fknauft
parents: 18
diff changeset
549
28
9e4f9cfd1edc start adding field structure to xml upload
casties
parents: 27
diff changeset
550 def createTableFromXML(self,schema,table,data, fields=None):
13
e2c73c077533 NEW - # 12: create table and upload data
casties
parents: 12
diff changeset
551 """create or replace a table with the given XML data"""
28
9e4f9cfd1edc start adding field structure to xml upload
casties
parents: 27
diff changeset
552 logging.debug("createTableFromXML schema=%s table=%s data=%s fields=%s"%(schema,table,data,fields))
17
ed997e639cfd more upload table
casties
parents: 16
diff changeset
553 tablename = sqlName(table)
28
9e4f9cfd1edc start adding field structure to xml upload
casties
parents: 27
diff changeset
554 self.importExcelXML(schema, tablename, data, fields)
17
ed997e639cfd more upload table
casties
parents: 16
diff changeset
555 return {"tablename": tablename}
76
e0a79d926902 Path for kml included
fknauft
parents: 55
diff changeset
556
28
9e4f9cfd1edc start adding field structure to xml upload
casties
parents: 27
diff changeset
557 def importExcelXML(self,schema,table,xmldata,fields=None,fieldsOnly=False):
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
558 '''
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
559 Import XML file in Excel format into the table
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
560 @param table: name of the table the xml shall be imported into
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
561 '''
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
562 from xml.dom.pulldom import parseString,parse
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
563
60
9fdadb60529f working on authentication and authorization
casties
parents: 58
diff changeset
564 if not (fieldsOnly or self.isAllowed("create", schema, table)):
9fdadb60529f working on authentication and authorization
casties
parents: 58
diff changeset
565 raise Unauthorized
9fdadb60529f working on authentication and authorization
casties
parents: 58
diff changeset
566
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
567 namespace = "urn:schemas-microsoft-com:office:spreadsheet"
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
568 containerTagName = "Table"
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
569 rowTagName = "Row"
16
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
570 colTagName = "Cell"
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
571 dataTagName = "Data"
26
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
572 xmlFields = []
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
573 sqlFields = []
16
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
574 numFields = 0
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
575 sqlInsert = None
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
576
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
577 logging.debug("import excel xml")
76
e0a79d926902 Path for kml included
fknauft
parents: 55
diff changeset
578
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
579 ret=""
16
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
580 if isinstance(xmldata, str):
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
581 logging.debug("importXML reading string data")
16
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
582 doc=parseString(xmldata)
73
695b6612d4c6 getKmlUrl eingebaut
fknauft
parents: 21
diff changeset
583 else:
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
584 logging.debug("importXML reading file data")
16
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
585 doc=parse(xmldata)
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
586
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
587 cnt = 0
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
588 while True:
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
589 node=doc.getEvent()
16
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
590
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
591 if node is None:
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
592 break
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
593
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
594 else:
16
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
595 #logging.debug("tag=%s"%node[1].localName)
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
596 if node[1].localName is not None:
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
597 tagName = node[1].localName.lower()
76
e0a79d926902 Path for kml included
fknauft
parents: 55
diff changeset
598 else:
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
599 # ignore non-tag nodes
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
600 continue
16
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
601
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
602 if tagName == rowTagName.lower():
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
603 # start of row
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
604 doc.expandNode(node[1])
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
605 cnt += 1
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
606 if cnt == 1:
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
607 # first row -- field names
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
608 names=node[1].getElementsByTagNameNS(namespace, dataTagName)
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
609 for name in names:
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
610 fn = getTextFromNode(name)
26
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
611 xmlFields.append({'name':sqlName(fn),'type':'text'})
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
612
26
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
613 if fieldsOnly:
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
614 # return just field names
26
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
615 return xmlFields
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
616
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
617 # create table
28
9e4f9cfd1edc start adding field structure to xml upload
casties
parents: 27
diff changeset
618 if fields is None:
31
6d2055f1b4fa more work on types
casties
parents: 30
diff changeset
619 fields = xmlFields
6d2055f1b4fa more work on types
casties
parents: 30
diff changeset
620
6d2055f1b4fa more work on types
casties
parents: 30
diff changeset
621 sqlFields = self.createEmptyTable(schema, table, fields)
16
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
622 numFields = len(sqlFields)
15
5e3edf980813 more work on xml import
casties
parents: 14
diff changeset
623 fieldString = ", ".join(['"%s"'%f['name'] for f in sqlFields])
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
624 valString = ", ".join(["%s" for f in sqlFields])
16
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
625 sqlInsert = 'insert into "%s"."%s" (%s) values (%s)'%(schema,table,fieldString,valString)
26
2b73f868d34f NEW - # 12: create table and upload data
casties
parents: 25
diff changeset
626 #logging.debug("importexcelsql: sqlInsert=%s"%sqlInsert)
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
627
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
628 else:
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
629 # following rows are data
16
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
630 colNodes=node[1].getElementsByTagNameNS(namespace, colTagName)
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
631 data = []
16
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
632 hasData = False
227
11dea0923d2f Upload of xml-tables with empty Cells possible
fknauft
parents: 87
diff changeset
633 lineIndex=0
16
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
634 for colNode in colNodes:
227
11dea0923d2f Upload of xml-tables with empty Cells possible
fknauft
parents: 87
diff changeset
635 lineIndex+=1
16
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
636 dataNodes=colNode.getElementsByTagNameNS(namespace, dataTagName)
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
637 if len(dataNodes) > 0:
227
11dea0923d2f Upload of xml-tables with empty Cells possible
fknauft
parents: 87
diff changeset
638 dataIndex=0
228
52112e9a8d72 Upload of xml-tables with empty Cells possible
fknauft
parents: 227
diff changeset
639 if colNode.hasAttribute(u'ss:Index'):
230
9e34cd7b2e57 Upload of xml-tables with empty Cells possible
fknauft
parents: 228
diff changeset
640 dataIndex=int(colNode.getAttribute(u'ss:Index'))
9e34cd7b2e57 Upload of xml-tables with empty Cells possible
fknauft
parents: 228
diff changeset
641 while dataIndex>lineIndex:
228
52112e9a8d72 Upload of xml-tables with empty Cells possible
fknauft
parents: 227
diff changeset
642 data.append(None)
230
9e34cd7b2e57 Upload of xml-tables with empty Cells possible
fknauft
parents: 228
diff changeset
643 lineIndex+=1
9e34cd7b2e57 Upload of xml-tables with empty Cells possible
fknauft
parents: 228
diff changeset
644 else:
9e34cd7b2e57 Upload of xml-tables with empty Cells possible
fknauft
parents: 228
diff changeset
645 val = getTextFromNode(dataNodes[0])
9e34cd7b2e57 Upload of xml-tables with empty Cells possible
fknauft
parents: 228
diff changeset
646 hasData = True
16
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
647 else:
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
648 val = None
19
78e70dfa7ad6 GoogleMaps related functions
fknauft
parents: 18
diff changeset
649
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
650 data.append(val)
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
651
16
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
652 if not hasData:
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
653 # ignore empty rows
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
654 continue
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
655
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
656 # fix number of data fields
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
657 if len(data) > numFields:
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
658 del data[numFields:]
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
659 elif len(data) < numFields:
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
660 missFields = numFields - len(data)
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
661 data.extend(missFields * [None,])
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
662
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
663 logging.debug("importexcel sqlinsert=%s data=%s"%(sqlInsert,data))
17
ed997e639cfd more upload table
casties
parents: 16
diff changeset
664 self.executeSQL(sqlInsert, data, hasResult=False)
14
05933707897f NEW - # 12: create table and upload data
casties
parents: 13
diff changeset
665
16
cbb73d103152 NEW - # 12: create table and upload data
casties
parents: 15
diff changeset
666 return cnt
38
f6d9a3caf986 fixed errors from merging
casties
parents: 37
diff changeset
667
4
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
668 def manage_editRestDbInterface(self, title=None, connection_id=None,
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
669 REQUEST=None):
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
670 """Change the object"""
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
671 if title is not None:
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
672 self.title = title
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
673
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
674 if connection_id is not None:
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
675 self.connection_id = connection_id
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
676
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
677 #checkPermission=getSecurityManager().checkPermission
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
678 REQUEST.RESPONSE.redirect('manage_main')
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly.
casties
parents: 2
diff changeset
679
2
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
680
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
681 manage_addRestDbInterfaceForm=PageTemplateFile('zpt/addRestDbInterface',globals())
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
682
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
683 def manage_addRestDbInterface(self, id, title='', label='', description='',
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
684 createPublic=0,
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
685 createUserF=0,
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
686 REQUEST=None):
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
687 """Add a new object with id *id*."""
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
688
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
689 ob=RestDbInterface(str(id),title)
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
690 self._setObject(id, ob)
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
691
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
692 #checkPermission=getSecurityManager().checkPermission
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
693 REQUEST.RESPONSE.redirect('manage_main')
61a3764cd5fb new version RestDbInterface with working traversal and templates
casties
parents:
diff changeset
694
5
7539a9e69365 small fix
casties
parents: 4
diff changeset
695