ChinaGisRestApi: RestDbInterface.py annotate

annotate RestDbInterface.py @ 47:698ef00f2717

more json store

author	casties
date	Fri, 03 Sep 2010 14:00:09 +0200
parents	ed8db63fab4f
children	f5bfcfa97e7e

rev	line source
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	1 '''
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	2 Created on 19.5.2010
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	3
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	4 @author: casties
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	5 '''
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	6
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	7 from OFS.Folder import Folder
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	8 from Products.PageTemplates.PageTemplateFile import PageTemplateFile
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	9 from Products.ZSQLExtend import ZSQLExtend
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	10 import logging
15 5e3edf980813 more work on xml import casties parents: 14 diff changeset	11 import re
16 cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	12 import psycopg2
18 060797795a4d work on json return values casties parents: 17 diff changeset	13 import json
36 b915bce65372 added kml url option fknauft parents: 9 diff changeset	14 import time
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	15
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	16 from zope.interface import implements
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	17 from zope.publisher.interfaces import IPublishTraverse
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	18 from ZPublisher.BaseRequest import DefaultPublishTraverse
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	19 #from zope.publisher.interfaces import NotFound
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	20 #from zope.app import zapi
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	21 #from zope.component import queryMultiAdapter
4 e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	22 import Shared.DC.ZRDB.DA
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	23 from Products.ZSQLMethods.SQL import SQLConnectionIDs
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	24
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	25
16 cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	26 def getTextFromNode(node):
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	27 """get the cdata content of a XML node"""
16 cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	28 if node is None:
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	29 return ""
16 cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	30
cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	31 if isinstance(node, list):
cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	32 nodelist = node
cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	33 else:
cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	34 nodelist=node.childNodes
cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	35
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	36 rc = ""
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	37 for node in nodelist:
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	38 if node.nodeType == node.TEXT_NODE:
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	39 rc = rc + node.data
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	40 return rc
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	41
16 cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	42 def sqlName(s,lc=True):
cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	43 """returns restricted ASCII-only version of string"""
15 5e3edf980813 more work on xml import casties parents: 14 diff changeset	44 if s is None:
5e3edf980813 more work on xml import casties parents: 14 diff changeset	45 return ""
5e3edf980813 more work on xml import casties parents: 14 diff changeset	46
5e3edf980813 more work on xml import casties parents: 14 diff changeset	47 # all else -> "_"
16 cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	48 s = re.sub(r'[^A-Za-z0-9_]','_',s)
15 5e3edf980813 more work on xml import casties parents: 14 diff changeset	49 if lc:
5e3edf980813 more work on xml import casties parents: 14 diff changeset	50 return s.lower()
5e3edf980813 more work on xml import casties parents: 14 diff changeset	51
5e3edf980813 more work on xml import casties parents: 14 diff changeset	52 return s
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	53
31 6d2055f1b4fa more work on types casties parents: 30 diff changeset	54 gisToSqlTypeMap = {
6d2055f1b4fa more work on types casties parents: 30 diff changeset	55 "text": "text",
32 c732c2ff61d9 fixed sql data types casties parents: 31 diff changeset	56 "number": "numeric",
31 6d2055f1b4fa more work on types casties parents: 30 diff changeset	57 "id": "text",
32 c732c2ff61d9 fixed sql data types casties parents: 31 diff changeset	58 "gis_id": "text",
c732c2ff61d9 fixed sql data types casties parents: 31 diff changeset	59 "coord_lat": "numeric",
c732c2ff61d9 fixed sql data types casties parents: 31 diff changeset	60 "coord_lon": "numeric"
31 6d2055f1b4fa more work on types casties parents: 30 diff changeset	61 }
6d2055f1b4fa more work on types casties parents: 30 diff changeset	62
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	63 class RestDbInterface(Folder):
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	64 """Object for RESTful database queries
7 bd52d9445a41 trying to rework casties parents: 5 diff changeset	65 path schema: /db/{schema}/{table}/
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	66 omitting table gives a list of schemas
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	67 omitting table and schema gives a list of schemas
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	68 """
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	69 implements(IPublishTraverse)
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	70
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	71 meta_type="RESTdb"
4 e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	72 manage_options=Folder.manage_options+(
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	73 {'label':'Config','action':'manage_editRestDbInterfaceForm'},
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	74 )
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	75
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	76 # management templates
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	77 manage_editRestDbInterfaceForm=PageTemplateFile('zpt/editRestDbInterface',globals())
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	78
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	79 # data templates
8 a9a49f5765c9 reworking templates and schema casties parents: 7 diff changeset	80 XML_index = PageTemplateFile('zpt/XML_index', globals())
a9a49f5765c9 reworking templates and schema casties parents: 7 diff changeset	81 XML_schema = PageTemplateFile('zpt/XML_schema', globals())
a9a49f5765c9 reworking templates and schema casties parents: 7 diff changeset	82 XML_schema_table = PageTemplateFile('zpt/XML_schema_table', globals())
a9a49f5765c9 reworking templates and schema casties parents: 7 diff changeset	83 HTML_index = PageTemplateFile('zpt/HTML_index', globals())
a9a49f5765c9 reworking templates and schema casties parents: 7 diff changeset	84 HTML_schema = PageTemplateFile('zpt/HTML_schema', globals())
a9a49f5765c9 reworking templates and schema casties parents: 7 diff changeset	85 HTML_schema_table = PageTemplateFile('zpt/HTML_schema_table', globals())
22 1a4b56716902 NEW - # 12: create table and upload data casties parents: 21 diff changeset	86 JSONHTML_index = PageTemplateFile('zpt/JSONHTML_index', globals())
1a4b56716902 NEW - # 12: create table and upload data casties parents: 21 diff changeset	87 JSONHTML_schema = PageTemplateFile('zpt/JSONHTML_schema', globals())
1a4b56716902 NEW - # 12: create table and upload data casties parents: 21 diff changeset	88 JSONHTML_schema_table = PageTemplateFile('zpt/JSONHTML_schema_table', globals())
24 f3811d9a36da start adding checkuploadtable casties parents: 23 diff changeset	89 # JSON_* templates are scripts
45 ed8db63fab4f more refactoring of templates casties parents: 44 diff changeset	90 def JSON_index(self):
30 51db9e78bf98 working on field types casties parents: 29 diff changeset	91 """JSON index function"""
51db9e78bf98 working on field types casties parents: 29 diff changeset	92 self.REQUEST.RESPONSE.setHeader("Content-Type", "application/json")
45 ed8db63fab4f more refactoring of templates casties parents: 44 diff changeset	93 json.dump(self.getListOfSchemas(), self.REQUEST.RESPONSE)
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	94
45 ed8db63fab4f more refactoring of templates casties parents: 44 diff changeset	95 def JSON_schema(self,schema):
30 51db9e78bf98 working on field types casties parents: 29 diff changeset	96 """JSON index function"""
51db9e78bf98 working on field types casties parents: 29 diff changeset	97 self.REQUEST.RESPONSE.setHeader("Content-Type", "application/json")
45 ed8db63fab4f more refactoring of templates casties parents: 44 diff changeset	98 json.dump(self.getListOfTables(schema), self.REQUEST.RESPONSE)
30 51db9e78bf98 working on field types casties parents: 29 diff changeset	99
44 c6c47034d2a4 more refactoring of templates casties parents: 43 diff changeset	100 def JSON_schema_table(self,schema,table):
30 51db9e78bf98 working on field types casties parents: 29 diff changeset	101 """JSON index function"""
51db9e78bf98 working on field types casties parents: 29 diff changeset	102 self.REQUEST.RESPONSE.setHeader("Content-Type", "application/json")
44 c6c47034d2a4 more refactoring of templates casties parents: 43 diff changeset	103 json.dump(self.getTable(schema, table), self.REQUEST.RESPONSE)
30 51db9e78bf98 working on field types casties parents: 29 diff changeset	104
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	105
4 e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	106 def __init__(self, id, title, connection_id=None):
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	107 """init"""
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	108 self.id = id
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	109 self.title = title
4 e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	110 # database connection id
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	111 self.connection_id = connection_id
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	112 # create template folder
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	113 self.manage_addFolder('template')
44 c6c47034d2a4 more refactoring of templates casties parents: 43 diff changeset	114
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	115
44 c6c47034d2a4 more refactoring of templates casties parents: 43 diff changeset	116 def getRestDbUrl(self):
c6c47034d2a4 more refactoring of templates casties parents: 43 diff changeset	117 """returns url to the RestDb instance"""
c6c47034d2a4 more refactoring of templates casties parents: 43 diff changeset	118 return self.absolute_url()
c6c47034d2a4 more refactoring of templates casties parents: 43 diff changeset	119
22 1a4b56716902 NEW - # 12: create table and upload data casties parents: 21 diff changeset	120 def getJsonString(self,object):
1a4b56716902 NEW - # 12: create table and upload data casties parents: 21 diff changeset	121 """returns a JSON formatted string from object"""
1a4b56716902 NEW - # 12: create table and upload data casties parents: 21 diff changeset	122 return json.dumps(object)
1a4b56716902 NEW - # 12: create table and upload data casties parents: 21 diff changeset	123
17 ed997e639cfd more upload table casties parents: 16 diff changeset	124 def getCursor(self,autocommit=True):
4 e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	125 """returns fresh DB cursor"""
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	126 conn = getattr(self,"_v_database_connection",None)
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	127 if conn is None:
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	128 # create a new connection object
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	129 try:
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	130 if self.connection_id is None:
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	131 # try to take the first existing ID
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	132 connids = SQLConnectionIDs(self)
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	133 if len(connids) > 0:
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	134 connection_id = connids[0][0]
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	135 self.connection_id = connection_id
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	136 logging.debug("connection_id: %s"%repr(connection_id))
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	137
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	138 da = getattr(self, self.connection_id)
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	139 da.connect('')
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	140 # we copy the DAs database connection
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	141 conn = da._v_database_connection
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	142 #conn._register() # register with the Zope transaction system
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	143 self._v_database_connection = conn
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	144 except Exception, e:
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	145 raise IOError("No database connection! (%s)"%str(e))
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	146
5 7539a9e69365 small fix casties parents: 4 diff changeset	147 cursor = conn.getcursor()
17 ed997e639cfd more upload table casties parents: 16 diff changeset	148 if autocommit:
ed997e639cfd more upload table casties parents: 16 diff changeset	149 # is there a better version to get to the connection?
ed997e639cfd more upload table casties parents: 16 diff changeset	150 cursor.connection.set_isolation_level(psycopg2.extensions.ISOLATION_LEVEL_AUTOCOMMIT)
ed997e639cfd more upload table casties parents: 16 diff changeset	151
4 e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	152 return cursor
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	153
17 ed997e639cfd more upload table casties parents: 16 diff changeset	154 def executeSQL(self, query, args=None, hasResult=True, autocommit=True):
4 e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	155 """execute query with args on database and return all results.
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	156 result format: {"fields":fields, "rows":data}"""
16 cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	157 logging.debug("executeSQL query=%s args=%s"%(query,args))
17 ed997e639cfd more upload table casties parents: 16 diff changeset	158 cur = self.getCursor(autocommit=autocommit)
4 e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	159 cur.execute(query, args)
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	160 # description of returned fields
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	161 fields = cur.description
17 ed997e639cfd more upload table casties parents: 16 diff changeset	162 if hasResult:
ed997e639cfd more upload table casties parents: 16 diff changeset	163 # get all data in an array
ed997e639cfd more upload table casties parents: 16 diff changeset	164 data = cur.fetchall()
ed997e639cfd more upload table casties parents: 16 diff changeset	165 cur.close()
ed997e639cfd more upload table casties parents: 16 diff changeset	166 return {"fields":fields, "rows":data}
ed997e639cfd more upload table casties parents: 16 diff changeset	167 else:
ed997e639cfd more upload table casties parents: 16 diff changeset	168 cur.close()
ed997e639cfd more upload table casties parents: 16 diff changeset	169 return None
16 cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	170
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	171 def publishTraverse(self,request,name):
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	172 """change the traversal"""
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	173 # get stored path
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	174 path = request.get('restdb_path', [])
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	175 logging.debug("publishtraverse: name=%s restdb_path=%s"%(name,path))
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	176
13 e2c73c077533 NEW - # 12: create table and upload data casties parents: 12 diff changeset	177 if name in ("index_html", "PUT"):
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	178 # end of traversal
13 e2c73c077533 NEW - # 12: create table and upload data casties parents: 12 diff changeset	179 if request.get("method") == "POST" and request.get("action",None) == "PUT":
e2c73c077533 NEW - # 12: create table and upload data casties parents: 12 diff changeset	180 # fake PUT by POST with action=PUT
e2c73c077533 NEW - # 12: create table and upload data casties parents: 12 diff changeset	181 name = "PUT"
e2c73c077533 NEW - # 12: create table and upload data casties parents: 12 diff changeset	182
e2c73c077533 NEW - # 12: create table and upload data casties parents: 12 diff changeset	183 return getattr(self, name)
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	184 #TODO: should we check more?
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	185 else:
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	186 # traverse
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	187 if len(path) == 0:
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	188 # first segment
8 a9a49f5765c9 reworking templates and schema casties parents: 7 diff changeset	189 if name == 'db':
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	190 # virtual path -- continue traversing
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	191 path = [name]
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	192 request['restdb_path'] = path
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	193 else:
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	194 # try real path
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	195 tr = DefaultPublishTraverse(self, request)
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	196 ob = tr.publishTraverse(request, name)
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	197 return ob
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	198 else:
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	199 path.append(name)
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	200
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	201 # continue traversing
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	202 return self
44 c6c47034d2a4 more refactoring of templates casties parents: 43 diff changeset	203
c6c47034d2a4 more refactoring of templates casties parents: 43 diff changeset	204
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	205 def index_html(self,REQUEST,RESPONSE):
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	206 """index method"""
8 a9a49f5765c9 reworking templates and schema casties parents: 7 diff changeset	207 # ReST path was stored in request
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	208 path = REQUEST.get('restdb_path',[])
36 b915bce65372 added kml url option fknauft parents: 9 diff changeset	209
8 a9a49f5765c9 reworking templates and schema casties parents: 7 diff changeset	210 # type and format are real parameter
26 2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	211 resultFormat = REQUEST.get('format','HTML').upper()
27 a2e4ca3f1cff NEW - # 12: create table and upload data casties parents: 26 diff changeset	212 queryType = REQUEST.get('type',None)
36 b915bce65372 added kml url option fknauft parents: 9 diff changeset	213
26 2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	214 logging.debug("index_html path=%s resultFormat=%s queryType=%s"%(path,resultFormat,queryType))
8 a9a49f5765c9 reworking templates and schema casties parents: 7 diff changeset	215
26 2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	216 if queryType is not None:
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	217 # non-empty queryType -- look for template
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	218 pt = getattr(self.template, "%s_%s"%(resultFormat,queryType), None)
8 a9a49f5765c9 reworking templates and schema casties parents: 7 diff changeset	219 if pt is not None:
26 2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	220 return pt(format=resultFormat,type=queryType,path=path)
8 a9a49f5765c9 reworking templates and schema casties parents: 7 diff changeset	221
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	222 if len(path) == 1:
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	223 # list of schemas
26 2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	224 return self.showListOfSchemas(resultFormat=resultFormat)
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	225 elif len(path) == 2:
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	226 # list of tables
26 2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	227 return self.showListOfTables(resultFormat=resultFormat,schema=path[1])
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	228 elif len(path) == 3:
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	229 # table
25 cef1bfa821cb fixed problem with not passing request casties parents: 24 diff changeset	230 if REQUEST.get("method") == "POST" and REQUEST.get("create_table_file",None) is not None:
24 f3811d9a36da start adding checkuploadtable casties parents: 23 diff changeset	231 # POST to table to check
26 2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	232 return self.checkTable(resultFormat=resultFormat,schema=path[1],table=path[2])
24 f3811d9a36da start adding checkuploadtable casties parents: 23 diff changeset	233 # else show table
26 2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	234 return self.showTable(resultFormat=resultFormat,schema=path[1],table=path[2])
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	235
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	236 # don't know what to do
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	237 return str(REQUEST)
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	238
12 0fa22c291ff1 NEW - # 12: create table and upload data casties parents: 9 diff changeset	239 def PUT(self, REQUEST, RESPONSE):
0fa22c291ff1 NEW - # 12: create table and upload data casties parents: 9 diff changeset	240 """
0fa22c291ff1 NEW - # 12: create table and upload data casties parents: 9 diff changeset	241 Implement WebDAV/HTTP PUT/FTP put method for this object.
0fa22c291ff1 NEW - # 12: create table and upload data casties parents: 9 diff changeset	242 """
13 e2c73c077533 NEW - # 12: create table and upload data casties parents: 12 diff changeset	243 logging.debug("RestDbInterface PUT")
e2c73c077533 NEW - # 12: create table and upload data casties parents: 12 diff changeset	244 #logging.debug("req=%s"%REQUEST)
12 0fa22c291ff1 NEW - # 12: create table and upload data casties parents: 9 diff changeset	245 #self.dav__init(REQUEST, RESPONSE)
0fa22c291ff1 NEW - # 12: create table and upload data casties parents: 9 diff changeset	246 #self.dav__simpleifhandler(REQUEST, RESPONSE)
13 e2c73c077533 NEW - # 12: create table and upload data casties parents: 12 diff changeset	247 # ReST path was stored in request
e2c73c077533 NEW - # 12: create table and upload data casties parents: 12 diff changeset	248 path = REQUEST.get('restdb_path',[])
e2c73c077533 NEW - # 12: create table and upload data casties parents: 12 diff changeset	249 if len(path) == 3:
e2c73c077533 NEW - # 12: create table and upload data casties parents: 12 diff changeset	250 schema = path[1]
e2c73c077533 NEW - # 12: create table and upload data casties parents: 12 diff changeset	251 tablename = path[2]
e2c73c077533 NEW - # 12: create table and upload data casties parents: 12 diff changeset	252 file = REQUEST.get("create_table_file",None)
e2c73c077533 NEW - # 12: create table and upload data casties parents: 12 diff changeset	253 if file is None:
e2c73c077533 NEW - # 12: create table and upload data casties parents: 12 diff changeset	254 RESPONSE.setStatus(400)
e2c73c077533 NEW - # 12: create table and upload data casties parents: 12 diff changeset	255 return
4 e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	256
30 51db9e78bf98 working on field types casties parents: 29 diff changeset	257 fields = None
51db9e78bf98 working on field types casties parents: 29 diff changeset	258 fieldsStr = REQUEST.get("create_table_fields",None)
51db9e78bf98 working on field types casties parents: 29 diff changeset	259 logging.debug("put with schema=%s table=%s file=%s fields=%s"%(schema,tablename,file,repr(fieldsStr)))
51db9e78bf98 working on field types casties parents: 29 diff changeset	260 if fieldsStr is not None:
51db9e78bf98 working on field types casties parents: 29 diff changeset	261 # unpack fields
51db9e78bf98 working on field types casties parents: 29 diff changeset	262 fields = [{"name":n, "type": t} for (n,t) in [f.split(":") for f in fieldsStr.split(",")]]
31 6d2055f1b4fa more work on types casties parents: 30 diff changeset	263
28 9e4f9cfd1edc start adding field structure to xml upload casties parents: 27 diff changeset	264 ret = self.createTableFromXML(schema, tablename, file, fields)
26 2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	265 # return the result as JSON
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	266 format = REQUEST.get("format","JSON")
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	267 if format == "JSON":
20 67ca17753cd5 NEW - # 12: create table and upload data casties parents: 18 diff changeset	268 RESPONSE.setHeader("Content-Type", "application/json")
67ca17753cd5 NEW - # 12: create table and upload data casties parents: 18 diff changeset	269 json.dump(ret, RESPONSE)
26 2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	270
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	271 elif format == "JSONHTML":
20 67ca17753cd5 NEW - # 12: create table and upload data casties parents: 18 diff changeset	272 RESPONSE.setHeader("Content-Type", "text/html")
67ca17753cd5 NEW - # 12: create table and upload data casties parents: 18 diff changeset	273 RESPONSE.write("<html>\n<body>\n<pre>")
67ca17753cd5 NEW - # 12: create table and upload data casties parents: 18 diff changeset	274 json.dump(ret, RESPONSE)
67ca17753cd5 NEW - # 12: create table and upload data casties parents: 18 diff changeset	275 RESPONSE.write("</pre>\n</body>\n</html>")
13 e2c73c077533 NEW - # 12: create table and upload data casties parents: 12 diff changeset	276
e2c73c077533 NEW - # 12: create table and upload data casties parents: 12 diff changeset	277 else:
e2c73c077533 NEW - # 12: create table and upload data casties parents: 12 diff changeset	278 # 400 Bad Request
e2c73c077533 NEW - # 12: create table and upload data casties parents: 12 diff changeset	279 RESPONSE.setStatus(400)
e2c73c077533 NEW - # 12: create table and upload data casties parents: 12 diff changeset	280 return
12 0fa22c291ff1 NEW - # 12: create table and upload data casties parents: 9 diff changeset	281
26 2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	282 def showTable(self,resultFormat='XML',schema='public',table=None,REQUEST=None,RESPONSE=None):
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	283 """returns PageTemplate with tables"""
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	284 logging.debug("showtable")
38 f6d9a3caf986 fixed errors from merging casties parents: 37 diff changeset	285 if REQUEST is None:
f6d9a3caf986 fixed errors from merging casties parents: 37 diff changeset	286 REQUEST = self.REQUEST
f6d9a3caf986 fixed errors from merging casties parents: 37 diff changeset	287
23 860ec92f99df added cross-site access header (for GET) casties parents: 22 diff changeset	288 # should be cross-site accessible
25 cef1bfa821cb fixed problem with not passing request casties parents: 24 diff changeset	289 if RESPONSE is None:
cef1bfa821cb fixed problem with not passing request casties parents: 24 diff changeset	290 RESPONSE = self.REQUEST.RESPONSE
38 f6d9a3caf986 fixed errors from merging casties parents: 37 diff changeset	291
23 860ec92f99df added cross-site access header (for GET) casties parents: 22 diff changeset	292 RESPONSE.setHeader('Access-Control-Allow-Origin', '*')
38 f6d9a3caf986 fixed errors from merging casties parents: 37 diff changeset	293
22 1a4b56716902 NEW - # 12: create table and upload data casties parents: 21 diff changeset	294 # everything else has its own template
26 2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	295 pt = getattr(self.template, '%s_schema_table'%resultFormat, None)
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	296 if pt is None:
26 2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	297 return "ERROR!! template %s_schema_table not found"%resultFormat
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	298
44 c6c47034d2a4 more refactoring of templates casties parents: 43 diff changeset	299 #data = self.getTable(schema,table)
c6c47034d2a4 more refactoring of templates casties parents: 43 diff changeset	300 return pt(schema=schema,table=table)
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	301
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	302 def getTable(self,schema='public',table=None,username='guest'):
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	303 """return table data"""
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	304 logging.debug("gettable")
16 cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	305 data = self.executeSQL('select * from "%s"."%s"'%(schema,table))
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	306 return data
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	307
26 2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	308 def hasTable(self,schema='public',table=None,username='guest'):
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	309 """return if table exists"""
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	310 logging.debug("hastable")
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	311 data = self.executeSQL('select 1 from information_schema.tables where table_schema=%s and table_name=%s',(schema,table))
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	312 ret = bool(data['rows'])
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	313 return ret
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	314
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	315 def showListOfTables(self,resultFormat='XML',schema='public',REQUEST=None,RESPONSE=None):
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	316 """returns PageTemplate with list of tables"""
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	317 logging.debug("showlistoftables")
23 860ec92f99df added cross-site access header (for GET) casties parents: 22 diff changeset	318 # should be cross-site accessible
25 cef1bfa821cb fixed problem with not passing request casties parents: 24 diff changeset	319 if RESPONSE is None:
cef1bfa821cb fixed problem with not passing request casties parents: 24 diff changeset	320 RESPONSE = self.REQUEST.RESPONSE
23 860ec92f99df added cross-site access header (for GET) casties parents: 22 diff changeset	321 RESPONSE.setHeader('Access-Control-Allow-Origin', '*')
24 f3811d9a36da start adding checkuploadtable casties parents: 23 diff changeset	322
26 2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	323 pt = getattr(self.template, '%s_schema'%resultFormat, None)
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	324 if pt is None:
26 2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	325 return "ERROR!! template %s_schema not found"%resultFormat
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	326
45 ed8db63fab4f more refactoring of templates casties parents: 44 diff changeset	327 #data = self.getListOfTables(schema)
ed8db63fab4f more refactoring of templates casties parents: 44 diff changeset	328 return pt(schema=schema)
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	329
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	330 def getListOfTables(self,schema='public',username='guest'):
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	331 """return list of tables"""
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	332 logging.debug("getlistoftables")
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	333 # get list of fields and types of db table
22 1a4b56716902 NEW - # 12: create table and upload data casties parents: 21 diff changeset	334 qstr="""SELECT c.relname AS tablename FROM pg_catalog.pg_class c
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	335 LEFT JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	336 WHERE c.relkind IN ('r','') AND n.nspname NOT IN ('pg_catalog', 'pg_toast')
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	337 AND pg_catalog.pg_table_is_visible(c.oid)"""
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	338 #qstr="select attname, format_type(pg_attribute.atttypid, pg_attribute.atttypmod) from pg_attribute, pg_class where attrelid = pg_class.oid and pg_attribute.attnum > 0"
4 e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	339 data=self.executeSQL(qstr)
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	340 return data
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	341
26 2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	342 def showListOfSchemas(self,resultFormat='XML',REQUEST=None,RESPONSE=None):
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	343 """returns PageTemplate with list of schemas"""
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	344 logging.debug("showlistofschemas")
23 860ec92f99df added cross-site access header (for GET) casties parents: 22 diff changeset	345 # should be cross-site accessible
25 cef1bfa821cb fixed problem with not passing request casties parents: 24 diff changeset	346 if RESPONSE is None:
cef1bfa821cb fixed problem with not passing request casties parents: 24 diff changeset	347 RESPONSE = self.REQUEST.RESPONSE
23 860ec92f99df added cross-site access header (for GET) casties parents: 22 diff changeset	348 RESPONSE.setHeader('Access-Control-Allow-Origin', '*')
24 f3811d9a36da start adding checkuploadtable casties parents: 23 diff changeset	349
26 2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	350 pt = getattr(self.template, '%s_index'%resultFormat, None)
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	351 if pt is None:
26 2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	352 return "ERROR!! template %s_index not found"%resultFormat
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	353
45 ed8db63fab4f more refactoring of templates casties parents: 44 diff changeset	354 #data = self.getListOfSchemas()
ed8db63fab4f more refactoring of templates casties parents: 44 diff changeset	355 return pt()
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	356
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	357 def getListOfSchemas(self,username='guest'):
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	358 """return list of schemas"""
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	359 logging.debug("getlistofschemas")
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	360 # TODO: really look up schemas
9 76ac7a721273 more rework of templates casties parents: 8 diff changeset	361 data={'fields': (('schemas',),), 'rows': [('public',),]}
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	362 return data
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	363
26 2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	364 def checkTable(self,resultFormat,schema,table,REQUEST=None,RESPONSE=None):
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	365 """check the table.
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	366 returns valid data fields and table name."""
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	367 if REQUEST is None:
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	368 REQUEST = self.REQUEST
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	369 RESPONSE = REQUEST.RESPONSE
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	370
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	371 file = REQUEST.get("create_table_file",None)
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	372 res = self.checkTableFromXML(schema, table, file)
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	373 logging.debug("checkTable result=%s"%repr(res))
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	374 # return the result as JSON
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	375 if resultFormat == "JSON":
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	376 RESPONSE.setHeader("Content-Type", "application/json")
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	377 json.dump(res, RESPONSE)
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	378
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	379 elif resultFormat == "JSONHTML":
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	380 RESPONSE.setHeader("Content-Type", "text/html")
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	381 RESPONSE.write("<html>\n<body>\n<pre>")
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	382 json.dump(res, RESPONSE)
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	383 RESPONSE.write("</pre>\n</body>\n</html>")
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	384
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	385 else:
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	386 return "ERROR: invalid resultFormat"
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	387
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	388 def checkTableFromXML(self,schema,table,data,REQUEST=None,RESPONSE=None):
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	389 """check the table with the given XML data.
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	390 returns valid data fields and table name."""
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	391 logging.debug("checkTableFromXML schema=%s table=%s"%(schema,table))
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	392 # clean table name
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	393 tablename = sqlName(table)
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	394 tableExists = self.hasTable(schema, table)
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	395 if data is None:
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	396 fieldNames = []
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	397 else:
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	398 # get list of field names from upload file
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	399 fields = self.importExcelXML(schema,tablename,data,fieldsOnly=True)
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	400
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	401 res = {'tablename': tablename, 'table_exists': tableExists}
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	402 res['fields'] = fields
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	403 return res
2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	404
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	405 def createEmptyTable(self,schema,table,fields):
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	406 """create a table with the given fields
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	407 returns list of created fields"""
15 5e3edf980813 more work on xml import casties parents: 14 diff changeset	408 logging.debug("createEmptyTable")
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	409 sqlFields = []
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	410 for f in fields:
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	411 if isinstance(f,dict):
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	412 # {name: XX, type: YY}
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	413 name = sqlName(f['name'])
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	414 type = f['type']
31 6d2055f1b4fa more work on types casties parents: 30 diff changeset	415 sqltype = gisToSqlTypeMap[type]
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	416
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	417 else:
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	418 # name only
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	419 name = sqlName(f)
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	420 type = 'text'
31 6d2055f1b4fa more work on types casties parents: 30 diff changeset	421 sqltype = 'text'
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	422
31 6d2055f1b4fa more work on types casties parents: 30 diff changeset	423 sqlFields.append({'name':name, 'type':type, 'sqltype':sqltype})
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	424
43 562717546168 refactoring... casties parents: 41 diff changeset	425 self.executeSQL('drop table if exists "%s"."%s"'%(schema,table),hasResult=False)
562717546168 refactoring... casties parents: 41 diff changeset	426 fieldString = ", ".join(['"%s" %s'%(f['name'],f['sqltype']) for f in sqlFields])
562717546168 refactoring... casties parents: 41 diff changeset	427 sqlString = 'create table "%s"."%s" (%s)'%(schema,table,fieldString)
562717546168 refactoring... casties parents: 41 diff changeset	428 logging.debug("createemptytable: SQL=%s"%sqlString)
562717546168 refactoring... casties parents: 41 diff changeset	429 self.executeSQL(sqlString,hasResult=False)
562717546168 refactoring... casties parents: 41 diff changeset	430 self.setTableMetaTypes(schema,table,sqlFields)
562717546168 refactoring... casties parents: 41 diff changeset	431 return sqlFields
26 2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	432
28 9e4f9cfd1edc start adding field structure to xml upload casties parents: 27 diff changeset	433 def createTableFromXML(self,schema,table,data, fields=None):
13 e2c73c077533 NEW - # 12: create table and upload data casties parents: 12 diff changeset	434 """create or replace a table with the given XML data"""
28 9e4f9cfd1edc start adding field structure to xml upload casties parents: 27 diff changeset	435 logging.debug("createTableFromXML schema=%s table=%s data=%s fields=%s"%(schema,table,data,fields))
17 ed997e639cfd more upload table casties parents: 16 diff changeset	436 tablename = sqlName(table)
28 9e4f9cfd1edc start adding field structure to xml upload casties parents: 27 diff changeset	437 self.importExcelXML(schema, tablename, data, fields)
17 ed997e639cfd more upload table casties parents: 16 diff changeset	438 return {"tablename": tablename}
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	439
28 9e4f9cfd1edc start adding field structure to xml upload casties parents: 27 diff changeset	440 def importExcelXML(self,schema,table,xmldata,fields=None,fieldsOnly=False):
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	441 '''
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	442 Import XML file in Excel format into the table
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	443 @param table: name of the table the xml shall be imported into
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	444 '''
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	445 from xml.dom.pulldom import parseString,parse
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	446
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	447 namespace = "urn:schemas-microsoft-com:office:spreadsheet"
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	448 containerTagName = "Table"
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	449 rowTagName = "Row"
16 cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	450 colTagName = "Cell"
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	451 dataTagName = "Data"
26 2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	452 xmlFields = []
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	453 sqlFields = []
16 cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	454 numFields = 0
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	455 sqlInsert = None
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	456
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	457 logging.debug("import excel xml")
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	458
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	459 ret=""
16 cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	460 if isinstance(xmldata, str):
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	461 logging.debug("importXML reading string data")
16 cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	462 doc=parseString(xmldata)
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	463 else:
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	464 logging.debug("importXML reading file data")
16 cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	465 doc=parse(xmldata)
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	466
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	467 cnt = 0
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	468 while True:
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	469 node=doc.getEvent()
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	470
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	471 if node is None:
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	472 break
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	473
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	474 else:
16 cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	475 #logging.debug("tag=%s"%node[1].localName)
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	476 if node[1].localName is not None:
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	477 tagName = node[1].localName.lower()
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	478 else:
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	479 # ignore non-tag nodes
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	480 continue
16 cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	481
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	482 if tagName == rowTagName.lower():
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	483 # start of row
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	484 doc.expandNode(node[1])
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	485 cnt += 1
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	486 if cnt == 1:
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	487 # first row -- field names
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	488 names=node[1].getElementsByTagNameNS(namespace, dataTagName)
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	489 for name in names:
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	490 fn = getTextFromNode(name)
26 2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	491 xmlFields.append({'name':sqlName(fn),'type':'text'})
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	492
26 2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	493 if fieldsOnly:
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	494 # return just field names
26 2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	495 return xmlFields
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	496
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	497 # create table
28 9e4f9cfd1edc start adding field structure to xml upload casties parents: 27 diff changeset	498 if fields is None:
31 6d2055f1b4fa more work on types casties parents: 30 diff changeset	499 fields = xmlFields
6d2055f1b4fa more work on types casties parents: 30 diff changeset	500
6d2055f1b4fa more work on types casties parents: 30 diff changeset	501 sqlFields = self.createEmptyTable(schema, table, fields)
16 cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	502 numFields = len(sqlFields)
15 5e3edf980813 more work on xml import casties parents: 14 diff changeset	503 fieldString = ", ".join(['"%s"'%f['name'] for f in sqlFields])
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	504 valString = ", ".join(["%s" for f in sqlFields])
16 cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	505 sqlInsert = 'insert into "%s"."%s" (%s) values (%s)'%(schema,table,fieldString,valString)
26 2b73f868d34f NEW - # 12: create table and upload data casties parents: 25 diff changeset	506 #logging.debug("importexcelsql: sqlInsert=%s"%sqlInsert)
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	507
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	508 else:
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	509 # following rows are data
16 cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	510 colNodes=node[1].getElementsByTagNameNS(namespace, colTagName)
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	511 data = []
16 cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	512 hasData = False
cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	513 for colNode in colNodes:
cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	514 dataNodes=colNode.getElementsByTagNameNS(namespace, dataTagName)
cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	515 if len(dataNodes) > 0:
cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	516 val = getTextFromNode(dataNodes[0])
cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	517 hasData = True
cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	518 else:
cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	519 val = None
cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	520
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	521 data.append(val)
05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	522
16 cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	523 if not hasData:
cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	524 # ignore empty rows
cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	525 continue
cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	526
cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	527 # fix number of data fields
cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	528 if len(data) > numFields:
cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	529 del data[numFields:]
cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	530 elif len(data) < numFields:
cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	531 missFields = numFields - len(data)
cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	532 data.extend(missFields * [None,])
cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	533
cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	534 logging.debug("importexcel sqlinsert=%s data=%s"%(sqlInsert,data))
17 ed997e639cfd more upload table casties parents: 16 diff changeset	535 self.executeSQL(sqlInsert, data, hasResult=False)
14 05933707897f NEW - # 12: create table and upload data casties parents: 13 diff changeset	536
16 cbb73d103152 NEW - # 12: create table and upload data casties parents: 15 diff changeset	537 return cnt
38 f6d9a3caf986 fixed errors from merging casties parents: 37 diff changeset	538
4 e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	539 def manage_editRestDbInterface(self, title=None, connection_id=None,
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	540 REQUEST=None):
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	541 """Change the object"""
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	542 if title is not None:
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	543 self.title = title
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	544
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	545 if connection_id is not None:
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	546 self.connection_id = connection_id
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	547
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	548 #checkPermission=getSecurityManager().checkPermission
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	549 REQUEST.RESPONSE.redirect('manage_main')
e3ee1f358fe6 new version that doesn't use ZSQLExtend but the database connection more directly. casties parents: 2 diff changeset	550
2 61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	551
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	552 manage_addRestDbInterfaceForm=PageTemplateFile('zpt/addRestDbInterface',globals())
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	553
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	554 def manage_addRestDbInterface(self, id, title='', label='', description='',
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	555 createPublic=0,
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	556 createUserF=0,
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	557 REQUEST=None):
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	558 """Add a new object with id id."""
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	559
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	560 ob=RestDbInterface(str(id),title)
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	561 self._setObject(id, ob)
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	562
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	563 #checkPermission=getSecurityManager().checkPermission
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	564 REQUEST.RESPONSE.redirect('manage_main')
61a3764cd5fb new version RestDbInterface with working traversal and templates casties parents: diff changeset	565
5 7539a9e69365 small fix casties parents: 4 diff changeset	566

Mercurial > hg > ChinaGisRestApi

annotate RestDbInterface.py @ 47:698ef00f2717