Mercurial > hg > drupalISMI

annotate importFromOpenMind/importer/model2neo4j_import.py @ 48:6625019a0c96

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
author casties
date Tue, 07 Feb 2017 21:06:13 +0100
parents
children 763b5d29fa5e
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
48
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
1 import networkx as nx
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
2 import csv
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
3 import sys
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
4
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
5 ## configure behaviour
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
6
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
7 # metworkx graph file
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
8 input_fn = 'ismi_graph.gpickle'
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
9
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
10 # label added to all nodes
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
11 project_label = '_ismi'
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
12
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
13 # neo4j import file
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
14 node_import_fn = "neo4j-nodes.csv"
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
15 relation_import_fn = "neo4j-relations.csv"
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
16
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
17 # name of type attribute
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
18 node_type_attribute = '_type'
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
19 rel_type_attribute = '_type'
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
20
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
21
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
22 ## setup
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
23
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
24 n4j_nodes = {}
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
25 n4j_relations = {}
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
26
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
27 # active log levels for logging
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
28 logLevels = {'DEBUG', 'INFO', 'WARNING', 'ERROR', 'SYSMSG'}
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
29 #logLevels = {'INFO', 'WARNING', 'ERROR', 'SYSMSG'}
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
30 #logLevels = {'INFO', 'ERROR', 'SYSMSG'}
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
31
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
32 def log(level, message):
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
33 if level in logLevels:
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
34 print("%s: %s"%(level, message))
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
35
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
36
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
37 def getNodeFields(nx_graph):
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
38 """returns a set of field names for nodes from nx_graph"""
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
39
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
40 log('INFO', "Creating node field list")
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
41 ismi_fields = set()
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
42 # collect types of all nodes
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
43 for node_id in nx.nodes_iter(nx_graph):
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
44 attrs = nx_graph.node[node_id]
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
45 # save all attribute names
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
46 for att in attrs.keys():
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
47 ismi_fields.add(att)
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
48
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
49 return list(ismi_fields)
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
50
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
51
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
52 def getRelationFields(nx_graph):
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
53 """returns a set of field names for relations from nx_graph"""
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
54
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
55 log('INFO', "Creating node field list")
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
56 ismi_fields = set()
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
57 # collect types of all relations
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
58 for nx_edge in nx.edges_iter(nx_graph):
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
59 (nx_src, nx_tar) = nx_edge
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
60 # get attributes of edge
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
61 attrs = nx_graph.edge[nx_src][nx_tar][0].copy()
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
62 # save all attribute names
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
63 for att in attrs.keys():
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
64 ismi_fields.add(att)
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
65
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
66 return list(ismi_fields)
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
67
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
68
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
69 def writeNodes(nx_graph, node_writer):
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
70 """write all nodes from nx_graph to node_writer"""
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
71
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
72 log('INFO', "Writing nodes to CSV file")
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
73 cnt = 0
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
74 for node_id in nx.nodes_iter(nx_graph):
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
75 attrs = nx_graph.node[node_id].copy()
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
76 # get entity type
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
77 ntype = attrs[node_type_attribute]
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
78 # add as label
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
79 attrs[':LABEL'] = ntype
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
80 # get ismi_id
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
81 ismi_id = attrs['ismi_id']
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
82 # add ismi_id as node id
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
83 attrs[':ID'] = ismi_id
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
84 # write row
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
85 node_writer.writerow(attrs)
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
86 # save node id
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
87 n4j_nodes[ismi_id] = ismi_id
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
88
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
89 cnt += 1
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
90 if cnt % 100 == 0:
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
91 log('INFO', "%s nodes"%cnt)
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
92
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
93 log('INFO', "%s nodes written"%cnt)
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
94
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
95 def writeRelations(nx_graph, relation_writer):
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
96 """write all relations from nx_graph to node_writer"""
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
97
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
98 log('INFO', "Writing relations to CSV file")
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
99 cnt = 0
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
100 for nx_edge in nx.edges_iter(nx_graph):
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
101 (nx_src, nx_tar) = nx_edge
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
102 # get attributes of edge
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
103 attrs = nx_graph.edge[nx_src][nx_tar][0].copy()
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
104 # get relation type
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
105 rtype = attrs[rel_type_attribute]
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
106 # add as label
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
107 attrs[':TYPE'] = rtype
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
108
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
109 # get ismi_id of source and target nodes
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
110 src_id = nx_graph.node[nx_src]['ismi_id']
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
111 tar_id = nx_graph.node[nx_tar]['ismi_id']
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
112 # check Neo4J nodes
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
113 src = n4j_nodes.get(src_id, None)
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
114 if src is None:
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
115 log("ERROR", "src node %s missing!"%src_id)
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
116 break
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
117
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
118 tar = n4j_nodes.get(tar_id, None)
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
119 if tar is None:
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
120 log("ERROR", "tar node %s missing!"%tar_id)
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
121 break
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
122
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
123 # use as start and end id
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
124 attrs[':START_ID'] = src_id
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
125 attrs[':END_ID'] = tar_id
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
126
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
127 # write row
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
128 relation_writer.writerow(attrs)
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
129
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
130 cnt += 1
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
131 if cnt % 100 == 0:
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
132 log('INFO', "%s relations"%cnt)
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
133
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
134 log('INFO', "%s relations written"%cnt)
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
135
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
136 ## main
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
137
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
138 log('SYSINFO', "Copy graph from networkx to Neo4J")
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
139
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
140 # read commandline parameters
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
141 if len(sys.argv) > 1:
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
142 input_fn = sys.argv[1]
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
143
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
144 if len(sys.argv) > 3:
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
145 node_import_fn = sys.argv[2]
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
146 relation_import_fn = sys.argv[3]
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
147
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
148 # read networkx graph from pickle
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
149 log('SYSINFO', "Reading graph from %s"%input_fn)
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
150 nx_graph = nx.read_gpickle(input_fn)
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
151 log('SYSINFO', "Graph info: %s"%nx.info(nx_graph))
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
152
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
153 # get field lists
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
154 node_fields = [':LABEL', ':ID'] + getNodeFields(nx_graph)
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
155 relation_fields = [':TYPE', ':START_ID', ':END_ID'] + getRelationFields(nx_graph)
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
156
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
157 # write neo4j CSV import files
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
158 log('SYSINFO', "Opening node import file at %s"%node_import_fn)
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
159 with open(node_import_fn, 'w', newline='', encoding='utf-8') as csvfile:
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
160 writer = csv.DictWriter(csvfile, fieldnames=node_fields, dialect=csv.excel, quoting=csv.QUOTE_NONNUMERIC)
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
161 writer.writeheader()
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
162 writeNodes(nx_graph, writer)
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
163
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
164 # write neo4j CSV import files
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
165 log('SYSINFO', "Opening relation import file at %s"%relation_import_fn)
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
166 with open(relation_import_fn, 'w', newline='', encoding='utf-8') as csvfile:
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
167 writer = csv.DictWriter(csvfile, fieldnames=relation_fields, dialect=csv.excel, quoting=csv.QUOTE_NONNUMERIC)
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
168 writer.writeheader()
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
169 writeRelations(nx_graph, writer)
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
170
6625019a0c96 old model2neo4j renamed to model2neo4j_restclient. new model2neo4j_client and model2neo4j_import. fixed ismixml2model and compare_models.
casties
parents:
diff changeset
171 print("Done.")