Mercurial > hg > drupalISMI

annotate importFromOpenMind/importer/model2model.py @ 25:5bdcb5805d29

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
updated openmind-networkx-neo4j conversion with dates, locations and links.
author casties
date Thu, 24 Sep 2015 18:17:41 +0200
parents 97f2da68fb5f
children 248bf8d1e2e7
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
24
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
1 import networkx as nx
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
2 import sys
25
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
3 import csv
24
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
4
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
5 ## configure behaviour
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
6
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
7 # metworkx graph files
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
8 input_fn = 'ismi_graph.gpickle'
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
9 output_fn = 'ismi_graph_mod.gpickle'
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
10
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
11 # operations
25
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
12 ops = ['locate', 'contract', 'inv_rels', 'add_links']
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
13
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
14 # types of object to locate
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
15 locate_objects_of_type = ['PLACE']
24
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
16
25
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
17 # file with place location information
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
18 places_fn = 'ismi_places_loc.csv'
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
19
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
20 # node types to remove from the graph
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
21 #remove_objects_of_type = ['DIGITALIZATION', 'REFERENCE']
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
22
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
23 # add relations to these objects as attributes with the relation's name
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
24 contract_relations_into_attributes = {'PLACE': ['label', 'latitude', 'longitude'],
24
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
25 'ALIAS': ['label']}
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
26
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
27
25
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
28 # add URLs to nodes using an attribute in a pattern
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
29 add_link_attributes = {'ismi_id': 'https://ismi-dev.mpiwg-berlin.mpg.de/drupal-ismi/entity/%s'}
24
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
30
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
31
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
32 def fixName(name, is_src_rel=False, is_tar_rel=False, att_from_rel=False):
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
33 # these are too embarrassing...
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
34 if 'FLORUIT' in name:
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
35 name = name.replace('FLORUIT', 'FLOURISH')
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
36
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
37 elif 'floruit' in name:
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
38 name = name.replace('floruit', 'flourish')
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
39
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
40 if is_src_rel:
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
41 #name = name + '>'
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
42 pass
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
43
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
44 if is_tar_rel:
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
45 name = '<' + name
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
46
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
47 if att_from_rel:
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
48 # clean up relations as attribute names
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
49 name = name.replace('is_', '')
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
50 name = name.replace('has_', '')
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
51 name = name.replace('was_', '')
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
52 name = name.replace('_of', '')
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
53
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
54 return name
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
55
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
56
25
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
57 def locatePlaces(nx_graph):
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
58 """add location information to objects in the graph"""
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
59
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
60 print("Adding location information from %s to %s."%(places_fn, locate_objects_of_type))
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
61 cnt = 0
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
62
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
63 # read place location file
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
64 locations = {}
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
65 with open(places_fn, encoding='utf-8') as csvfile:
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
66 reader = csv.DictReader(csvfile)
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
67 for row in reader:
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
68 lat = row['Latitude']
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
69 lon = row['Longitude']
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
70 name = row['Address']
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
71 if lat and lon:
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
72 locations[name] = {'latitude': lat, 'longitude': lon}
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
73
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
74 # iterate all nodes
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
75 for n in nx.nodes_iter(nx_graph):
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
76 attrs = nx_graph.node[n]
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
77 if attrs['type'] in locate_objects_of_type:
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
78 # locatable object
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
79 name = attrs['label']
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
80 if name in locations:
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
81 # place name match
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
82 location = locations[name]
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
83 attrs['latitude'] = location['latitude']
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
84 attrs['longitude'] = location['longitude']
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
85
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
86 else:
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
87 print("WARNING: no location for name '%s'"%name)
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
88
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
89 cnt += 1
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
90 if cnt % 100 == 0:
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
91 print(" %s nodes"%cnt)
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
92
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
93
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
94
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
95 def genAttName(attrs, name):
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
96 """Generate new attribute name.
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
97 """
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
98 while attrs.get(name, None) is not None:
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
99 # attribute exists
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
100 if name[-1].isnumeric(): # increment last digit
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
101 name = name[:-1] + str(int(name[-1]) + 1)
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
102 else:
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
103 name += '2'
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
104
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
105 return name
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
106
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
107
24
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
108 def contractRelations(nx_graph):
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
109 """contract relations into attributes"""
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
110
25
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
111 print("Contracting relations to attributes.")
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
112 cnt = 0
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
113 for nx_edge in nx.edges_iter(nx_graph):
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
114 (nx_src, nx_tar) = nx_edge
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
115 # get attributes of edge
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
116 rel_attrs = nx_graph.edge[nx_src][nx_tar][0]
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
117 rel_type = rel_attrs['type']
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
118 # get attributes of source and target nodes
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
119 src_attrs = nx_graph.node[nx_src]
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
120 tar_attrs = nx_graph.node[nx_tar]
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
121
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
122 # contract source relations
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
123 tar_type = tar_attrs['type']
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
124 if tar_type in contract_relations_into_attributes:
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
125 # get list of attributes to transfer
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
126 transfer_atts = contract_relations_into_attributes[tar_type]
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
127 for transfer_att in transfer_atts:
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
128 if transfer_att not in tar_attrs:
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
129 # target has no attribute
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
130 continue
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
131
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
132 # name for new attribute starts with relation name
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
133 att_name = fixName(rel_type, att_from_rel=True)
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
134 # then attribute name
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
135 if transfer_att != 'label':
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
136 att_name += "_%s"%transfer_att
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
137
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
138 # then generate unique name
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
139 att_name = genAttName(src_attrs, att_name)
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
140 # add target node's attribute
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
141 src_attrs[att_name] = tar_attrs.get(transfer_att)
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
142
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
143 # contract target relations
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
144 src_type = src_attrs['type']
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
145 if src_type in contract_relations_into_attributes:
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
146 # get list of attributes to transfer
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
147 transfer_atts = contract_relations_into_attributes[src_type]
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
148 for transfer_att in transfer_atts:
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
149 if transfer_att not in src_attrs:
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
150 # target has no attribute
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
151 continue
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
152
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
153 # name for new attribute starts with relation name
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
154 att_name = fixName(rel_type, att_from_rel=True)
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
155 # then attribute name
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
156 if transfer_att != 'label':
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
157 att_name += "_%s"%transfer_att
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
158
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
159 # then generate unique name
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
160 att_name = genAttName(tar_attrs, att_name)
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
161 # add target node's attribute
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
162 tar_attrs[att_name] = src_attrs.get(transfer_att)
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
163
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
164 cnt += 1
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
165 if cnt % 100 == 0:
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
166 print(" %s relations"%cnt)
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
167
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
168
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
169 def invertRelations(nx_graph):
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
170 """Add inverse relations to each relation"""
24
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
171
25
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
172 print("Adding inverse relations.")
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
173 # copy list of edges because we add edges in the loop
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
174 edges = nx.edges(nx_graph)[:]
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
175 # iterate list
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
176 cnt = 0
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
177 for nx_edge in edges:
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
178 (nx_src, nx_tar) = nx_edge
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
179 # get attributes of edge
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
180 rel_attrs = nx_graph.edge[nx_src][nx_tar][0]
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
181 rel_type = rel_attrs['type']
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
182 rel_id = rel_attrs['ismi_id']
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
183 # create new relation
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
184 nx_graph.add_edge(nx_tar, nx_src, type=fixName(rel_type, is_tar_rel=True), ismi_id=-rel_id)
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
185
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
186 cnt += 1
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
187 if cnt % 100 == 0:
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
188 print(" %s relations"%cnt)
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
189
24
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
190
25
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
191 def addLinks(nx_graph):
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
192 """Add link attributes to all nodes."""
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
193
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
194 print("Adding links: %s"%repr(add_link_attributes))
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
195 cnt = 0
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
196 for link_att, link_pattern in add_link_attributes.items():
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
197 # iterate all nodes
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
198 for n in nx.nodes_iter(nx_graph):
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
199 attrs = nx_graph.node[n]
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
200 if link_att in attrs:
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
201 url = link_pattern%attrs[link_att]
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
202 # TODO: which target attribute for multiple?
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
203 attrs['link'] = url
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
204
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
205 cnt += 1
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
206 if cnt % 100 == 0:
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
207 print(" %s nodes"%cnt)
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
208
24
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
209
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
210 ## main
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
211
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
212 print("Modify networkx graph")
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
213
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
214 # read commandline parameters
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
215 if len(sys.argv) > 2:
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
216 input_fn = sys.argv[1]
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
217 output_fn = sys.argv[2]
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
218 #ops = sys.argv[3]
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
219
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
220 # read networkx graph from pickle
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
221 print("Reading graph from %s"%input_fn)
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
222 nx_graph = nx.read_gpickle(input_fn)
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
223 print("Graph info: %s"%nx.info(nx_graph))
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
224
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
225 # operate
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
226 for op in ops:
25
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
227 if op == 'locate':
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
228 locatePlaces(nx_graph)
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
229
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
230 elif op == 'contract':
24
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
231 contractRelations(nx_graph)
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
232
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
233 elif op == 'inv_rels':
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
234 invertRelations(nx_graph)
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
235
25
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
236 elif op == 'add_links':
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
237 addLinks(nx_graph)
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
238
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
239 print("Writing graph to %s"%output_fn)
5bdcb5805d29 updated openmind-networkx-neo4j conversion with dates, locations and links.
casties
parents: 24
diff changeset
240 nx_graph = nx.write_gpickle(nx_graph, output_fn)
24
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
241
97f2da68fb5f first version of model2model graph manipulation tool. doesn't work yet.
casties
parents:
diff changeset
242 print("Done.")