1 """GNUmed database object business class.
2
3 Overview
4 --------
5 This class wraps a source relation (table, view) which
6 represents an entity that makes immediate business sense
7 such as a vaccination or a medical document. In many if
8 not most cases this source relation is a denormalizing
9 view. The data in that view will in most cases, however,
10 originate from several normalized tables. One instance
11 of this class represents one row of said source relation.
12
13 Note, however, that this class does not *always* simply
14 wrap a single table or view. It can also encompass several
15 relations (views, tables, sequences etc) that taken together
16 form an object meaningful to *business* logic.
17
18 Initialization
19 --------------
20 There are two ways to initialize an instance with values.
21 One way is to pass a "primary key equivalent" object into
22 __init__(). Refetch_payload() will then pull the data from
23 the backend. Another way would be to fetch the data outside
24 the instance and pass it in via the <row> argument. In that
25 case the instance will not initially connect to the databse
26 which may offer a great boost to performance.
27
28 Values API
29 ----------
30 Field values are cached for later access. They can be accessed
31 by a dictionary API, eg:
32
33 old_value = object['field']
34 object['field'] = new_value
35
36 The field names correspond to the respective column names
37 in the "main" source relation. Accessing non-existant field
38 names will raise an error, so does trying to set fields not
39 listed in self.__class__._updatable_fields. To actually
40 store updated values in the database one must explicitly
41 call save_payload().
42
43 The class will in many cases be enhanced by accessors to
44 related data that is not directly part of the business
45 object itself but are closely related, such as codes
46 linked to a clinical narrative entry (eg a diagnosis). Such
47 accessors in most cases start with get_*. Related setters
48 start with set_*. The values can be accessed via the
49 object['field'] syntax, too, but they will be cached
50 independantly.
51
52 Concurrency handling
53 --------------------
54 GNUmed connections always run transactions in isolation level
55 "serializable". This prevents transactions happening at the
56 *very same time* to overwrite each other's data. All but one
57 of them will abort with a concurrency error (eg if a
58 transaction runs a select-for-update later than another one
59 it will hang until the first transaction ends. Then it will
60 succeed or fail depending on what the first transaction
61 did). This is standard transactional behaviour.
62
63 However, another transaction may have updated our row
64 between the time we first fetched the data and the time we
65 start the update transaction. This is noticed by getting the
66 XMIN system column for the row when initially fetching the
67 data and using that value as a where condition value when
68 updating the row later. If the row had been updated (xmin
69 changed) or deleted (primary key disappeared) in the
70 meantime the update will touch zero rows (as no row with
71 both PK and XMIN matching is found) even if the query itself
72 syntactically succeeds.
73
74 When detecting a change in a row due to XMIN being different
75 one needs to be careful how to represent that to the user.
76 The row may simply have changed but it also might have been
77 deleted and a completely new and unrelated row which happens
78 to have the same primary key might have been created ! This
79 row might relate to a totally different context (eg. patient,
80 episode, encounter).
81
82 One can offer all the data to the user:
83
84 self.original_payload
85 - contains the data at the last successful refetch
86
87 self.modified_payload
88 - contains the modified payload just before the last
89 failure of save_payload() - IOW what is currently
90 in the database
91
92 self._payload
93 - contains the currently active payload which may or
94 may not contain changes
95
96 For discussion on this see the thread starting at:
97
98 http://archives.postgresql.org/pgsql-general/2004-10/msg01352.php
99
100 and here
101
102 http://groups.google.com/group/pgsql.general/browse_thread/thread/e3566ba76173d0bf/6cf3c243a86d9233
103 (google for "XMIN semantic at peril")
104
105 Problem cases with XMIN:
106
107 1) not unlikely
108 - a very old row is read with XMIN
109 - vacuum comes along and sets XMIN to FrozenTransactionId
110 - now XMIN changed but the row actually didn't !
111 - an update with "... where xmin = old_xmin ..." fails
112 although there is no need to fail
113
114 2) quite unlikely
115 - a row is read with XMIN
116 - a long time passes
117 - the original XMIN gets frozen to FrozenTransactionId
118 - another writer comes along and changes the row
119 - incidentally the exact same old row gets the old XMIN *again*
120 - now XMIN is (again) the same but the data changed !
121 - a later update fails to detect the concurrent change !!
122
123 TODO:
124 The solution is to use our own column for optimistic locking
125 which gets updated by an AFTER UPDATE trigger.
126 """
127
128 __version__ = "$Revision: 1.60 $"
129 __author__ = "K.Hilbert <Karsten.Hilbert@gmx.net>"
130 __license__ = "GPL"
131
132 import sys, copy, types, inspect, logging, datetime
133
134
135 if __name__ == '__main__':
136 sys.path.insert(0, '../../')
137 from Gnumed.pycommon import gmExceptions, gmPG2
138
139
140 _log = logging.getLogger('gm.db')
141 _log.info(__version__)
142
144 """Represents business objects in the database.
145
146 Rules:
147 - instances ARE ASSUMED TO EXIST in the database
148 - PK construction (aPK_obj): DOES verify its existence on instantiation
149 (fetching data fails)
150 - Row construction (row): allowed by using a dict of pairs
151 field name: field value (PERFORMANCE improvement)
152 - does NOT verify FK target existence
153 - does NOT create new entries in the database
154 - does NOT lazy-fetch fields on access
155
156 Class scope SQL commands and variables:
157
158 <_cmd_fetch_payload>
159 - must return exactly one row
160 - where clause argument values are expected
161 in self.pk_obj (taken from __init__(aPK_obj))
162 - must return xmin of all rows that _cmds_store_payload
163 will be updating, so views must support the xmin columns
164 of their underlying tables
165
166 <_cmds_store_payload>
167 - one or multiple "update ... set ... where xmin_* = ..." statements
168 which actually update the database from the data in self._payload,
169 - the last query must refetch at least the XMIN values needed to detect
170 concurrent updates, their field names had better be the same as
171 in _cmd_fetch_payload,
172 - when subclasses tend to live a while after save_payload() was
173 called and they support computed fields (say, _(some_column)
174 you need to return *all* columns (see cEncounter)
175
176 <_updatable_fields>
177 - a list of fields available for update via object['field']
178
179
180 A template for new child classes:
181
182 *********** start of template ***********
183
184 #------------------------------------------------------------
185 from Gnumed.pycommon import gmBusinessDBObject
186 from Gnumed.pycommon import gmPG2
187
188 #============================================================
189 # short description
190 #------------------------------------------------------------
191 # use plural form, search-replace get_XXX
192 _SQL_get_XXX = u\"""
193 SELECT *, (xmin AS xmin_XXX)
194 FROM XXX.v_XXX
195 WHERE %s
196 \"""
197
198 class cXxxXxx(gmBusinessDBObject.cBusinessDBObject):
199 \"""Represents ...\"""
200
201 _cmd_fetch_payload = _SQL_get_XXX % u"pk_XXX = %s"
202 _cmds_store_payload = [
203 u\"""
204 UPDATE xxx.xxx SET -- typically the underlying table name
205 xxx = %(xxx)s, -- typically "table_col = %(view_col)s"
206 xxx = gm.nullify_empty_string(%(xxx)s)
207 WHERE
208 pk = %(pk_XXX)s
209 AND
210 xmin = %(xmin_XXX)s
211 RETURNING
212 pk as pk_XXX,
213 xmin as xmin_XXX
214 \"""
215 ]
216 # view columns that can be updated:
217 _updatable_fields = [
218 u'xxx',
219 u'xxx'
220 ]
221 #--------------------------------------------------------
222 def format(self):
223 return u'%s' % self
224
225 #------------------------------------------------------------
226 def get_XXX(order_by=None):
227 if order_by is None:
228 order_by = u'true'
229 else:
230 order_by = u'true ORDER BY %s' % order_by
231
232 cmd = _SQL_get_XXX % order_by
233 rows, idx = gmPG2.run_ro_queries(queries = [{'cmd': cmd}], get_col_idx = True)
234 return [ cXxxXxx(row = {'data': r, 'idx': idx, 'pk_field': 'xxx'}) for r in rows ]
235 #------------------------------------------------------------
236 def create_xxx(xxx=None, xxx=None):
237
238 args = {
239 u'xxx': xxx,
240 u'xxx': xxx
241 }
242 cmd = u\"""
243 INSERT INTO xxx.xxx (
244 xxx,
245 xxx,
246 xxx
247 ) VALUES (
248 %(xxx)s,
249 %(xxx)s,
250 gm.nullify_empty_string(%(xxx)s)
251 )
252 RETURNING pk
253 \"""
254 rows, idx = gmPG2.run_rw_queries(queries = [{'cmd': cmd, 'args': args}], return_data = True, get_col_idx = False)
255
256 return cXxxXxx(aPK_obj = rows[0]['pk'])
257 #------------------------------------------------------------
258 def delete_xxx(xxx=None):
259 args = {'pk': xxx}
260 cmd = u"DELETE FROM xxx.xxx WHERE pk = %(pk)s"
261 gmPG2.run_rw_queries(queries = [{'cmd': cmd, 'args': args}])
262 return True
263 #------------------------------------------------------------
264
265 *********** end of template ***********
266
267 """
268
269 - def __init__(self, aPK_obj=None, row=None):
270 """Init business object.
271
272 Call from child classes:
273
274 super(cChildClass, self).__init__(aPK_obj = aPK_obj, row = row)
275 """
276
277
278
279 self.pk_obj = '<uninitialized>'
280 self._idx = {}
281 self._payload = []
282 self._ext_cache = {}
283 self._is_modified = False
284
285
286 self.__class__._cmd_fetch_payload
287 self.__class__._cmds_store_payload
288 self.__class__._updatable_fields
289
290 if aPK_obj is not None:
291 self.__init_from_pk(aPK_obj=aPK_obj)
292 else:
293 self._init_from_row_data(row=row)
294
295 self._is_modified = False
296
298 """Creates a new clinical item instance by its PK.
299
300 aPK_obj can be:
301 - a simple value
302 * the primary key WHERE condition must be
303 a simple column
304 - a dictionary of values
305 * the primary key where condition must be a
306 subselect consuming the dict and producing
307 the single-value primary key
308 """
309 self.pk_obj = aPK_obj
310 result = self.refetch_payload()
311 if result is True:
312 self.original_payload = {}
313 for field in self._idx.keys():
314 self.original_payload[field] = self._payload[self._idx[field]]
315 return True
316
317 if result is False:
318 raise gmExceptions.ConstructorError, "[%s:%s]: error loading instance" % (self.__class__.__name__, self.pk_obj)
319
321 """Creates a new clinical item instance given its fields.
322
323 row must be a dict with the fields:
324 - pk_field: the name of the primary key field
325 - idx: a dict mapping field names to position
326 - data: the field values in a list (as returned by
327 cursor.fetchone() in the DB-API)
328
329 row = {'data': row, 'idx': idx, 'pk_field': 'the PK column name'}
330
331 rows, idx = gmPG2.run_ro_queries(queries = [{'cmd': cmd, 'args': args}], get_col_idx = True)
332 objects = [ cChildClass(row = {'data': r, 'idx': idx, 'pk_field': 'the PK column name'}) for r in rows ]
333 """
334 try:
335 self._idx = row['idx']
336 self._payload = row['data']
337 self.pk_obj = self._payload[self._idx[row['pk_field']]]
338 except:
339 _log.exception('faulty <row> argument structure: %s' % row)
340 raise gmExceptions.ConstructorError, "[%s:??]: error loading instance from row data" % self.__class__.__name__
341
342 if len(self._idx.keys()) != len(self._payload):
343 _log.critical('field index vs. payload length mismatch: %s field names vs. %s fields' % (len(self._idx.keys()), len(self._payload)))
344 _log.critical('faulty <row> argument structure: %s' % row)
345 raise gmExceptions.ConstructorError, "[%s:??]: error loading instance from row data" % self.__class__.__name__
346
347 self.original_payload = {}
348 for field in self._idx.keys():
349 self.original_payload[field] = self._payload[self._idx[field]]
350
352 if self.__dict__.has_key('_is_modified'):
353 if self._is_modified:
354 _log.critical('[%s:%s]: loosing payload changes' % (self.__class__.__name__, self.pk_obj))
355 _log.debug('original: %s' % self.original_payload)
356 _log.debug('modified: %s' % self._payload)
357
359 tmp = []
360 try:
361 for attr in self._idx.keys():
362 if self._payload[self._idx[attr]] is None:
363 tmp.append(u'%s: NULL' % attr)
364 else:
365 tmp.append('%s: >>%s<<' % (attr, self._payload[self._idx[attr]]))
366 return '[%s:%s]: %s' % (self.__class__.__name__, self.pk_obj, str(tmp))
367 except:
368 return 'nascent [%s @ %s], cannot show payload and primary key' %(self.__class__.__name__, id(self))
369
371
372
373
374 try:
375 return self._payload[self._idx[attribute]]
376 except KeyError:
377 pass
378
379
380 getter = getattr(self, 'get_%s' % attribute, None)
381 if not callable(getter):
382 _log.warning('[%s]: no attribute [%s]' % (self.__class__.__name__, attribute))
383 _log.warning('[%s]: valid attributes: %s' % (self.__class__.__name__, str(self._idx.keys())))
384 _log.warning('[%s]: no getter method [get_%s]' % (self.__class__.__name__, attribute))
385 methods = filter(lambda x: x[0].startswith('get_'), inspect.getmembers(self, inspect.ismethod))
386 _log.warning('[%s]: valid getter methods: %s' % (self.__class__.__name__, str(methods)))
387 raise KeyError('[%s]: cannot read from key [%s]' % (self.__class__.__name__, attribute))
388
389 self._ext_cache[attribute] = getter()
390 return self._ext_cache[attribute]
391
393
394
395 if attribute in self.__class__._updatable_fields:
396 try:
397 if self._payload[self._idx[attribute]] != value:
398 self._payload[self._idx[attribute]] = value
399 self._is_modified = True
400 return
401 except KeyError:
402 _log.warning('[%s]: cannot set attribute <%s> despite marked settable' % (self.__class__.__name__, attribute))
403 _log.warning('[%s]: supposedly settable attributes: %s' % (self.__class__.__name__, str(self.__class__._updatable_fields)))
404 raise KeyError('[%s]: cannot write to key [%s]' % (self.__class__.__name__, attribute))
405
406
407 if hasattr(self, 'set_%s' % attribute):
408 setter = getattr(self, "set_%s" % attribute)
409 if not callable(setter):
410 raise AttributeError('[%s] setter [set_%s] not callable' % (self.__class__.__name__, attribute))
411 try:
412 del self._ext_cache[attribute]
413 except KeyError:
414 pass
415 if type(value) is types.TupleType:
416 if setter(*value):
417 self._is_modified = True
418 return
419 raise AttributeError('[%s]: setter [%s] failed for [%s]' % (self.__class__.__name__, setter, value))
420 if setter(value):
421 self._is_modified = True
422 return
423
424
425 _log.error('[%s]: cannot find attribute <%s> or setter method [set_%s]' % (self.__class__.__name__, attribute, attribute))
426 _log.warning('[%s]: settable attributes: %s' % (self.__class__.__name__, str(self.__class__._updatable_fields)))
427 methods = filter(lambda x: x[0].startswith('set_'), inspect.getmembers(self, inspect.ismethod))
428 _log.warning('[%s]: valid setter methods: %s' % (self.__class__.__name__, str(methods)))
429 raise AttributeError('[%s]: cannot set [%s]' % (self.__class__.__name__, attribute))
430
431
432
434 raise NotImplementedError('comparison between [%s] and [%s] not implemented' % (self, another_object))
435
437 return self._is_modified
438
440 try:
441 return self._idx.keys()
442 except AttributeError:
443 return 'nascent [%s @ %s], cannot return keys' %(self.__class__.__name__, id(self))
444
447
449 data = {}
450 for field in self._idx.keys():
451 val = self._payload[self._idx[field]]
452 if val is None:
453 data[field] = u''
454 continue
455 if isinstance(val, datetime.datetime):
456 try:
457 data[field] = val.strftime(date_format).decode('utf8', 'replace')
458 except ValueError:
459 data[field] = val.isoformat()
460 continue
461 try:
462 data[field] = unicode(val, encoding = 'utf8', errors = 'replace')
463 except TypeError:
464 try:
465 data[field] = unicode(val)
466 except (UnicodeDecodeError, TypeError):
467 val = '%s' % str(val)
468 data[field] = val.decode('utf8', 'replace')
469 return data
470
472 _log.error('[%s:%s]: forgot to override get_patient()' % (self.__class__.__name__, self.pk_obj))
473 return None
474
477
479 """Fetch field values from backend.
480 """
481 if self._is_modified:
482 if ignore_changes:
483 _log.critical('[%s:%s]: loosing payload changes' % (self.__class__.__name__, self.pk_obj))
484 _log.debug('original: %s' % self.original_payload)
485 _log.debug('modified: %s' % self._payload)
486 else:
487 _log.critical('[%s:%s]: cannot reload, payload changed' % (self.__class__.__name__, self.pk_obj))
488 return False
489
490 if type(self.pk_obj) == types.DictType:
491 arg = self.pk_obj
492 else:
493 arg = [self.pk_obj]
494 rows, self._idx = gmPG2.run_ro_queries (
495 queries = [{'cmd': self.__class__._cmd_fetch_payload, 'args': arg}],
496 get_col_idx = True
497 )
498 if len(rows) == 0:
499 _log.error('[%s:%s]: no such instance' % (self.__class__.__name__, self.pk_obj))
500 return False
501 self._payload = rows[0]
502 return True
503
506
507 - def save(self, conn=None):
509
511 """Store updated values (if any) in database.
512
513 Optionally accepts a pre-existing connection
514 - returns a tuple (<True|False>, <data>)
515 - True: success
516 - False: an error occurred
517 * data is (error, message)
518 * for error meanings see gmPG2.run_rw_queries()
519 """
520 if not self._is_modified:
521 return (True, None)
522
523 args = {}
524 for field in self._idx.keys():
525 args[field] = self._payload[self._idx[field]]
526 self.modified_payload = args
527
528 close_conn = self.__noop
529 if conn is None:
530 conn = gmPG2.get_connection(readonly=False)
531 close_conn = conn.close
532
533
534
535
536
537
538 queries = []
539 for query in self.__class__._cmds_store_payload:
540 queries.append({'cmd': query, 'args': args})
541 rows, idx = gmPG2.run_rw_queries (
542 link_obj = conn,
543 queries = queries,
544 return_data = True,
545 get_col_idx = True
546 )
547
548
549
550
551
552 if len(rows) == 0:
553 return (False, (u'cannot update row', _('[%s:%s]: row not updated (nothing returned), row in use ?') % (self.__class__.__name__, self.pk_obj)))
554
555
556 row = rows[0]
557 for key in idx:
558 try:
559 self._payload[self._idx[key]] = row[idx[key]]
560 except KeyError:
561 conn.rollback()
562 close_conn()
563 _log.error('[%s:%s]: cannot update instance, XMIN refetch key mismatch on [%s]' % (self.__class__.__name__, self.pk_obj, key))
564 _log.error('payload keys: %s' % str(self._idx))
565 _log.error('XMIN refetch keys: %s' % str(idx))
566 _log.error(args)
567 raise
568
569 conn.commit()
570 close_conn()
571
572 self._is_modified = False
573
574 self.original_payload = {}
575 for field in self._idx.keys():
576 self.original_payload[field] = self._payload[self._idx[field]]
577
578 return (True, None)
579
580
582
583 """ turn the data into a list of dicts, adding "class hints".
584 all objects get turned into dictionaries which the other end
585 will interpret as "object", via the __jsonclass__ hint,
586 as specified by the JSONRPC protocol standard.
587 """
588 if isinstance(obj, list):
589 return map(jsonclasshintify, obj)
590 elif isinstance(obj, gmPG2.dbapi.tz.FixedOffsetTimezone):
591
592
593 res = {'__jsonclass__': ["jsonobjproxy.FixedOffsetTimezone"]}
594 res['name'] = obj._name
595 res['offset'] = jsonclasshintify(obj._offset)
596 return res
597 elif isinstance(obj, datetime.timedelta):
598
599
600 res = {'__jsonclass__': ["jsonobjproxy.TimeDelta"]}
601 res['days'] = obj.days
602 res['seconds'] = obj.seconds
603 res['microseconds'] = obj.microseconds
604 return res
605 elif isinstance(obj, datetime.time):
606
607
608 res = {'__jsonclass__': ["jsonobjproxy.Time"]}
609 res['hour'] = obj.hour
610 res['minute'] = obj.minute
611 res['second'] = obj.second
612 res['microsecond'] = obj.microsecond
613 res['tzinfo'] = jsonclasshintify(obj.tzinfo)
614 return res
615 elif isinstance(obj, datetime.datetime):
616
617
618 res = {'__jsonclass__': ["jsonobjproxy.DateTime"]}
619 res['year'] = obj.year
620 res['month'] = obj.month
621 res['day'] = obj.day
622 res['hour'] = obj.hour
623 res['minute'] = obj.minute
624 res['second'] = obj.second
625 res['microsecond'] = obj.microsecond
626 res['tzinfo'] = jsonclasshintify(obj.tzinfo)
627 return res
628 elif isinstance(obj, cBusinessDBObject):
629
630
631 res = {'__jsonclass__': ["jsonobjproxy.%s" % obj.__class__.__name__]}
632 for k in obj.get_fields():
633 t = jsonclasshintify(obj[k])
634 res[k] = t
635 print "props", res, dir(obj)
636 for attribute in dir(obj):
637 if not attribute.startswith("get_"):
638 continue
639 k = attribute[4:]
640 if res.has_key(k):
641 continue
642 getter = getattr(obj, attribute, None)
643 if callable(getter):
644 res[k] = jsonclasshintify(getter())
645 return res
646 return obj
647
648
649 if __name__ == '__main__':
650
651 if len(sys.argv) < 2:
652 sys.exit()
653
654 if sys.argv[1] != u'test':
655 sys.exit()
656
657
668
669 from Gnumed.pycommon import gmI18N
670 gmI18N.activate_locale()
671 gmI18N.install_domain()
672
673 data = {
674 'pk_field': 'bogus_pk',
675 'idx': {'bogus_pk': 0, 'bogus_field': 1, 'bogus_date': 2},
676 'data': [-1, 'bogus_data', datetime.datetime.now()]
677 }
678 obj = cTestObj(row=data)
679
680
681
682 print obj.fields_as_dict()
683
684
685