1 """GNUmed database object business class.
2
3 Overview
4 --------
5 This class wraps a source relation (table, view) which
6 represents an entity that makes immediate business sense
7 such as a vaccination or a medical document. In many if
8 not most cases this source relation is a denormalizing
9 view. The data in that view will in most cases, however,
10 originate from several normalized tables. One instance
11 of this class represents one row of said source relation.
12
13 Note, however, that this class does not *always* simply
14 wrap a single table or view. It can also encompass several
15 relations (views, tables, sequences etc) that taken together
16 form an object meaningful to *business* logic.
17
18 Initialization
19 --------------
20 There are two ways to initialize an instance with values.
21 One way is to pass a "primary key equivalent" object into
22 __init__(). Refetch_payload() will then pull the data from
23 the backend. Another way would be to fetch the data outside
24 the instance and pass it in via the <row> argument. In that
25 case the instance will not initially connect to the databse
26 which may offer a great boost to performance.
27
28 Values API
29 ----------
30 Field values are cached for later access. They can be accessed
31 by a dictionary API, eg:
32
33 old_value = object['field']
34 object['field'] = new_value
35
36 The field names correspond to the respective column names
37 in the "main" source relation. Accessing non-existant field
38 names will raise an error, so does trying to set fields not
39 listed in self.__class__._updatable_fields. To actually
40 store updated values in the database one must explicitly
41 call save_payload().
42
43 The class will in many cases be enhanced by accessors to
44 related data that is not directly part of the business
45 object itself but are closely related, such as codes
46 linked to a clinical narrative entry (eg a diagnosis). Such
47 accessors in most cases start with get_*. Related setters
48 start with set_*. The values can be accessed via the
49 object['field'] syntax, too, but they will be cached
50 independantly.
51
52 Concurrency handling
53 --------------------
54 GNUmed connections always run transactions in isolation level
55 "serializable". This prevents transactions happening at the
56 *very same time* to overwrite each other's data. All but one
57 of them will abort with a concurrency error (eg if a
58 transaction runs a select-for-update later than another one
59 it will hang until the first transaction ends. Then it will
60 succeed or fail depending on what the first transaction
61 did). This is standard transactional behaviour.
62
63 However, another transaction may have updated our row
64 between the time we first fetched the data and the time we
65 start the update transaction. This is noticed by getting the
66 XMIN system column for the row when initially fetching the
67 data and using that value as a where condition value when
68 updating the row later. If the row had been updated (xmin
69 changed) or deleted (primary key disappeared) in the
70 meantime the update will touch zero rows (as no row with
71 both PK and XMIN matching is found) even if the query itself
72 syntactically succeeds.
73
74 When detecting a change in a row due to XMIN being different
75 one needs to be careful how to represent that to the user.
76 The row may simply have changed but it also might have been
77 deleted and a completely new and unrelated row which happens
78 to have the same primary key might have been created ! This
79 row might relate to a totally different context (eg. patient,
80 episode, encounter).
81
82 One can offer all the data to the user:
83
84 self.original_payload
85 - contains the data at the last successful refetch
86
87 self.modified_payload
88 - contains the modified payload just before the last
89 failure of save_payload() - IOW what is currently
90 in the database
91
92 self._payload
93 - contains the currently active payload which may or
94 may not contain changes
95
96 For discussion on this see the thread starting at:
97
98 http://archives.postgresql.org/pgsql-general/2004-10/msg01352.php
99
100 and here
101
102 http://groups.google.com/group/pgsql.general/browse_thread/thread/e3566ba76173d0bf/6cf3c243a86d9233
103 (google for "XMIN semantic at peril")
104
105 Problem cases with XMIN:
106
107 1) not unlikely
108 - a very old row is read with XMIN
109 - vacuum comes along and sets XMIN to FrozenTransactionId
110 - now XMIN changed but the row actually didn't !
111 - an update with "... where xmin = old_xmin ..." fails
112 although there is no need to fail
113
114 2) quite unlikely
115 - a row is read with XMIN
116 - a long time passes
117 - the original XMIN gets frozen to FrozenTransactionId
118 - another writer comes along and changes the row
119 - incidentally the exact same old row gets the old XMIN *again*
120 - now XMIN is (again) the same but the data changed !
121 - a later update fails to detect the concurrent change !!
122
123 TODO:
124 The solution is to use our own column for optimistic locking
125 which gets updated by an AFTER UPDATE trigger.
126 """
127
128 __version__ = "$Revision: 1.60 $"
129 __author__ = "K.Hilbert <Karsten.Hilbert@gmx.net>"
130 __license__ = "GPL"
131
132 import sys, copy, types, inspect, logging, datetime
133
134
135 if __name__ == '__main__':
136 sys.path.insert(0, '../../')
137 from Gnumed.pycommon import gmExceptions, gmPG2
138 from Gnumed.pycommon.gmTools import tex_escape_string
139
140
141 _log = logging.getLogger('gm.db')
142 _log.info(__version__)
143
145 """Represents business objects in the database.
146
147 Rules:
148 - instances ARE ASSUMED TO EXIST in the database
149 - PK construction (aPK_obj): DOES verify its existence on instantiation
150 (fetching data fails)
151 - Row construction (row): allowed by using a dict of pairs
152 field name: field value (PERFORMANCE improvement)
153 - does NOT verify FK target existence
154 - does NOT create new entries in the database
155 - does NOT lazy-fetch fields on access
156
157 Class scope SQL commands and variables:
158
159 <_cmd_fetch_payload>
160 - must return exactly one row
161 - where clause argument values are expected
162 in self.pk_obj (taken from __init__(aPK_obj))
163 - must return xmin of all rows that _cmds_store_payload
164 will be updating, so views must support the xmin columns
165 of their underlying tables
166
167 <_cmds_store_payload>
168 - one or multiple "update ... set ... where xmin_* = ..." statements
169 which actually update the database from the data in self._payload,
170 - the last query must refetch at least the XMIN values needed to detect
171 concurrent updates, their field names had better be the same as
172 in _cmd_fetch_payload,
173 - when subclasses tend to live a while after save_payload() was
174 called and they support computed fields (say, _(some_column)
175 you need to return *all* columns (see cEncounter)
176
177 <_updatable_fields>
178 - a list of fields available for update via object['field']
179
180
181 A template for new child classes:
182
183 *********** start of template ***********
184
185 #------------------------------------------------------------
186 from Gnumed.pycommon import gmBusinessDBObject
187 from Gnumed.pycommon import gmPG2
188
189 #============================================================
190 # short description
191 #------------------------------------------------------------
192 # use plural form, search-replace get_XXX
193 _SQL_get_XXX = u\"""
194 SELECT *, (xmin AS xmin_XXX)
195 FROM XXX.v_XXX
196 WHERE %s
197 \"""
198
199 class cXxxXxx(gmBusinessDBObject.cBusinessDBObject):
200 \"""Represents ...\"""
201
202 _cmd_fetch_payload = _SQL_get_XXX % u"pk_XXX = %s"
203 _cmds_store_payload = [
204 u\"""
205 UPDATE xxx.xxx SET -- typically the underlying table name
206 xxx = %(xxx)s, -- typically "table_col = %(view_col)s"
207 xxx = gm.nullify_empty_string(%(xxx)s)
208 WHERE
209 pk = %(pk_XXX)s
210 AND
211 xmin = %(xmin_XXX)s
212 RETURNING
213 pk as pk_XXX,
214 xmin as xmin_XXX
215 \"""
216 ]
217 # view columns that can be updated:
218 _updatable_fields = [
219 u'xxx',
220 u'xxx'
221 ]
222 #--------------------------------------------------------
223 def format(self):
224 return u'%s' % self
225
226 #------------------------------------------------------------
227 def get_XXX(order_by=None):
228 if order_by is None:
229 order_by = u'true'
230 else:
231 order_by = u'true ORDER BY %s' % order_by
232
233 cmd = _SQL_get_XXX % order_by
234 rows, idx = gmPG2.run_ro_queries(queries = [{'cmd': cmd}], get_col_idx = True)
235 return [ cXxxXxx(row = {'data': r, 'idx': idx, 'pk_field': 'xxx'}) for r in rows ]
236 #------------------------------------------------------------
237 def create_xxx(xxx=None, xxx=None):
238
239 args = {
240 u'xxx': xxx,
241 u'xxx': xxx
242 }
243 cmd = u\"""
244 INSERT INTO xxx.xxx (
245 xxx,
246 xxx,
247 xxx
248 ) VALUES (
249 %(xxx)s,
250 %(xxx)s,
251 gm.nullify_empty_string(%(xxx)s)
252 )
253 RETURNING pk
254 \"""
255 rows, idx = gmPG2.run_rw_queries(queries = [{'cmd': cmd, 'args': args}], return_data = True, get_col_idx = False)
256
257 return cXxxXxx(aPK_obj = rows[0]['pk'])
258 #------------------------------------------------------------
259 def delete_xxx(xxx=None):
260 args = {'pk': xxx}
261 cmd = u"DELETE FROM xxx.xxx WHERE pk = %(pk)s"
262 gmPG2.run_rw_queries(queries = [{'cmd': cmd, 'args': args}])
263 return True
264 #------------------------------------------------------------
265
266 *********** end of template ***********
267
268 """
269
270 - def __init__(self, aPK_obj=None, row=None):
271 """Init business object.
272
273 Call from child classes:
274
275 super(cChildClass, self).__init__(aPK_obj = aPK_obj, row = row)
276 """
277
278
279
280 self.pk_obj = '<uninitialized>'
281 self._idx = {}
282 self._payload = []
283 self._ext_cache = {}
284 self._is_modified = False
285
286
287 self.__class__._cmd_fetch_payload
288 self.__class__._cmds_store_payload
289 self.__class__._updatable_fields
290
291 if aPK_obj is not None:
292 self.__init_from_pk(aPK_obj=aPK_obj)
293 else:
294 self._init_from_row_data(row=row)
295
296 self._is_modified = False
297
299 """Creates a new clinical item instance by its PK.
300
301 aPK_obj can be:
302 - a simple value
303 * the primary key WHERE condition must be
304 a simple column
305 - a dictionary of values
306 * the primary key where condition must be a
307 subselect consuming the dict and producing
308 the single-value primary key
309 """
310 self.pk_obj = aPK_obj
311 result = self.refetch_payload()
312 if result is True:
313 self.original_payload = {}
314 for field in self._idx.keys():
315 self.original_payload[field] = self._payload[self._idx[field]]
316 return True
317
318 if result is False:
319 raise gmExceptions.ConstructorError, "[%s:%s]: error loading instance" % (self.__class__.__name__, self.pk_obj)
320
322 """Creates a new clinical item instance given its fields.
323
324 row must be a dict with the fields:
325 - pk_field: the name of the primary key field
326 - idx: a dict mapping field names to position
327 - data: the field values in a list (as returned by
328 cursor.fetchone() in the DB-API)
329
330 row = {'data': row, 'idx': idx, 'pk_field': 'the PK column name'}
331
332 rows, idx = gmPG2.run_ro_queries(queries = [{'cmd': cmd, 'args': args}], get_col_idx = True)
333 objects = [ cChildClass(row = {'data': r, 'idx': idx, 'pk_field': 'the PK column name'}) for r in rows ]
334 """
335 try:
336 self._idx = row['idx']
337 self._payload = row['data']
338 self.pk_obj = self._payload[self._idx[row['pk_field']]]
339 except:
340 _log.exception('faulty <row> argument structure: %s' % row)
341 raise gmExceptions.ConstructorError, "[%s:??]: error loading instance from row data" % self.__class__.__name__
342
343 if len(self._idx.keys()) != len(self._payload):
344 _log.critical('field index vs. payload length mismatch: %s field names vs. %s fields' % (len(self._idx.keys()), len(self._payload)))
345 _log.critical('faulty <row> argument structure: %s' % row)
346 raise gmExceptions.ConstructorError, "[%s:??]: error loading instance from row data" % self.__class__.__name__
347
348 self.original_payload = {}
349 for field in self._idx.keys():
350 self.original_payload[field] = self._payload[self._idx[field]]
351
353 if self.__dict__.has_key('_is_modified'):
354 if self._is_modified:
355 _log.critical('[%s:%s]: loosing payload changes' % (self.__class__.__name__, self.pk_obj))
356 _log.debug('original: %s' % self.original_payload)
357 _log.debug('modified: %s' % self._payload)
358
360 tmp = []
361 try:
362 for attr in self._idx.keys():
363 if self._payload[self._idx[attr]] is None:
364 tmp.append(u'%s: NULL' % attr)
365 else:
366 tmp.append('%s: >>%s<<' % (attr, self._payload[self._idx[attr]]))
367 return '[%s:%s]: %s' % (self.__class__.__name__, self.pk_obj, str(tmp))
368 except:
369 return 'nascent [%s @ %s], cannot show payload and primary key' %(self.__class__.__name__, id(self))
370
372
373
374
375 try:
376 return self._payload[self._idx[attribute]]
377 except KeyError:
378 pass
379
380
381 getter = getattr(self, 'get_%s' % attribute, None)
382 if not callable(getter):
383 _log.warning('[%s]: no attribute [%s]' % (self.__class__.__name__, attribute))
384 _log.warning('[%s]: valid attributes: %s' % (self.__class__.__name__, str(self._idx.keys())))
385 _log.warning('[%s]: no getter method [get_%s]' % (self.__class__.__name__, attribute))
386 methods = filter(lambda x: x[0].startswith('get_'), inspect.getmembers(self, inspect.ismethod))
387 _log.warning('[%s]: valid getter methods: %s' % (self.__class__.__name__, str(methods)))
388 raise KeyError('[%s]: cannot read from key [%s]' % (self.__class__.__name__, attribute))
389
390 self._ext_cache[attribute] = getter()
391 return self._ext_cache[attribute]
392
394
395
396 if attribute in self.__class__._updatable_fields:
397 try:
398 if self._payload[self._idx[attribute]] != value:
399 self._payload[self._idx[attribute]] = value
400 self._is_modified = True
401 return
402 except KeyError:
403 _log.warning('[%s]: cannot set attribute <%s> despite marked settable' % (self.__class__.__name__, attribute))
404 _log.warning('[%s]: supposedly settable attributes: %s' % (self.__class__.__name__, str(self.__class__._updatable_fields)))
405 raise KeyError('[%s]: cannot write to key [%s]' % (self.__class__.__name__, attribute))
406
407
408 if hasattr(self, 'set_%s' % attribute):
409 setter = getattr(self, "set_%s" % attribute)
410 if not callable(setter):
411 raise AttributeError('[%s] setter [set_%s] not callable' % (self.__class__.__name__, attribute))
412 try:
413 del self._ext_cache[attribute]
414 except KeyError:
415 pass
416 if type(value) is types.TupleType:
417 if setter(*value):
418 self._is_modified = True
419 return
420 raise AttributeError('[%s]: setter [%s] failed for [%s]' % (self.__class__.__name__, setter, value))
421 if setter(value):
422 self._is_modified = True
423 return
424
425
426 _log.error('[%s]: cannot find attribute <%s> or setter method [set_%s]' % (self.__class__.__name__, attribute, attribute))
427 _log.warning('[%s]: settable attributes: %s' % (self.__class__.__name__, str(self.__class__._updatable_fields)))
428 methods = filter(lambda x: x[0].startswith('set_'), inspect.getmembers(self, inspect.ismethod))
429 _log.warning('[%s]: valid setter methods: %s' % (self.__class__.__name__, str(methods)))
430 raise AttributeError('[%s]: cannot set [%s]' % (self.__class__.__name__, attribute))
431
432
433
435 raise NotImplementedError('comparison between [%s] and [%s] not implemented' % (self, another_object))
436
438 return self._is_modified
439
441 try:
442 return self._idx.keys()
443 except AttributeError:
444 return 'nascent [%s @ %s], cannot return keys' %(self.__class__.__name__, id(self))
445
448
449 - def fields_as_dict(self, date_format='%c', none_string=u'', escape_style=None, bool_strings=None):
450 if bool_strings is None:
451 bools = {True: u'true', False: u'false'}
452 else:
453 bools = {True: bool_strings[0], False: bool_strings[1]}
454 data = {}
455 for field in self._idx.keys():
456
457
458
459
460 val = self._payload[self._idx[field]]
461 if val is None:
462 data[field] = none_string
463 continue
464 if isinstance(val, bool):
465 data[field] = bools[val]
466 continue
467 if isinstance(val, datetime.datetime):
468 try:
469 data[field] = val.strftime(date_format).decode('utf8', 'replace')
470 except ValueError:
471 data[field] = val.isoformat()
472 if escape_style in [u'latex', u'tex']:
473 data[field] = tex_escape_string(data[field])
474 continue
475 try:
476 data[field] = unicode(val, encoding = 'utf8', errors = 'replace')
477 except TypeError:
478 try:
479 data[field] = unicode(val)
480 except (UnicodeDecodeError, TypeError):
481 val = '%s' % str(val)
482 data[field] = val.decode('utf8', 'replace')
483 if escape_style in [u'latex', u'tex']:
484 data[field] = tex_escape_string(data[field])
485
486 return data
487
489 _log.error('[%s:%s]: forgot to override get_patient()' % (self.__class__.__name__, self.pk_obj))
490 return None
491
494
496 """Fetch field values from backend.
497 """
498 if self._is_modified:
499 if ignore_changes:
500 _log.critical('[%s:%s]: loosing payload changes' % (self.__class__.__name__, self.pk_obj))
501 _log.debug('original: %s' % self.original_payload)
502 _log.debug('modified: %s' % self._payload)
503 else:
504 _log.critical('[%s:%s]: cannot reload, payload changed' % (self.__class__.__name__, self.pk_obj))
505 return False
506
507 if type(self.pk_obj) == types.DictType:
508 arg = self.pk_obj
509 else:
510 arg = [self.pk_obj]
511 rows, self._idx = gmPG2.run_ro_queries (
512 queries = [{'cmd': self.__class__._cmd_fetch_payload, 'args': arg}],
513 get_col_idx = True
514 )
515 if len(rows) == 0:
516 _log.error('[%s:%s]: no such instance' % (self.__class__.__name__, self.pk_obj))
517 return False
518 self._payload = rows[0]
519 return True
520
523
524 - def save(self, conn=None):
526
528 """Store updated values (if any) in database.
529
530 Optionally accepts a pre-existing connection
531 - returns a tuple (<True|False>, <data>)
532 - True: success
533 - False: an error occurred
534 * data is (error, message)
535 * for error meanings see gmPG2.run_rw_queries()
536 """
537 if not self._is_modified:
538 return (True, None)
539
540 args = {}
541 for field in self._idx.keys():
542 args[field] = self._payload[self._idx[field]]
543 self.modified_payload = args
544
545 close_conn = self.__noop
546 if conn is None:
547 conn = gmPG2.get_connection(readonly=False)
548 close_conn = conn.close
549
550
551
552
553
554
555 queries = []
556 for query in self.__class__._cmds_store_payload:
557 queries.append({'cmd': query, 'args': args})
558 rows, idx = gmPG2.run_rw_queries (
559 link_obj = conn,
560 queries = queries,
561 return_data = True,
562 get_col_idx = True
563 )
564
565
566
567
568
569 if len(rows) == 0:
570 return (False, (u'cannot update row', _('[%s:%s]: row not updated (nothing returned), row in use ?') % (self.__class__.__name__, self.pk_obj)))
571
572
573 row = rows[0]
574 for key in idx:
575 try:
576 self._payload[self._idx[key]] = row[idx[key]]
577 except KeyError:
578 conn.rollback()
579 close_conn()
580 _log.error('[%s:%s]: cannot update instance, XMIN refetch key mismatch on [%s]' % (self.__class__.__name__, self.pk_obj, key))
581 _log.error('payload keys: %s' % str(self._idx))
582 _log.error('XMIN refetch keys: %s' % str(idx))
583 _log.error(args)
584 raise
585
586 conn.commit()
587 close_conn()
588
589 self._is_modified = False
590
591 self.original_payload = {}
592 for field in self._idx.keys():
593 self.original_payload[field] = self._payload[self._idx[field]]
594
595 return (True, None)
596
597
599
600 """ turn the data into a list of dicts, adding "class hints".
601 all objects get turned into dictionaries which the other end
602 will interpret as "object", via the __jsonclass__ hint,
603 as specified by the JSONRPC protocol standard.
604 """
605 if isinstance(obj, list):
606 return map(jsonclasshintify, obj)
607 elif isinstance(obj, gmPG2.dbapi.tz.FixedOffsetTimezone):
608
609
610 res = {'__jsonclass__': ["jsonobjproxy.FixedOffsetTimezone"]}
611 res['name'] = obj._name
612 res['offset'] = jsonclasshintify(obj._offset)
613 return res
614 elif isinstance(obj, datetime.timedelta):
615
616
617 res = {'__jsonclass__': ["jsonobjproxy.TimeDelta"]}
618 res['days'] = obj.days
619 res['seconds'] = obj.seconds
620 res['microseconds'] = obj.microseconds
621 return res
622 elif isinstance(obj, datetime.time):
623
624
625 res = {'__jsonclass__': ["jsonobjproxy.Time"]}
626 res['hour'] = obj.hour
627 res['minute'] = obj.minute
628 res['second'] = obj.second
629 res['microsecond'] = obj.microsecond
630 res['tzinfo'] = jsonclasshintify(obj.tzinfo)
631 return res
632 elif isinstance(obj, datetime.datetime):
633
634
635 res = {'__jsonclass__': ["jsonobjproxy.DateTime"]}
636 res['year'] = obj.year
637 res['month'] = obj.month
638 res['day'] = obj.day
639 res['hour'] = obj.hour
640 res['minute'] = obj.minute
641 res['second'] = obj.second
642 res['microsecond'] = obj.microsecond
643 res['tzinfo'] = jsonclasshintify(obj.tzinfo)
644 return res
645 elif isinstance(obj, cBusinessDBObject):
646
647
648 res = {'__jsonclass__': ["jsonobjproxy.%s" % obj.__class__.__name__]}
649 for k in obj.get_fields():
650 t = jsonclasshintify(obj[k])
651 res[k] = t
652 print "props", res, dir(obj)
653 for attribute in dir(obj):
654 if not attribute.startswith("get_"):
655 continue
656 k = attribute[4:]
657 if res.has_key(k):
658 continue
659 getter = getattr(obj, attribute, None)
660 if callable(getter):
661 res[k] = jsonclasshintify(getter())
662 return res
663 return obj
664
665
666 if __name__ == '__main__':
667
668 if len(sys.argv) < 2:
669 sys.exit()
670
671 if sys.argv[1] != u'test':
672 sys.exit()
673
674
685
686 from Gnumed.pycommon import gmI18N
687 gmI18N.activate_locale()
688 gmI18N.install_domain()
689
690 data = {
691 'pk_field': 'bogus_pk',
692 'idx': {'bogus_pk': 0, 'bogus_field': 1, 'bogus_date': 2},
693 'data': [-1, 'bogus_data', datetime.datetime.now()]
694 }
695 obj = cTestObj(row=data)
696
697
698
699 print obj.fields_as_dict()
700
701
702