Lab Importers

Lab importers are complex scripts that can need to take into account any variety of use cases for lab test ordering as well as for results processing. In view of the large international and even regional differences in how lab tests are requested, and how lab results are delivered, an outline of the requesting of tests and results return is offered, followed by importer specifications (both universal and specific).

Overview of lab requesting and results return

The ordering (requesting) of lab tests need not be tracked in GNUmed in order for GNUmed to be able to import lab data. Tracking will, however, offer the added benefits of pre-attributing the most relevant episode to which the results will relate, and will make it easier to determine the status of any results.

Lab requesting generally involves:

• a doctor to authorize one or more tests
• a method for the doctor to submit that authorization to a lab
  • in Canada, by paper request ("requisition") deliverable to the lab by fax, or hand-delivered by the patient, who must regardless attend a lab in order for their blood, urine and any other "specimens" to be collected, or
  • in Germany, by the attachment of pre-coded labels onto the specimen bottles or tubes, sometimes called probes, with or without associated paperwork, when the doctor or assistant had already collected the blood, urine, or other specimens at a point of care
  • in ?? by electronic submission of the request
• a way for the authorizing doctor to receive the results, and for a copy of these results to be delivered to any other doctors or facilities (or even the patient) when specified by the ordering doctor or — in some cases — after a doctor (who had not been copied) arranges to be sent a copy. Encoding and delivery methods for electronic results include:
  • British Columbia, Canada:
    • e.g. Excelleris XML-wrapped HL7 files (either through https scripting, or browser-based login, and saving, of an XML file)
  • Germany:
    • encoding in the Labordatenträger (LDT) specification among the *DT protocols endorsed by the German physicians statutory association and briefly outlined here
  • Australia:
    • unencoded PIT files supplying individual patient results as text blobs (thread here) and delivered by encrypted email

Operationally:

• in order for a request on a patient to be recorded in GNUmed (in clin.lab_requests), the request must be associated with an encounter as well as with an episode, and must be assigned a request_id and a lab test organization that is expected to perform the test (even though the true identity of the lab may not be known in some locales like British Columbia, Canada, because the patient can have the ability to take their requisition to a lab of their choice)
• if a request does get recorded in GNUmed, it gets created with the default status of is_pending, and such pending records will need to be reconciled and cleared off, so that they do not exhibit limitless accumulation. It therefore becomes necessary for returning lab data to be able to associate with an existing clin.lab_request record in one of two ways:
  • explicitly, by the results carrying with them information (such as a unique request_id or possibly lab_request_id after partial results were received) to permit a unique match to the GNUmed request record or else
  • by matching of the results to a single GNUmed pending request, in which the ordering doctor and the patient matched what was in the returning data
• otherwise, a request can be allowed to be created from the returning data when no request had yet been recorded in GNUmed... this can happen when the praxis is not yet using GNUmed to capture this phase of patient care, or when copies of externally-ordered results are received

Importer specifications

Importers need to:

• have some way to be signalled, perhaps daisy-chained to the "retriever" process, or through a cron tab, to monitor or react to newly-received results
• evaluate each line (or message) in the source file, and ensure that any and all invalid lines or messages are identified, and that any valid lines or messages that cannot be matched to a patient are also identified, in order that no patients are put at risk from a lack of any information getting handled
• undertake matching of lines (messages) to unique patients that already exist in GNUmed as follows
  • where defined certainty permits the data to be attached to an existing patient:
    • accept results-related data into clin.test_result as well as into associated tables clin.test_type, clin.test_org and clin.lab_requests as well as auto-creating new reference values for test_orgs and test_types where necessary
    • append to clin.encounter a record with clin.encounter_type "description" = "automated data import"
    • if the request was being tracked in GNUmed's clin.lab_requests, update the request information, including is_pending status, and identify the attributable episode; otherwise, spawn a lab request from the results, and assign an unattributed episode "lab data"
  • where there is uncertainty about who the patient may be, or if the patient did not yet exist, move these lines' (messages') data into clin.incoming_data_unmatched

Specific importer specifications, by source

British Columbia, Canada (source: Excelleris)

These XML-wrapped Hl7 messages contain MSH, PID, ORC and OBR segments, followed by none, one, or more OBX segments, depending on whether the "test" that was requested was a single test, or whether in the form of a "panel" of tests, such as "Electrolytes". An OBX segment will appear for each specific observation (or "panel" component) as it becomes available and added to the "built" message.

Until the last component of a request becomes finalized, each download of this request's results will include all previously-built message results for this request. How much to handle this in GNUmed is yet to be figured out. It has been proposed in a first iteration to endure all of the duplicates that would accrue however that warrants careful consideration.

Note that fuller (including original) documentation of this source must be obtained directly from Excelleris. This wiki aims no infringement, and limits itself to defining the field mappings and processing that are of interest to GNUmed.

Auto-match to a specific existing patient

Auto-match criteria:

• primary identity key (applicable after manual reconciliation in GNUmed appends a patient primary key to an unmatched message, at which point the importer can effect a match)
• Personal Health Number AND {dob or lastnames}
• dob AND case-insensitive lastnames AND 2-character wordmatch in firstnames AND gender [unless source data is missing or unknown]
• future: request_id AND is_pending AND {date of birth OR lastnames OR firstnames OR [null dob and person identifiers]}

Table location of the auto-match targets in the GNUmed schema:

Source fields from lab broker HL7 messages:

In the case of a failed auto-match (no matches, or multiple matches) go to Handle_unmatched_patient_data.

In the case of successful auto-match, import data as follows:

check whether this lab request already exists in GNUmed

• ORC 003 (Filler Order Number / Order Number ID of lab performing tests / Accession number-test code-tiebreaker)
  • check whether the combination fk_test_org + ORC 003 (as lab_request_id) has already been imported into clin.lab_request
  • if yes, update fields lab_rxd_when from OBR 014 (Specimen received Date timestamp), results_reported_when from OBR 022 (Report Status Change timestamp), new_field_result_copies_to from OBR 028 (Result Copies To), request_status from OBR 025 (Result Status), is_pending with if( request_status is in [F,C], false, otherwise true)
  • if an additional field note_test_org becomes available, refresh any contents from each of the OBR 024 "Diagnostic Service Section" names, delimited with space or ^ (tbd)
  • else, auto-create this record as a request that was not being tracked inside GNUmed
  • future : do not auto-create if lab_requests already has something pending for this patient under this requestor (evaluate fk_requestor data against ORC 12 "Ordering Provider" data). In the event of a match, this pending request could be updated in place of auto-creating a new record.

Importing test results

Creating test types

Creating test encounters

This is only necessary if there is no corresponding lab request for a given test result.

• append to clin.encounter a record with clin.encounter_type.description = "automated data import"

Creating test episodes

This is only necessary if there is no corresponding lab request for a given test result.

• check for unattributed episode "lab data"
• if necessary create it in clin.episode
  • fk_health_issue = NULL (unattributed)
  • description = "lab data"
  • is_open = FALSE

Creating laboratories (test orgs)

• check whether the test-providing facility (lab org) already exists in GNUmed
  • MSH 004 (Sending Facility e.g. BCB MDS VCH)
• if it does not yet exist in clin.test_org, auto-create it
• establish its key value in clin.test_org.pk to supply fk_test_org in lab_requests (needed to manage ORC 003)

Notify clinicians' inboxes

Whenever new data arrives in the test results table, the new data's lack of a review will trigger a notification in the inbox of the provider that is intended to review a particular new result.

Example file mapping

MSH|^~\&|PATHL7|BCB|HTTPCLIENT|vendor1|20071101120533||ORU^R01|MDC20071101120533673|P|2.3|||ER|AL

PID||9012345678|||EXCELLERIS^BPATIENT||19430102|F|||||(604)658-2107

ORC|RE||07-9999999-PT-0|||||||||90909^MDCARE^BOB

OBR|1||07-9999999-PT-0|PT^INR||20071009092600|20071009092600|||||||20071009092600||90909^MDCARE^BOB||079999999||07|079999999|20071010002500||HAEM3|F|||90909^MDCARE^BOB

OBX|1|NM|6301-6^INR||2.5||2.0 - 3.0||||F|||20071009134500

ORC|RE||07-9999999-CLOZ-0|||||||||90909^MDCARE^BOB

OBR|2||07-9999999-CLOZ-0|CLOZ^Clozapine||20071009092600|20071009092600|||||||20071009092600||90909^MDCARE^BOB||079999999||07|079999999|20071010002500||REFER1|F|||90909^MDCARE^BOB

OBX|1|FT|X500^Referred Test||Sent to Provincial Toxicology Centre.\.br\Telephone: 604-707-2710||||||F|||20071009092700

• clin.test_result.pk
• clin.lab_request.pk

Handle unmatched patient data

- messages are to be written into =clin.incoming_data_unmatched=
- table specification is here
   http://salaam.homeunix.com/~ncq/gnumed/schema/devel/gnumed-schema.html#clin.table.incoming-data-unmatched
- data (presumably the XML-stripped HL7 form) is to be written into
   clin.incoming_data_unmatched.data (bytea)
- importer information e.g. hook for source-specific HAPI routine or Mirth channel goes into
   .external_data_id
- from the data, extractable values for assisting manual (& later soft / fuzzy) matching will go into:
   request_id
   firstnames
   lastnames
   dob
   postcode
   other_info
- in the case of main BC data broker HL7 segments and fields, the sources would be
   PID 002 (External patient ID / BC Personal Health Number [PHN] if known)
   -> other_info
   PID 003 ( Addn Patient ID / Additional Patient Identifier – MRN nonunique)
   -> ? concatenate into other_info using internal delimiter ^ ?
   PID 004 (Alternate External Patient ID / e.g. Chart number)
   -> ? concatenate into other_info using internal delimiter ^ ?
   PID 005 (Patient Name / Last^First^Middle (as supplied))
   -> substring1 into lastnames
   -> concatenate substrings2,3 into firstnames (space-delimited)
   PID 007 (Date of Birth YYYYMMDD (null if unknown))
   -> dob
   PID 013 (Home Phone / May be partial and irregularly formatted)
   -> ? concatenate into other_info using internal delimiter ^ ?
   ORC 004 (Placer Group Number 20 External Order ID (eg:  Requisition number) 
   -> request_id
   (need to confirm that 2 more fields have been added in the schema for incoming_data_unmatched and _unmatchable:)
   PID 008 (Sex / Gender of patient F/M/U-unknown)
   -> gender
   ORC 012 (Ordering Provider / BCMSP billing number etc)
   -> requestor

GNUmed should in future

• provide a wizard to offer suggested matches. User confirmation of a match would then cause the patient id to be written into the incoming-data-unmatched field .fk_identity_disambiguated
• determine how to deal with the eventuality of Redundant (duplicate) messages.

Signal Admin of any import problems