The Wired Room - Automated Digital Sound, Image and Motion Analysis for Enhanced Medical Diagnostics from A Patient Encounter

Project Background/Scope: The role of the health care encounter – whether it be at the medical office, clinic, hospital, home or field is critical in obtaining relevant information to guide and direct the delivery and accuracy of care. Studies have demonstrated that more than 70% of diagnoses and advancement of care emanates from the physician or health worker carefully questioning and observing the patient. Sadly, patient encounters today have become shorter as to time spent, with the physician often hampered in examination focus by requirements of electronic health record (HER) data entry and use of a computer. Studies have also shown that many proper diagnoses are made by the doctor using information, such as: what and in what way the patient speaks, how the patient looks and acts, how the patient behaves, how the patient sits, how the patient walks, how the patient smells, and other information gained by focused attentive one–on–one patient encounters and consultations. In routine doctor-patient interactions today much of this information is not recorded and is lost.

This project will develop a “Smart Room” for patient examination able to capture much of this lost information. A kit will be designed that is portable, able to be placed in a given space utilized for patient examination. The kit will consist of: 1. Cameras (2-4) for image and motion capture; 2. microphones (2-4) for sound capture; 3. a handheld microphone for close patient examination (e.g. convertible or integrated with a stethoscope); 4. a handheld camera for close examination; and 5. a handheld microscope for extreme close-up image capture. Central to the kit will be an interface able to input all of these sources, and a computer system with large graphic display able to portray all inputs. The sophistication of the system will be in developed algorithms - i.e. in the systems’ ability to measure specific parameters from the input captured signals and display the processed time-synched information. Parameters to be measured include: accurate dimensions of a body part, accurate movement analysis (the system will incorporate MOCA - a system recently developed allowing image and motion quantitation (see Applied Sciences, 12(12), 6173. https://doi.org/10.3390/app12126173; sounds – including speech, breathing, heart sounds, AV graft/fistula sounds – analyzing them as to frequency, amplitude, pitch, phase and other subtle metadata including HNR (harmonic to noise ratio), NHR (noise to harmonic ratio), energy entropy (EE), short time energy (STE), zero crossing rate (ZCR), spectral roll off (SR), spectral centroid (SC) and spectral flux (SF) to estimate stress related acoustic high level features such as jitter and shimmer. Patient vital signs and weight will also be measured and inputed. Processed data needs to be able to be displayed as well as be compressed and stored in a fashion that may be incorporated in the electronic health record for ready serial access and display of evolving trends

The long-term goal and hope here it that from additional patient quantitative data gathered by the wired room, that new “digital biomarkers” may emerge stemming from serial evaluation of parameters measured over time. These new biomarkers will ultimately enhance the precision of clinical care.

Requirements: (1) Develop a smart room kit able to capture patient image, motion and sound data. (2) Develop algorithms to analyze all inputs and quantitate parameters based on input data. (3) Develop a record keeping and display system for serial trend analysis. (4) Integrate raw and processed data into means of storage and recall from an electronic health record.