USER TARGETED MEDICAL IMAGING AND INFORMATION PACKAGING, COMPRESSION AND DISTRIBUTION SYSTEM
A computer-based method and system of distributing biological sample data acquired as a digital image of a subject biological sample. The acquired digital image and image capture data are processed according to at least one user. This results in processed image data and capture metadata. The processed image data represents biological sample data of the subject biological sample. A package processing combines the processed image data and capture metadata into a working Package. The method and system enables simultaneous electronic access to the working Package by multiple users, across multiple sectors, in addition to the one user.
Latest CORISTA LLC Patents:
- Virtual slide stage (VSS) method for viewing whole slide images
- Multi-sample whole slide image processing in digital pathology via multi-resolution registration and machine learning
- Virtual slide stage (VSS) method for viewing whole slide images
- Virtual slide stage (VSS) method for viewing whole slide images
- Multi-sample whole slide image processing in digital pathology via multi-resolution registration and machine learning
This application is a continuation of U.S. application Ser. No. 12/942,911, filed on Nov. 9, 2010, which is a continuation-in-part of U.S. application Ser. No. 12/843,138, filed Jul. 26, 2010, now U.S. Pat. No. 8,244,912, which claims the benefit of U.S. Provisional Application No. 61/228,819, filed on Jul. 27, 2009. The entire teachings of the above application(s) are incorporated herein by reference.
FIELD OF INVENTIONThe present invention is in the field of Health Informatics as applied to Pathology, more specifically covering those aspects of Digital Pathology concerned with the imaging, data processing, and data distribution required by and unique to Digital Pathology.
BACKGROUNDIn the field of Digital Pathology, there is a requirement to have an exchange of studies for the purpose of a primary or secondary pathological diagnosis. Studies typically consist of one or more lower-resolution images, the references to the corresponding higher-resolution images, associated image metadata, study metadata and patient metadata. Further, access is provided to the necessary data to perform the proper review of the study. Physical and security-based constraints to this access represent a primary barrier to wide scale deployment of Digital Pathology systems. The physical constraints include bandwidth and storage issues for local and remote access, and are typical of most networked applications with the additional requirements imposed by the exceedingly high resolution nature of the data. Additionally, given the sensitivity of medical data, security based constraints, such as ensuring patient privacy and data security are the key areas needing additional focus in this field.
Digital Pathology, in itself, is a compelling enough technology to reach widespread adoption in all but the smallest of practices. The reduction in cost, time and management headache of no longer needing to distribute glass slides to in-house physicians, to second opinion or referrals, to search for slides for publications and presentations will quickly prove itself invaluable, as the workflow of slide production, digitization and immediate archival comes to be. Further, being able to transform, reduce or restrict the data being sent to a user can increase the diagnostic ability of a user, can increase the number of users who can benefit from the data and provide more efficient access to the data.
Digital Pathology provides the opportunity to apply additional processing and analysis on behalf of the user as part of the process, allowing new modalities of diagnosis for the field of Pathology.
Digital Pathology can, as Health Informatics has, enable the filtering of sensitive patient data from a study, allowing the studies use beyond the scope of the hospital, as an educational or research dataset. Structured and encoded information allows new forms of access and distribution of data through the compression, progressive and predictive packaging of the data. Examples of the predictive packaging include only sending the data which the user requires, a resource and bandwidth optimized manner.
SUMMARYThe invention packages and processes Digital Pathology and patient data to be included in a pathology study in such a way as to facilitate the exchange of studies between users. The utility of the invention comes primarily from the transformation of the data into packages that can be used to route and access the information in a manner that automates existing pathology study workflow while enabling additional workflows uniquely suited to the electronic form of pathology study workflow, namely Digital Pathology.
The pathology studies typically consist of one or more digital pathology images, associated or derived image metadata, and patient metadata (
The data contained within a study needs to be sufficient for the purposes of a targeted user, that user typically being a pathologist. Depending upon what is requested of the pathologist for the pathology review, this may include one or more images of varying, progressively detailed quality, patient data, imaging metadata or the results of preliminary (often automated) analysis, such as measurements, cell counts or other derivative data. Through user specific processing and packaging, the superset of available data is filtered and progressively provided to the target user based on patient privacy, authorization, access restrictions, or simple physical limitations of the target user's connection to the data. This user specific processing and packaging enables simultaneous access of the case by multiple users, each with potentially different imaging and data requirements, each with potentially different permissions on the data, and allows for the auditing of a particular user's session, either collaboratively in real-time or after the fact as an audit trail, teaching tool or other uses.
In one embodiment, a computer-based method of distributing biological sample data includes the steps of:
acquiring a digital image of a subject biological sample;
processing the acquired digital image and image capture data according to at least one user, resulting in processed image data and capture metadata, the processed image data representing biological sample data of the subject biological sample;
through a package processing, combining the processed image data and capture metadata into a Package; and
enabling simultaneous electronic access to the Package by multiple users, across multiple sectors, in addition to the one user.
Another computer-based method embodying the present invention includes the steps of:
acquiring a digital image of a subject biological sample;
processing the acquired digital image to form a progressive image stream representing biological sample data of the subject biological sample, the processing additionally including preparation of access to the progressive image stream, and access being in terms of resolution, scale and sub-regions of the biological sample; and
transmitting the progressive image stream to one or more users, such that a user views spatially progressive images, in terms of resolution, scale and sub-regions, of the subject biological sample.
According to a computer system of the present invention distributing biological sample data, there is a source of one or more digital (e.g. acquired) images of a subject biological sample. A processor processes a digital image from the source. This processing includes processing the digital image and corresponding image capture data according to at least one user, resulting in (i) processed image data representing biological sample data of the subject biological sample, and (ii) capture metadata. A package process executable by a computer is responsive to the processor processing and is configured to combine the processed image data and capture metadata. This combining forms a package that is electronically accessible simultaneously by multiple users, across multiple sectors, in addition to the one user.
The foregoing will be apparent from the following more particular description of example embodiments of the invention, as illustrated in the accompanying drawings in which like reference characters refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating embodiments of the present invention.
The first element of the package is User Specific Image Data (120), which is the result of User Specific Processing (330) on a Whole Slide Image (122), equivalently referred to as a WSI or Whole Slide Image Data or WSI Data. A WSI is obtained by capturing a digital image of a Biological Specimen (102), which is equivalently referred to as a Biological Sample, from a Patient (101).
The second element of the package is User Specific Patient Metadata (130), which is the result of User Specific Processing (330) of Patient Data (132). Patient Data is retrieved from an Electronic Medical Record (103), equivalently referred to as EMR, Laboratory Information System (104), equivalently referred to as LIS, or Hospital Information System (105), equivalently referred to as HIS, or other systems containing patient data.
The third element of the package is User Specific Image Metadata (140), which may be generated from data obtained at the time of Image Acquisition, data from a library of similar images or from automated or preliminary image analysis.
The fourth and fifth elements of the package are a Pathology Analysis Request (150) to the target user (325), which will result in that user generating and adding a Pathology Analysis (160) back into the package (110). Data contributing to the Pathology Analysis Request (150) include Review Criteria (152) and Routing, Assignment and Schedule Information (153). The components of the Pathology Analysis (160) may include some format of a Pathology Review (162) and Annotations and Notes (163).
Types of User Specific Image Data (210, also equivalent to 120 in
Types of User Specific Patient Metadata (220, also equivalent to 130 in
Types of User Specific Image Metadata (230, also equivalent to 140 in
Types of Notification Messages (240) may include Notifications (241), Notification Acceptance (242) and Completion Notification (243) messages.
In the first step, Acquire Digital Images (310), a Whole Slide Digital Image (WSI) (122 in
Once the package has the necessary elements for the target user, Electronic Access (340) is enabled for the target users. Electronic access is initiated by sending each target user a Notification Message (355, equivalent to 241 in
Within the scope of User Specific Processing (330 in
During User Specific Processing (330 in
Specialized User Specific Processing 330 (
During the User Specific Processing 330 (
As a result of these processes, Automated Image Analysis data (660, equivalent to 211 of
All of the data generated in these steps is added to the package 110 (
During the determination of the user's business requirements (710), the package can undergo a plurality of packaging functions, one of which is packaging for a pathology review (
A second example packaging might be for education or research (730). For education and research, education or research data is added (731), such as the scope of a study, or the focus of a particular analysis, and the patient metadata in the package is filtered to remove identifying patient information (732).
Yet another non-limiting packaging might be for review by a tumor board or second opinion (740). This is a specific instance of a Recursive Request for Diagnosis, in which one of the metadata items of the package is a Pathology Analysis 160 (
During the Determine User (810, equivalent to 320 in
A Target User (910, equivalent to 325 in
During the Fulfill Progressive Request (1010, equivalent to 940 in
A Target User (1110) generates a Request Session Sampling Data (1120) message, which triggers a process to Transform Session Sampling Data (1130). The Transform Session Sampling Data process (1130) takes the Session Sampling Data (1140, equivalent to 1040 of
Further, the Temporal Progressive Image Data (1154) may be combined with the original Progressive Image Data (1160, equivalent to 970
A Target User (1205, equivalent to 1020 in
Through the act of generating an additional progressive image data request by Target User 2, a second Fulfill Progressive Request (1250) step is entered, which in turn Generates Session Sampling Data (1260). Target User 1, who is part of the Mirrored Progressive Image Session implicitly initiates a second Request Mirrored Image Stream (1270) step, Transforming Session Sampling Data (1271) and Transmitting Progressive Image Data (1272) in turn, generating a new respective Progressive Image Stream (1280) to Target User 1, effectively mirroring the actions of Target User 2 to both users.
User Specific Image Data (1350, equivalent to data 930 in
A description of example embodiments of the invention follows.
Elements of the Current Invention
Embodiments of the invention include several combinations of the following: five data structures, two types of packages, and four types of notification and response messages.
Whole Slide Image
The first data structure is a Whole Slide Image (also known as Whole Slide Image Data, WSI or WSI Data) 122 (
Patient Data
The next data structure is Patient Data (also known as Patient Metadata) 132 (
Image Metadata
The next data structure is Image Metadata, which may include Image Capture Data 142 (
User Specific Image Data
The next data structure is User Specific Image Data 120 (
User Specific Metadata
The next data structure is User Specific Metadata, which takes the form of both User Specific Patient Data 130 (
Package
The next data structure is a Package (also referred to as a Detailed Study or Study) 110 (see
Preview Study
The next data structure is the Preview Study, which contains one or more samples of the study images which provide a survey of the study images, combined with both image metadata 140, 230 and patient metadata 130, 220. The preview study itself forms its own package which has undergone a separate Packaging Process (
Pathology Analysis Request
The next data structure is the Pathology Analysis Request 150 (
Pathology Analysis
The final data structure is the Pathology Analysis (also referred to as a Review, Pathology Review, Pathology Diagnosis or Diagnosis) 160, 550 (
Notification, Notification Acceptance and Completion Notification
The message data structures 240 of the invention includes Notification 241, Notification Acceptance 242 and Completion Notification 243.
As illustrated in
When the Target User 325 accepts or rejects the package 110 for review, a Notification Acceptance Message 242 (
When the Target User 325 completes a Pathology Analysis 160, 550 (
The Packaging Process
The Packaging Process (
Once the user(s) is identified, User Specific Processing 330 may occur (
The package 110 may be targeted to a user who has specific business needs or is a member of a specific sector which may dictate the inclusion or exclusion of some image or metadata, and which may dictate whether a Pathology Analysis Request 150, 520 will be added to the package 110. One embodiment of this Sector Specific Processing (also known as Business Requirement Enhanced User Specific Processing—
A preferred embodiment of this Sector Specific Processing (
User Specific and Specialized Image Processing
User Specific Image Processing (
Progressive Image Streaming and Session Sampling
Once the data is prepared, the Image may be progressively streamed (
As the Progressive Image Stream 990 is assembled and sent to the Target User 325, 910, Session Sampling Data is recorded at the Originating Node (
Session Sampling Data Transformation, Auditing and Collaboration
The Session Sampling Data 1040 captured during Progressive Image Streaming (
In one embodiment of a non-temporal session, the detailed account of what is accessed from the Session Sampling Data 1040, 1140 can be reduced into a pixel and resolution representation of what was accessed, in conjunction with the start and end times of the session access.
In a preferred non-Temporal embodiment, there is a non-Temporal Image 1152 (
A preferred temporal embodiment is a real-time based list of progressive image data 1154 (
Another preferred temporal embodiment is similar to above, but allows for a mirrored session between multiple Target Users (
Utilizing the Session Sampling Data 1040, 1140 in another way, a preferred embodiment enables a Target User 910, 1330 to cache a copy of the viewed Progressive Image Stream 990, 1180, 1240, 1280 on his local or associated Consuming Node 1320 system (
While this invention has been particularly shown and described with references to example embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the invention encompassed by the appended claims.
Claims
1. A computer-based method of distributing biological sample data, comprising:
- acquiring a digital image of a subject biological sample;
- processing the acquired digital image and image capture data according to at least one user, resulting in processed image data and capture metadata, the processed image data representing biological sample data of the subject biological sample;
- through a package processing, combining the processed image data and capture metadata into a Package; and
- enabling simultaneous electronic access to the Package by multiple users, across multiple sectors, in addition to the one user.
2. A method as claimed in claim 1 wherein electronic access to the processed image data comprises notifying the user of availability of the package, and providing access to the processed image data, followed by a completion notification.
3. A method as claimed in claim 1 wherein the step of processing includes patient data processing and user processing, each comprising:
- patient data processing including retrieval, inclusion of data from external data stores, resulting in the user requested patient data;
- user processing including retrieval and inclusion of image metadata and secondary data from external data stores, as well as further processing secondary data into the user requested metadata;
- package processing additionally including the user requested patient data and user requested metadata in the Package.
4. A method as claimed in claim 1 wherein the one user is a pathologist, and the step of enabling electronic access includes enabling the user to perform a diagnostic processing step, wherein the user records data regarding the biological sample data, the recorded data comprising a written or audio diagnosis, subsampled image data, annotations and comments.
5. A method as claimed in claim 4 wherein the diagnostic processing step is performed partially or wholly by an automated system or user other than a pathologist.
6. A method as claimed in claim 1 wherein the step of processing includes a modeling process including any one or combination of transmission, storage and classification modeling methods,
- wherein transmission modeling includes combinations of channel coding and version controlled conditional transmission,
- storage modeling includes at least source coding, and
- classification modeling includes combinations of predictive encoding, compressed sensing, and progressive image streaming.
7. A method as claimed in claim 3 wherein the step of processing is aggregate processing, comprising the aggregation of user specifications for two or more users, and, wherein the step of enabling electronic access resolves the package data for each user according to his respective user-specification, as if each individual User Request were independently processed.
8. A method as claimed in claim 3 wherein one or more of the user processing, patient data processing or package processing steps initiates a complete or partial diagnostic processing step as an input to its own processing step.
9. A computer-based method of distributing biological sample data, comprising:
- acquiring a digital image of a subject biological sample;
- processing the acquired digital image to form a progressive image stream representing biological sample data of the subject biological sample; and
- processing additionally including preparation of access to the progressive image stream, said access being in terms of resolution, scale and sub-regions of the biological sample; and
- transmitting the progressive image stream to one or more users, such that a user views spatially progressive images, in terms of resolution, scale and sub-regions, of the subject biological sample.
10. A computer method as claimed in claim 9 wherein the step of acquiring acquires a series of images, and the step of processing employs compressed sensing.
11. A method as claimed in claim 10 further comprising recording a list of processed image data requested by a user, said list being recorded as session sampling data, wherein the session sampling data is able to be recalled in a manner enabling auditing of portions of an image viewed by the user.
12. A method as claimed in claim 11 wherein the session sampling data is able to be recalled and accessed as unordered data providing an overview of image portions that the user accessed.
13. A method as claimed in claim 11 wherein the session sampling data is recalled and accessed in an order in which the user accessed and reviewed the processed image data, allowing reconstruction of a session.
14. A method as claimed in claim 11 wherein a plurality of session sampling data for a same image may be sampled to gather information about common points across multiple sessions.
15. A method as claimed in claim 11 wherein the session sampling data is recalled by one or more other users as it is being generated, resulting in all users having a synchronized viewing experience.
16. A method as claimed in claim 15 further comprising enabling a user involved in the synchronized viewing experience to (a) navigate the processed image data, resulting in additional progressive image stream requests, and (b) annotate the image viewed, said annotating being in a manner viewable by the other users.
17. A method as claimed in claim 11 wherein the user accesses the requested processed image data remotely and the session sampling data is augmented by a locally cached copy of the requested processed image data.
18. A method as claimed in claim 17 further comprising enabling the cached copy of the processed image data to be further processed to include the entirety of the image data.
19. A computer system distributing biological sample data comprising:
- a source of one or more digital images of a subject biological sample;
- one or more processors configured to process a digital image from the source, said processing including processing the digital image and corresponding image capture data according to at least one user, resulting in (i) processed image data representing biological sample data of the subject biological sample, and (ii) capture metadata;
- a package process executable by a computer in response to the processor processing and configured to combine the processed image data and capture metadata, said combining forming a package that is electronically accessible simultaneously by multiple users, across multiple sectors, in addition to the one user.
Type: Application
Filed: Jul 18, 2017
Publication Date: May 10, 2018
Applicant: CORISTA LLC (CONCORD, MA)
Inventors: Eric W. Wirch (Concord, MA), Charles P. Pace (Manchester Center, VT)
Application Number: 15/653,337