1. Situation – Problem
The government agency’s service provider faced significant challenges in processing injury-related claims due to the reliance on manual efforts from nurses and doctors. These professionals were tasked with assessing the validity of claims based on complex medical histories, work-related injuries, and ICD-10 codes (international codes for classifications of diseases). The process involved junior staff preparing initial reports, followed by senior doctors verifying the findings, often requiring extensive review of medical records. This labour-intensive process took 1.5-2 hours per claim for junior staff and an additional 30 minutes for senior doctors, leading to a substantial backlog. The scarcity of qualified medical professionals and the need to control costs further exacerbated the situation, making it difficult to meet the demand for accurate and timely claim assessments.
2. Task
Data Stream Labs (DSLabs) was engaged to streamline and optimise the claim processing workflow. The objective was to develop an AI-driven system that could reduce processing times, improve the accuracy and consistency of assessments, and alleviate the workload on medical staff.
3. Action – Solution – Hurdles – Paths Taken
Solution Approach:
DSLabs conducted a comprehensive analysis of the existing value process, identifying critical areas of concern, including time consumption, limited staff availability, inconsistencies in report writing, the quality of reporting, and the training requirements. They proposed a new value process leveraging Generative AI (GenAI) to enhance machine and human interactions.
Proof of Concept (POC) Stages:
1. Definition of Objectives and Metrics: Set clear goals, such as increasing the number of processed claims by 5x.
2. Data Preparation and Privacy Compliance: Collect and anonymise medical claims data, ensuring compliance with healthcare regulations like Australia’s Privacy Act 1988.
3. Architecture and Coding: Design the solution architecture, selecting tools like OpenAI API, LangChain, MongoDB, and infrastructure options (e.g., Azure, AWS).
4. Integration and Testing: Integrate the trained model into the existing workflow and conduct thorough testing to evaluate performance.
5. Evaluation and Feedback Loop: Assess the model’s effectiveness, gather feedback and iterate improvements.
Obstacles and Solutions during POC Execution:
– Data Quality and Variability: The variability in medical data formats presented (text, images, graphs) posed a challenge. Solution: Standardisation of data formats, robust data cleaning, using OCR and annotation tools to convert non-text data into text and employing human medical encoders.
– Data Privacy and Security: Handling sensitive medical data required strict compliance with privacy laws and robust security measures. Solution: Anonymisation and de-identification of data, secure data handling practices, and adherence to healthcare privacy frameworks.
– Bias and Fairness in AI Models: The risk of biased outcomes was a concern. Solution: Implementing bias detection and mitigation strategies, using diverse datasets, and continuous monitoring of the model’s performance.
4. Tech Stack Used
The tech stack for the POC included OpenAI API for language processing, LangChain for task automation, MongoDB for data management, and cloud infrastructure options like Azure and AWS. Additional tools were considered for on-premise execution and cost optimisation.
5. Result
DSLabs successfully developed an MVP featuring a user-friendly interface for uploading medical records, selecting or uploading claimant job descriptions, and incorporating human-generated claims for reinforcement learning. The system included several report templates in accordance with government agency standards. The OpenAI GPT-4 model processed claims and generated reports with ICD-10 codes and citations in approximately 10 seconds, significantly reducing the time and effort required for manual processing.
6. Challenges and Areas of Improvement
Despite the successful MVP, challenges remained:
– Data Variability: The presence of different formats, including handwritten notes, required further refinement in data preparation techniques.
– Data Privacy and Security: Ensuring compliance with privacy laws and protecting against malicious prompts remained crucial for production deployment.
– Bias and Data Imbalance: Ongoing efforts were necessary to identify and mitigate biases in the training data.
– Integration with Existing Systems: Integrating the AI solution with legacy systems is challenging. Solution for Production deployment: Development of APIs, adherence to interoperability standards, and incremental integration strategies.
– User Adoption and Training: Although resistance to new technology is not anticipated among healthcare professionals at this particular service provider, we anticipate such a challenge working with larger health service providers. Solution: User-centric design, comprehensive training programs, and effective change management strategies.
Data Stream Labs continues to refine the system, focusing on enhancing data security, privacy, and bias mitigation, to deliver even more accurate and reliable claim assessments.