Google Summer of Code 2023: AI Model Service Auto Selection by Modality - Final Report

Introduction

GSoC is one amazing program that lets students and professionals get into the world of open source, contribute code and also learn back from the program. I definitely learnt a lot by contributing to my mentoring organization LibreHealth through GSoC'23. I now understand the value of open source and will become a lifelong contributor wherever and whenever I can.

LibreHealth is an umbrella organization where multiple health information technology projects collaborate. I was actively involved in contributing code to their LibreHealth Radiology application through the project "AI Model Service Auto Selection by Modality".

My contributions to the application were more involved into implementing AI models in the application and also building modules that do automatic model selection, example, when a Radiologist opens up a Mammography image the application automatically triggers the Mammography AI model to produce inferences on the image.




Objectives

The project description and it's objectives are posted in one of the LibreHealth Forum posts. The high level objectives of the project were to,
  • Develop a module in the LibreHealth Radiology application that will perform automated model selection based on the DICOM metadata information of the Radiograph. To give more context, all Radiographs (like X-ray images) have metadata associated with them. The metadata has information on things like what part of the body the image is from and other patient details.
  • Integrate state of the art AI models to the application that can infer and diagnose medical Radiographs of Mammograms, Chest X-rays, etc
  • Develop an AI Tool in the image viewer's toolbar that has buttons corresponding to each AI model that is available and on click they would call the respective AI model and produce inferences on the image

Achievements

Automated Model Selection Implementation

  • Modified the administration settings page to allow User to add AI model specific configurations
  • Modified the java modules to collect metadata information of the Radiographs and pass them to the AI Model Service
  • Added methods and configurations to the AI Model Service Python backend to perform model selection by dissecting the metadata
Link to Merge Requests for the Automated Model Selection Implementation on the LibreHealth Toolkit app

AI Model Service Implementations

  • Revamped the existing CheXnet inaccurate model to perform considerably well
  • Updated the AI Model Service codebase and reorganization of files to enable a systematic way of adding more AI models for different modalities like Mammograms
  • Added an AI Tool in the OHIF viewer toolbar that has buttons corresponding to each AI model. Any specific AI model can be called from the OHIF viewer using the AI tool
Link to Merge Requests made on the AI Model Service end on the LibreHealth Radiology Application

Documentation

Added Sphinx developer guide documentation for the new configurations required and also on starting the application. Find the merge request here.

What I Learned

  • I gained technical skills on Java, Maven, React. I improved my Python skills and especially on Flask and PyTorch.
  • I learnt a lot in the process of implementing AI models. I was introduced to new computer vision architectures like DenseNet, YOLOX, ConvNext.
  • I learnt software product application architectures and also their development.

Challenges and How I Overcame Them

I overcame few challenges through the course of this project,
  • At the start of the project I would describe my area of technical skills were more in AI, Python. Through this project I also had to work in Java. I struggled with it initially since I had to pick it up quickly and implement the modules and I was able to do that successfully.
  • Some of the AI models were too complex, like the Mammography model. I had to spend lot more hours on them to dissect the code and finally implement it.
  • I also had to work on front-end code (OHIF viewer) to implement functionalities related to triggering AI Models. Since I consider myself to have less experience in front-end development this was tough for me. I was able to implement the component successfully.
  • There were issues I faced with Docker file dependencies in the project that took me some time to figure out.
I was able to overcome these challenges by mainly discussing them with my Mentor and also by tracking previous commits and merge requests made by developers who have worked on this application previously. It helped me identify the areas that need to be modified or code added.

Future Work and Recommendations

Future work may also include adding additional AI models for more modalities. There is currently a Kaggle competition that is on Abdominal Trauma Detection. This can be implemented to the application once the competition is over. More models similarly can also be implemented.

Other works can be related to adding training pipelines to the application for each model. It is tedious but would be a good feature addition.

Acknowledgments

I would like to thank GSoC for this wonderful opportunity. The LibreHealth Community, Admins have been very supportive by checking on us from time to time, posting all relevant information on time and having our backs.

Shoutout to my Mentor Dr. Saptarshi Purkayastha for helping me with my every challenge and struggle. It would have not been possible without my mentor.

Conclusion

Over the course of my GSoC journey with LibreHealth, I've been deeply immersed in the confluence of open source, AI, and healthcare. Through challenges and triumphs, I've grown immensely, both technically and in my appreciation for collaborative development. With invaluable guidance from the LibreHealth community and my mentor, Dr. Saptarshi Purkayastha, I've contributed to bridging the gap between radiology and advanced AI techniques. As GSoC 2023 concludes, I remain deeply committed to the open-source ethos and look forward to furthering this mission.

Contact Information

Email: rohan.isaac27@gmail.com

GitLab        GitHub         LinkedIn

Comments

Post a Comment

Popular posts from this blog

My First GSoC - LibreHealth Community Bonding Period

GSoC Acceptance I was amazed to know that I got accepted into Google Summer of Code 2023. This is my first time applying and getting into GSoC. I remember the time when I sat at the library for hours, working on the preliminary tasks, figuring out solutions, and writing an extensive proposal. Reiterating through the solution and proposal back and forth, I watched lots of YouTube videos and read blogs providing tips to draft a good proposal. All the efforts paid off when I received the email of acceptance. The Org & Project I drafted a proposal to the LibreHealth Organization. LibreHealth is collaborative community for free & open source software projects in Health IT, and is a member project of Software Freedom Conservancy. It is amazing to just think about how contributors from all over the world come together and build things that impact the world positively. This idea is what brought me into open source. Before jumping into the project details, here are a few things about m
Read more

GSoC Week 2

Image
By the end of last week, I was able to meet my mentor, which helped me gain direction for the project. We discussed details and challenges regarding the implementation of the AI model selection based on image modality. Although Java is a new language to me, I wanted to learn and work my way through it for the project. This week, my task was to understand parts of the LibreHealth toolkit code and prepare a list of tasks at an atomic level. After that, I will need to update them to the GitLab issues list. Coming up with the task list To come up with the task list, during the meeting with my mentor, I received some guidance on where to start with code exploration and understanding. Initially, it was challenging for me to understand the code, so I started looking at Java tutorials on YouTube. I gained some understanding of how Java works. However, I knew this was not enough for me to completely understand the application, and it would take time, so I also started looking for patterns in th
Read more