GSoC LibreHealth 2023

Introduction GSoC is one amazing program that lets students and professionals get into the world of open source, contribute code and also learn back from the program. I definitely learnt a lot by contributing to my mentoring organization LibreHealth through GSoC'23. I now understand the value of open source and will become a lifelong contributor wherever and whenever I can. LibreHealth is an umbrella organization where multiple health information technology projects collaborate. I was actively involved in contributing code to their LibreHealth Radiology application  through the project "AI Model Service Auto Selection by Modality". My contributions to the application were more involved into implementing AI models in the application and also building modules that do automatic model selection, example, when a Radiologist opens up a Mammography image the application automatically triggers the Mammography AI model to produce inferences on the image. Objectives The project des...

 This week I worked on two things,  - Implemented the AI Tool OHIF component - Minor bug fixes and logging Implementing the AI Tool OHIF component To give some context, in the OHIF viewer I had to add a button in the toolbar (refer "AI Tool" in the image above) that allows calling any of the existing AI models in the AI Model Service. Prior to this a component was implemented that takes a look at the metadata of the image and calls the respective model that was configured in the admin settings, but only one model could be called for a particular image. In a scenario where we would need to call two models for an image, say we have a model for identifying breast cancer region and another model for identifying type of cancer it would become impossible. We implement the AI Tool so that we can call a secondary or tertiary model. This task was quiet difficult for me since I was new working in JS and also there were new components that I had to work with. I met with my mentor twice ...

This week with the help of my mentor I was able to figure out the issue where the UI buttons were not reflecting. I progressed on that part as well on implementing Java methods that on click of the OHIF AI react hook buttons the right AI model is called from the AI Model Service. re-Building the viewer component So I had to build the source again after making the changes to the UI component. This is something completely new to me so I will explain the steps, First I had to uncomment the line (as shown in the image) in the docker compose file that rebuilds the OHIF viewer extension changes, Before rebuilding the module, we need to pull the ohif viewer submodules to our codebase, to do that we need to run the following commands marked in the image. This is present in the README.md file in the 'nsf-ftw' branch. Now to start rebuilding the module we can use IntelliJ IDE to open the Maven build project tab. Here select the 'build-viewer' option in the 'Profiles'. You...

 This week can be characterized as "Struggle and Progress" I worked on two major things this week, - Integrated the ever so complex kaggle RSNA mammography top performing model - Codebase deep dive and coding for React OHIF hook implementation Kaggle RSNA Mammography Model Integration After going through the codebase multiple times and fixing the inference code on my local everything was working fine except for the part where the image is cropped using a YOLO model or an Otsu thresholding method before being passed to the final classification ConvNext Model. The issues I faced were, The models were tuned for specific GPUs to run much faster but for the LH application we would need to run on CPU or GPU based on their availability. So I needed to use the unoptimized models and figure out a few other complexities surrounding them such as modifying a chunk of codebase, identifying the right model to initialize, etc There were a lot of dependency issues for the YOLO model which I ...

There was a lot of struggle this week and I am also expecting a lot of difficulties to come by the further weeks. The two biggest challenges I am facing, - Integration of the RSNA Kaggle Mammography classification model - React hook implementation in the OHIF viewer RSNA Kaggle Mammography Model The code base and model complexity that are put available by the top Kaggle solutions of the RSNA Kaggle competitions are large and takes time to decode. I've been working on the top solution that uses a YOLOX model to crop the image and then a CONVNEXT model to perform the classification. The code that's provided is for training and evaluation. It needs to be converted to perform a single image inference. The code base is optimized for specific GPUs but for our case it needs to be generalized to run on CPU or GPU (any). I was able to run the models but the YOLOX model is throwing smaller boxes to crop off. I need to debug and check what is missing. If it's still the case I will nee...

 This week I worked on one big task which is, - Integrating mammography cancer classification model The Challenge Initially I planned and still plan on integrating the RSNA mammography cancer classification top ranked model from kaggle. As I dug deep down into the code base of the top solution I realized it is going to take time to integrate it. It uses a YOLO model to detect the mammogram area to crop out and then uses a convnext model to perform classification. I already used a day to figure out integration aspects and how I can productionize it in the LH radiology application. Thinking about this complicated model and also integration strategies it was quiet overwhelming. So I wanted to somehow break down my tasks, strategize and integrate it. Plan of Action Meanwhile I remembered of the VGG16 model that I created to do mammography cancer classification at the start of GSoC. I decided to first integrate the VGG16 model since that would allow me to setup the codebase in the appli...

 This week I worked on three things - Refactoring the model selection code base to move from endpoint redirection to calling the actual methods by importing them - Added test cases for model selection and fix issue with docker image - Progressing on integrating the mammography breast cancer classification model Refactoring the model selection code base The model selection code base was implemented initially where it looks at the imaging information and redirects to the appropriate model endpoint. Something like this, The problem faced with this way of implementing it, the base url was hardcoded and pointed to localhost. Mind that only if systems had set their localhost in their hosts file this would work and hence it was not system agnostic. This was causing my model selection test cases run in my local but not in the gitlab ci/cd pipeline. The other thing I read was in RESTful services this sort of redirection is generally not a good design since in RESTful each endpoint is consid...

From LH-RIS Viewer to AI Model Service Last week, I primarily focused on coding and making updates to the LH-RIS (Libre Health Radiology Information System) viewer. This week, my work shifted towards the LH AI Model Service. The current architecture of the Flask REST-API within the AI Model Service was unfamiliar to me. It was built in an object-oriented style, which seemed complex compared to the simpler implementations I've encountered in the past. As such, I took my time to understand the system's components before implementation. Implementing the Model Selection API Once I grasped the system, I dived into implementing the Model Selection API. We introduced a few methods on the viewer side that now transmit Dicom tags information and the new model configuration settings, input by the admin in the administration settings. This information is crucial for the Model Selection API, as it determines which model to invoke on the fly. The concept behind the Model Selection API is st...

By the end of last week, I was able to meet my mentor, which helped me gain direction for the project. We discussed details and challenges regarding the implementation of the AI model selection based on image modality. Although Java is a new language to me, I wanted to learn and work my way through it for the project. This week, my task was to understand parts of the LibreHealth toolkit code and prepare a list of tasks at an atomic level. After that, I will need to update them to the GitLab issues list. Coming up with the task list To come up with the task list, during the meeting with my mentor, I received some guidance on where to start with code exploration and understanding. Initially, it was challenging for me to understand the code, so I started looking at Java tutorials on YouTube. I gained some understanding of how Java works. However, I knew this was not enough for me to completely understand the application, and it would take time, so I also started looking for patterns in th...

This week I was researching AI models to implement for the different modalities. More specifically, Mammography. I also finally had the opportunity to talk to my mentor on Thursday (June 1st, 2023). You can find the recorded meeting link  here . Building a Breast Cancer Classification Model from Scratch At the start of the week, I attempted to implement a VGG16 classifier to distinguish between malignant and benign calcifications in a mammogram. Using PyTorch, I developed the model and achieved a validation accuracy of approximately 70%. The source code can be accessed  here . This model and code serve as a starting point for implementing a more complex model mentioned in this  paper . Briefly, the paper discusses building a model solely on the region of interest (ROI) of the calcification/tumor and adding layers and training on the entire images. This method has significantly reduced the training loss and, consequently, improved overall accuracy. Mentor Discussion and Ch...

Prepping for GSoC The final week of community bonding period ends and official coding begins. I was able to describe myself, set up my blog, and set up payment mechanisms, among other things. Google's instructions were spot on. I still need to set up my tax forms, for which I contacted Google. They told the tax forms will be available by June 5th, and I would have to file them by July 1st (for anyone who is interested). Exploring the Open Source Community I also read through the blogs of my fellow GSoC colleagues at LibreHealth. It was great reading about their journey and how they ended up being chosen for GSoC. They are all really capable. They explained about their exciting projects to which they are contributing towards. I was able to understand what an open source community is and get an essence of how people come together to build great products. Unfortunately,  I have not yet had the opportunity to interact with my mentors about my GSoC project proposal. I heard on other pro...

GSoC Acceptance I was amazed to know that I got accepted into Google Summer of Code 2023. This is my first time applying and getting into GSoC. I remember the time when I sat at the library for hours, working on the preliminary tasks, figuring out solutions, and writing an extensive proposal. Reiterating through the solution and proposal back and forth, I watched lots of YouTube videos and read blogs providing tips to draft a good proposal. All the efforts paid off when I received the email of acceptance. The Org & Project I drafted a proposal to the LibreHealth Organization. LibreHealth is collaborative community for free & open source software projects in Health IT, and is a member project of Software Freedom Conservancy. It is amazing to just think about how contributors from all over the world come together and build things that impact the world positively. This idea is what brought me into open source. Before jumping into the project details, here are a few things abo...