GSoC LibreHealth 2023

 This week I worked on two things,  - Implemented the AI Tool OHIF component - Minor bug fixes and logging Implementing the AI Tool OHIF component To give some context, in the OHIF viewer I had to add a button in the toolbar (refer "AI Tool" in the image above) that allows calling any of the existing AI models in the AI Model Service. Prior to this a component was implemented that takes a look at the metadata of the image and calls the respective model that was configured in the admin settings, but only one model could be called for a particular image. In a scenario where we would need to call two models for an image, say we have a model for identifying breast cancer region and another model for identifying type of cancer it would become impossible. We implement the AI Tool so that we can call a secondary or tertiary model. This task was quiet difficult for me since I was new working in JS and also there were new components that I had to work with. I met with my mentor twice

This week with the help of my mentor I was able to figure out the issue where the UI buttons were not reflecting. I progressed on that part as well on implementing Java methods that on click of the OHIF AI react hook buttons the right AI model is called from the AI Model Service. re-Building the viewer component So I had to build the source again after making the changes to the UI component. This is something completely new to me so I will explain the steps, First I had to uncomment the line (as shown in the image) in the docker compose file that rebuilds the OHIF viewer extension changes, Before rebuilding the module, we need to pull the ohif viewer submodules to our codebase, to do that we need to run the following commands marked in the image. This is present in the README.md file in the 'nsf-ftw' branch. Now to start rebuilding the module we can use IntelliJ IDE to open the Maven build project tab. Here select the 'build-viewer' option in the 'Profiles'. You

 This week can be characterized as "Struggle and Progress" I worked on two major things this week, - Integrated the ever so complex kaggle RSNA mammography top performing model - Codebase deep dive and coding for React OHIF hook implementation Kaggle RSNA Mammography Model Integration After going through the codebase multiple times and fixing the inference code on my local everything was working fine except for the part where the image is cropped using a YOLO model or an Otsu thresholding method before being passed to the final classification ConvNext Model. The issues I faced were, The models were tuned for specific GPUs to run much faster but for the LH application we would need to run on CPU or GPU based on their availability. So I needed to use the unoptimized models and figure out a few other complexities surrounding them such as modifying a chunk of codebase, identifying the right model to initialize, etc There were a lot of dependency issues for the YOLO model which I

There was a lot of struggle this week and I am also expecting a lot of difficulties to come by the further weeks. The two biggest challenges I am facing, - Integration of the RSNA Kaggle Mammography classification model - React hook implementation in the OHIF viewer RSNA Kaggle Mammography Model The code base and model complexity that are put available by the top Kaggle solutions of the RSNA Kaggle competitions are large and takes time to decode. I've been working on the top solution that uses a YOLOX model to crop the image and then a CONVNEXT model to perform the classification. The code that's provided is for training and evaluation. It needs to be converted to perform a single image inference. The code base is optimized for specific GPUs but for our case it needs to be generalized to run on CPU or GPU (any). I was able to run the models but the YOLOX model is throwing smaller boxes to crop off. I need to debug and check what is missing. If it's still the case I will nee

 This week I worked on one big task which is, - Integrating mammography cancer classification model The Challenge Initially I planned and still plan on integrating the RSNA mammography cancer classification top ranked model from kaggle. As I dug deep down into the code base of the top solution I realized it is going to take time to integrate it. It uses a YOLO model to detect the mammogram area to crop out and then uses a convnext model to perform classification. I already used a day to figure out integration aspects and how I can productionize it in the LH radiology application. Thinking about this complicated model and also integration strategies it was quiet overwhelming. So I wanted to somehow break down my tasks, strategize and integrate it. Plan of Action Meanwhile I remembered of the VGG16 model that I created to do mammography cancer classification at the start of GSoC. I decided to first integrate the VGG16 model since that would allow me to setup the codebase in the applicati

 This week I worked on three things - Refactoring the model selection code base to move from endpoint redirection to calling the actual methods by importing them - Added test cases for model selection and fix issue with docker image - Progressing on integrating the mammography breast cancer classification model Refactoring the model selection code base The model selection code base was implemented initially where it looks at the imaging information and redirects to the appropriate model endpoint. Something like this, The problem faced with this way of implementing it, the base url was hardcoded and pointed to localhost. Mind that only if systems had set their localhost in their hosts file this would work and hence it was not system agnostic. This was causing my model selection test cases run in my local but not in the gitlab ci/cd pipeline. The other thing I read was in RESTful services this sort of redirection is generally not a good design since in RESTful each endpoint is considered