Improving Clinical Diagnoses with a Computer Vision AI App

Streamlining Digital Pathology Analysis with Andrew Ng’s AI Vision

Introduction

Over the past month, our team has worked with several subject matter experts (SMEs) to build out custom AI computer vision applications to help them with their everyday workflows. After using LandingLens to train and test models quickly, these users worked with the newly released Python Library to build computer vision apps of their own. In each case, we were able to hack together a working prototype in less than an hour. This has led to several breakthroughs as these SMEs have brought concepts to life in a working proof of concept.

Use Case

One case in particular which has received a lot of attention in the medical world is that of Dr. Brian Cone, a Computational Pathologist based in Los Angeles. Pathology, or the study of disease on a cellular level, involves the examination of tissues and cells with microscopes to diagnose and treat various medical conditions. Dr. Cone and his team spend their time characterizing the “morphology” of tissues and cells to help better diagnose issues.

Challenges

Dr. Cone grew tired of manually reviewing whole slide images (WSI), which are digitized glass slides containing tissue samples, and started looking for faster solutions which would streamline his workflow and allow him and his team to focus on corner cases. He then found LandingLens, the computer vision platform from Landing AI. By using our newly-released Visual Prompting approach, he’s been able to train unique models to look for specific disease characteristics. Once he got the model working, he was able to utilize our APIs to deploy that model on his own app, which is now being used to quickly classify large numbers of WSI images—normally thousands of hours of work—instantaneously.

Solution

Let’s take a look at one of Dr. Cone’s real-life use cases. In this example, Dr. Cone has several images of lung tissue, and he’s trying to determine whether they’re cancerous or not.

To start, Dr. Cone uses Visual Prompting in LandingLens to develop a model that detects cancerous regions. He uploads some images to LandingLens and labels a few pixels, specifically a few pixels of Tumors in Blue and a few pixels of Non-Tumor tissue in Yellow. He then clicks Run and waits a few seconds for LandingLens to process his prompts and return predictions on all the pixels, including those which are unlabeled.

He’s then able to review the results to make sure his new pathology model is performing adequately. After checking a few images, he’s satisfied and ready to ship his model to his application via an API endpoint to be used widely by his team.

Dr. Cone uses the models he trains in LandingLens to power the first Digital Pathology application that automatically detects tumors. In this early example you can see one of Dr. Cone’s teammates uploading and running inference on several hundred images, a task that would manually take days. The app quickly classifies and tags images with their diagnoses, summarizing the findings with a simple pie chart at the bottom. The app also shows previews of the images with their predictions (dark blue in this case is tumor). As you can imagine, this new flow allows Dr. Cone and his team to focus on the most critical items that require their trained attention. Whenever a new use case or task comes around, Dr. Cone quickly trains a new model on LandingLens and deploys it to his app, adding new predictive capabilities in less than 10 minutes. This sort of agile flow is essential for Dr. Cone, who regularly works on different disease types. By imparting his knowledge to an AI computer vision model, Dr. Cone is able to scale his efforts and rapidly deliver insights to push pathology projects forward.

Conclusion

If you’re interested in becoming a Landing AI design partner and have a specific deployment use case in mind, we’d love to get in touch. It’s never been easier to build and test a proof-of-concept with little to no AI experience. The primary challenge is finding experts with a problem worth solving. My hope with this post is to encourage those experts, with world changing problems, to make the leap and build a quick turnaround proof-of-concept exploring whether AI computer vision could help improve your workflow and that of your industry. You can start using LandingLens for free. If you’ve trained a model and want to build your own app like Dr. Cone’s, you can access example projects in our Python and JavaScript repos or follow our step by step deployment guide.