A friend drove two hours today for her breast cancer chemotherapy only to learn it was canceled due to NIH cuts. Clinical trial ended midstream. No treatment, no restart, no plan, no information. Her bro-in-law's chemo was also canceled.
For a first demonstration, we focus on the large scope of neuroimage analysis. VoxelPrompt can characterize and segment hundreds of anatomical and disease regions-of-interest, support various morphological measurements (e.g. tumor growth over visits), and produce language-based characteristics.
Jon Rauch
This flexible framework excels across many neuroimaging tasks, enabling a wide range of clinical and research goals.
Clinicians and scientists face a broad range of analytical imaging aims, yet most AI medical imaging tools used in practice are designed for narrow, predefined computational goals and data modalities.
Some examples of what VoxelPrompt can do!
Adrian Dalca
Adrian Dalca
Most practitioners lack compute resources or machine-learning expertise to build new models for their specific needs. This creates significant barriers to translating AI advancements into scientific and clinical workflows.
A core technical contribution is jointly learning the language-based agent with the adaptive vision network, which leads to a powerful mechanism for flexibly addressing complex analytical targets.
Adrian Dalca
Introducing VoxelPrompt, an agent-driven model that can tackle a landscape of diverse radiological tasks, extending far beyond those addressed by today’s specialized models. VoxelPrompt addresses complex objectives with a solution grounded in interpretable vision operations.
Paper: lnkd.in/ezWSX6zQ
Led by Andrew Hoopes (ahoopes.github.io), also with Victor Butoi, John Guttag and myself.
At @csail.mit.edu and the The MGH/HST Martinos Center for Biomedical Imaging
VoxelPrompt uses a language agent that interprets user prompts, predicts step-by-step instructions (as code), and employs a powerful vision network to analyze any number of patient-specific scans at once.
Adrian Dalca
Adrian Dalca
Adrian Dalca
lnkd.in
We present VoxelPrompt, an agent-driven vision-language framework that tackles diverse radiological tasks through joint modeling of natural language, image volumes, and analytical metrics. VoxelPrompt...
