AI Fundamentals

James Hillis
Attending Neurologist, Massachusetts General Hospital
Instructor in Neurology, Harvard Medical School

Department of Neurology
Disclosures
Disclosures relevant to this presentation:
Funding through the Mass General Brigham Data Science Office for AI
algorithm research including from industry partners including
Annalise.ai, GE Healthcare, Nuance, NVIDIA, Viz.ai.

Other disclosures:
Grant funding from Project Data Sphere, LLC for research related to
neurologic immune-related adverse events of cancer immunotherapy.
Investor in Elly Health.
Artificial intelligence: definitions
• Definitions of artificial intelligence include:
• “What we don’t know how to do yet” (Anderson, Foundations of computer technology,
1994).
• “To make computers do the sorts of things that minds do” (Boden, AI: its nature and future,
2016).
• Artificial general intelligence is “a hypothetical computer program that can perform
intellectual tasks as well as, or better than, a human” (Hodson, 1843 magazine, The
Economist, 2019).

• Turing test = a human interrogator interacts with a machine and human via written
statements yet cannot distinguish between them (Turing, 1950).
Artificial intelligence: computing power

https://en.wikipedia.org/wiki/Moore%27s_law#/media/File:Moore's_Law_Transistor_Count_1970-2020.png
 Artificial intelligence: definitions
Supervised learning = providing labels so that a
computer can learn to identify those labels

Unsupervised learning = allowing a computer

to infer relationships without labels

Reinforcement learning = computer receives

and incorporates feedback

Artificial neural network =

a network that goes from
input to hidden to output
layers, which is inspired
by neurons and synapses.

https://en.wikipedia.org/wiki/Artificial_neural_net
https://en.wikipedia.org/wiki/Deep_learning#/media/File:AI-ML-DL.svg work#/media/File:Colored_neural_network.svg
Machine learning: dataset
• Training set: the cases used to train the model.

• Validation set: the cases used to tune the model (exposed during training).

• Test set: the cases used to test the model (withheld during training).

• Pragmatic considerations:
• Common machine learning practice is to split a dataset 80/10/10% into the three sets.
• Cross-validation is a strategy for “rotating” training / validation sets.
• US Food and Drug Administration (and other regulatory bodies) typically wants an entirely
separate test set (including from different sites) to better demonstrate generalizability.
Machine learning: challenges
• Generalizability: ensuring a model works with different data (demographic
groups, technical parameters, etc).
• Overfitting: when a model works well for data that are like the training data
but not new data.
• Explainability: being able explain how a model came to an outcome (with
the concern being that a model is a “black-box”).
• Data drift: the change of input data that decreases model robustness.
• Concept drift: the principles behind a model changing.
Machine learning: supervised learning
• Providing labels so that a model can learn to identify those labels.

• These labels can be generated to different degrees of “resolution” with the

following examples for a CT intracranial hemorrhage algorithm:
• Case-level binary classification: hemorrhage present or absent.
• Segmentation: outlining the hemorrhage.

• An increased degree of resolution often allows for a smaller training dataset.

• Semi-supervised learning uses a combination of labeled / unlabeled data.

Machine learning: key statistics
• Machine learning models with classification
outputs (e.g., positive / negative) will typically
output a classification score between 0 and 1.

• A receiver operating characteristic (ROC) curve

can be plotted by taking the sensitivity /
specificity at each of these scores.

• The area under the ROC curve (AUROC or AUC)

provides a metric for assessing the model; it
ranges between 0 and 1 with an AUC closer to 1
showing better performance.
Hillis et al, JAMA Network Open, 2022
Artificial intelligence: clinical impact
Three parting thoughts about how I see AI impacting medicine:

• Tip of the iceberg: it is easiest to conceptualize how AI can replicate human tasks,
but the more exciting part is how AI can augment medicine through beyond-human
tasks.

• Rome wasn’t built in a day: AI will be gradually introduced for specific tasks (e.g.,
prioritizing radiology worklists) rather than arriving overnight.

• Art versus science: Medicine will involve more consideration of probabilities and a
key role for health care providers will be the interpretation for patients.