Prescribing in the Age of AI

Welcome. This session is about using AI tools well when you make medication decisions. The message in one line: AI can make you a better prescriber, if you stay in charge. Most of you already use it, and the evidence says that clinicians who use it well can make better decisions and spend less time on documentation.

By the end you should be able to do four things: describe what the evidence shows AI can do for clinical work; judge when to trust an AI answer and when to check it; use AI well with the 6 A's of AI-assisted care (the evidence-based medicine cycle you already know, plus one new step) and a few key prompts; and use AI responsibly as a resident.

How to use these notes: read them alongside each slide; they cover everything in the session. Several slides are interactive: the poll, the dose decision on the Anticipate slide, the "save your standing instructions" pop-up on the Ask slide, the claim check on the Appraise slide, and the two buttons on the Apply slide. Do them. The Assess slide uses the answer you lock in.

Start with a quick question for yourself: in the last week, did you ask an AI tool a medication question? Many residents have. Usually that's a general chatbot, sometimes a clinical tool like OpenEvidence, and more and more it's something built into the EHR or a search engine's summary.

That's the starting point for today. The question isn't whether you'll use AI. It's how.

Pick the option that fits you, then think about the last drug question you asked a chatbot. Did you check the answer? Against what source?

Here's the message for today in one line: AI can make you a better prescriber, if you stay in charge.

A useful way to think about it: use AI like a sharp consultant. A good consultant is often helpful and sometimes wrong, and you trust them more on some questions than others. You listen, you weigh what they say, and you still make the call. You're still the attending.

We'll cover four things:

1. Why use it: what the evidence shows AI can do when clinicians use it well.
2. When to trust it, and when to check: how these tools produce answers, and where they predictably slip.
3. How to use it well: a framework I call the 6 A's of AI-assisted care, worked through one patient, plus a few prompts that make AI a better second reader.
4. How to use it responsibly: privacy, transparency, supervision, and owning the order.

Where the 6 A's of AI-assisted care come from. You already know the evidence-based medicine (EBM) cycle, often taught as the "5 A's": Ask a focused clinical question, Acquire the evidence, Appraise it, Apply it to your patient, and Assess how it went. It's the same cycle behind the AAMC's Core EPA 7, "form clinical questions and retrieve evidence to advance patient care," one of the skills U.S. medical schools expect graduates to bring to residency. Using AI is mostly a new way to do the Acquire step, but it changes the other steps too, and it calls for one new step at the start: Anticipate, committing to your own answer before you ask the AI. That makes six. To be clear, this isn't an official guideline from any society. I developed it for this session by extending the established 5 A's, so you can use AI well without learning a whole new framework.

First, why use it at all? Ong and colleagues (Cell Reports Medicine, 2025) built 91 prescribing-error scenarios and compared three setups: a pharmacist alone, an LLM alone, and a pharmacist working with the LLM.

The pair beat either one alone. For errors with potential for serious harm, the pair was about 1.5 times more accurate than the pharmacist working alone. Two details worth knowing: the model was grounded in reference sources, meaning it looked up trusted references and answered from them rather than from memory (we'll come back to this on the Acquire slide), and even the best arm was only about 61% accurate overall. This is a hard task, and the team was better, not perfect.

Notice what the winning setup was. It was an expert who did their own thinking and used the model as a second reader, not a model replacing the expert. Everything else today is about how to be the human half of that team.

Before we get to how AI fails, here's what it can do when clinicians use it well. Three randomized trials:

• Better management decisions. Goh and colleagues (Nature Medicine, 2025) randomized 92 practicing physicians to GPT-4 plus usual resources or usual resources alone. They worked through five complex management cases based on real patient encounters, with information revealed step by step. Expert-written rubrics scored each answer as a percent of possible points. The GPT-4 group scored 6.5 percentage points higher, about the same as GPT-4 working alone, and spent about two minutes longer per case. Management reasoning (what to treat, what to test, how to weigh risk) is the heart of prescribing.
• AI as a teammate. Everett and colleagues (npj Digital Medicine, 2026) randomized 70 clinicians. Their AI was designed to collaborate: the clinician and the AI each made an independent assessment, then the AI produced a synthesis showing where they agreed and disagreed. Diagnostic accuracy was 85% when the AI went first and 82% when it went second, compared with 75% using usual resources.
• Less time on notes. Lukac and colleagues (NEJM AI, 2025) randomized 238 outpatient physicians across 14 specialties to one of two commercial ambient AI scribes (Microsoft DAX Copilot or Nabla) or usual care. Nabla cut time spent writing notes by 9.5%; DAX didn't change it significantly, so the time savings depended on the product. Both improved burnout, task-load, and work-exhaustion scores, though those were secondary outcomes that need confirmation. Clinicians reported clinically significant inaccuracies "occasionally."

The caveat: two of these used simulated cases, and real-practice evidence is still coming. But the direction is consistent with the Ong study: the benefit comes from clinicians and AI working together.

So AI can help. It can also fail, and the failures are predictable once you know how it works. One idea explains most of what follows. A large language model produces an answer by predicting the most plausible next words, given your prompt and the patterns it learned in training. By default it generates. It doesn't look anything up in a drug label.

Some clinical tools, like OpenEvidence, look up sources first and write the answer from them. That helps a lot, but the answer is still generated, so it can still be wrong.

Three predictions follow from this:

1. Sounding sure isn't being right. The model states a wrong dose or interaction with the same confidence as a correct one, so the tone of an answer tells you nothing about whether it's accurate.
2. Ask twice, get two answers. The model writes a fresh answer each time instead of looking one up, so the same question can come back differently.
3. Harder questions, weaker answers. The model learned from huge amounts of general text, so a common question like "What's the usual apixaban dose for AF?" has been answered thousands of times in what it learned from, and it handles that well. Your patient's exact combination of age, weight, creatinine, and other drugs isn't written anywhere. To answer, the model has to apply the rules to her, and that's where it makes more mistakes. One common mistake is answering the familiar version of the question and skipping the details that make her different. You'll see that in a moment, when one response treats her as "elderly, so lower the dose" and ignores the actual criteria.

If you remember the mechanism, you don't need a list of warnings. You can predict where the tool will fail.

The studies line up with those predictions.

Inconsistent. Van Nuland and colleagues (2024) gave ChatGPT (GPT-3.5) 30 real questions received by hospital pharmacists, including hard ones on drug levels, extravasation, pharmacogenetics, and IV compatibility. It gave a recommendation on 23. Of those, about half (48%) were correct: 26% correct and complete, and 22% correct but missing something. Another 30% were partly right and partly wrong, and 22% were completely wrong. The biggest problem was consistency: when the team regenerated the answer, only 10% came back the same.

Sounds complete when it isn't. Stroop and colleagues (Br J Clin Pharmacol, 2025) had clinicians rate GPT-4's answers to pain pharmacology questions. 96% were rated accurate, but only 63% were complete, and the gaps were concentrated in pharmacokinetics and drug interactions. Hold on to that; it matters in the case.

Harder question, worse answer. Hsu and colleagues (JMIR Med Educ, 2023) found ChatGPT gave appropriate answers to 61% of the public's medication questions, such as how to take a drug, but only 39% of questions from hospital clinicians, which tended to be more specific and patient-dependent. The authors concluded it could help with basic medication questions; the risk was in the complex, patient-specific ones.

Van Nuland's data also back up the third prediction: fully correct answers fell from 4 of 13 simple questions to 0 of 9 complex ones.

"Aren't these older models?" You may be wondering that, and it's a fair question. Van Nuland and Hsu tested GPT-3.5, and Stroop tested GPT-4. Newer tools are better. But Ipema and colleagues (2026) tested current clinically focused chatbots, including OpenEvidence, and every one still scored below drug-information pharmacists; we'll see that study on the Acquire slide. More importantly, these failure patterns come from how the tools work, so the habits we build today still apply as the tools improve.

This slide shows what can happen when you ask a chatbot the same vague question twice and hit regenerate. The two responses are illustrative, written for teaching to show what commonly happens.

Response 1 sounds sensible and confident: lower the dose because she's elderly. It's wrong. Age alone isn't a reason to reduce apixaban. Response 2 gives the actual labeled criteria. Both read with the same confidence.

Try this yourself with any chatbot: ask a vague dosing question, regenerate, and compare. Then notice the real problem. The vague question invited a guess. We'll fix the question in the Ask step.

So how much should you trust it? It isn't all or nothing; it depends on the task. Here's a map of common tasks in a resident's day. Some items preview tools we'll cover next, like asking "What am I missing?" and committing to your own answer first.

Use it, then review. These are high-value, lower-risk uses: explaining a mechanism, a guideline, or an unfamiliar drug; getting a second read on your plan by asking what you're missing; drafting patient instructions and education; finding and summarizing evidence (then opening the citations); and drafting letters, summaries, and teaching materials.

Use it, then verify. AI can help with these, but check the answer against the label, a calculator, or a drug database: doses and dose adjustments, renal and hepatic adjustment, drug interactions, anything that changes the order, and AI-drafted notes and inbox replies before you sign them.

Don't. Don't put patient identifiers into tools your institution hasn't approved. Don't let AI make the final call. Don't cite a reference you haven't opened. And don't ask AI before you've formed your own answer; that's how AI ends up replacing your judgment instead of sharpening it.

So why doesn't everyone get that benefit? Goh and colleagues (JAMA Network Open, 2024) randomized 50 physicians in family medicine, internal medicine, and emergency medicine, 24 of them residents. They worked through up to six challenging diagnostic cases, with GPT-4 plus usual resources (such as UpToDate and Google) or usual resources alone.

How it was scored. For each case, physicians filled out a diagnostic reasoning worksheet: their top three possible diagnoses, the findings for and against each, their final diagnosis, and up to three next steps. Expert graders, blinded to group, awarded points for each part, and a case's score was the percent of possible points earned.

The results. Physicians with GPT-4 scored a median of 76%, versus 74% without it, not a significant difference. The researchers also entered each case into GPT-4 with no physician involved, using a prompt they had carefully designed. GPT-4 alone scored a median of 92%, outperforming both physician groups.

The authors point to prompting as part of the explanation. The researchers wrote a careful prompt; the physicians had no training in how to use the tool. They suggest that teaching clinicians to prompt well could help close the gap, which is what the Ask step and the standing instructions are for. One caveat: these were written cases with the information already gathered and summarized, so this doesn't mean AI should diagnose on its own. Gathering the right information from a real patient is a big part of the work.

How can the tool alone beat the physicians using it? Siden and colleagues (npj Digital Medicine, 2025) analyzed physicians' chat logs from these trials. Some pasted the whole case, some pasted pieces, some summarized, and some typed short searches. No style scored higher. The authors suggest that prompting strategy, cognitive engagement, and how physicians interpreted the answers may matter more.

Put that beside the trials on the earlier slide. When the workflow is built for collaboration, clinicians improve. Simply having the tool open isn't enough. That's why we need a framework, and it's what the rest of this session is about.

Here's the framework, which I call the 6 A's of AI-assisted care. It isn't an official guideline; I developed it for this session from something you already know, the evidence-based medicine (EBM) cycle: Ask a focused question, Acquire evidence, Appraise it, Apply it to the patient, and Assess how it went.

Using AI is mostly a new way of doing Acquire. But it changes the other steps too, and it needs one new step at the front: Anticipate. Before you ask the AI anything, commit to your own answer and how sure you are. That's the guard against automation bias, the tendency to accept a confident machine answer.

Each step has one question to ask yourself and one failure it protects against. We'll walk through all six with one patient. Today we'll use a prescribing case, but these steps work for any clinical question you bring to AI.

Mrs. R is 82 and was found to have new nonvalvular atrial fibrillation in clinic. She has hypertension and diet-controlled type 2 diabetes, so her stroke risk is high, and the team wants to start apixaban.

She was just started on diltiazem ER 240 mg daily for rate control and also takes lisinopril and atorvastatin. Her serum creatinine is 1.2 mg/dL, which gives a Cockcroft-Gault creatinine clearance of about 36 mL/min using the chart weight.

Her weight in the chart is 64 kg. Note that it was recorded three months ago, and she mentions her appetite has been poor. Keep that in mind.

The question: what dose of apixaban?

Before any AI: commit. Pick a dose and a confidence level, then lock it in. If you're reading this as a printed handout, write your answer down.

This takes 20 seconds and does two things. First, it gives you something to compare the AI's answer against, so you notice when it disagrees with you instead of quietly adopting it. Second, it lets you check your calibration at the end: were you as right as you felt?

Don't look anything up yet. Gut answer from what you know.

Most bad AI answers start as bad questions. The weak prompt has two problems.

• It leaves out the numbers that drive the decision (age, weight, creatinine, interacting drugs), so the model fills the gap with something generic.
• It's leading. "Should I lower it since she's 82?" invites agreement, and models tend to go along with the premise of a question.

The strong prompt gives the decision-relevant data, asks neutrally, names the source you want the answer anchored to (the FDA label), and asks the model to say what it's unsure about. Hsu's study showed answers get worse as questions get more clinical, so the context you supply matters most exactly when the question is hard.

One more rule: include only the clinical details the question needs. No names, record numbers, or dates of birth. De-identify, and use the tools your institution has approved for patient data.

Type less by saving the standing part once. Most of a good prompt is the same every time (name the source, cite it, flag what's missing or uncertain). Only the patient details change. Save the standing part as custom instructions, a custom GPT, or a project instruction or skill in Claude, and the per-patient prompt can be as short as the one on the slide. A starting set of instructions:

• Answer from the primary source: the FDA label for drug questions, or the guideline I name. Cite the section you used.
• List the patient factors you used and any you'd need but don't have.
• If a value that changes the answer is missing or may be out of date, say so before answering.
• Flag anything you're unsure of, and separate what the source says from your own reasoning.
• Don't just agree with how I phrased the question.

Use a tool your institution has approved, never put patient identifiers in saved instructions, and remember that saved instructions make the habits automatic but don't replace verifying what changes the order.

Acquire is choosing the right tool for each part of the question, and sometimes the right tool isn't a chatbot.

• Dose-reduction criteria live in the label or a drug database, so open it yourself. You may have just asked the AI to use the label, and that does point it in the right direction. But an AI saying "per the FDA label" doesn't mean it read the label. A general chatbot usually generates what the label probably says. For criteria this short and this decisive, reading the label is faster than checking the AI's version of it.
• Creatinine clearance and risk scores are arithmetic, and a calculator doesn't hallucinate.
• Interactions: start with an interaction checker, then use AI to explain the mechanism if you want.

A term you'll see on this slide: grounded. A grounded AI answers from a specific source it retrieves first, like the drug label or published studies, rather than from what it absorbed during training. Think of it as an open-book exam instead of a closed-book one. Some tools do this automatically (OpenEvidence draws on the published literature), and you can do it yourself by pasting the label section into your prompt and saying "answer only from this." Grounding helps a lot, but the answer is still generated, so the model can still misread or misapply the source. That's why you still open the citation.

• "What's the evidence?" An evidence-grounded tool like OpenEvidence is a strong starting point, but open the citations and confirm they say what the answer claims.
• Patient instructions: AI is genuinely useful here, especially when it's grounded in the label, as long as you review the result.

Ipema and colleagues (JACCP, 2026) compared four clinically focused chatbots (OpenEvidence, Glass Health, Clair, and DougallGPT) and ChatGPT against drug-information pharmacists on 30 real drug-information questions. OpenEvidence scored highest, with ChatGPT second, and OpenEvidence was rated "very good" on adverse-reaction and stability/compatibility questions. Across tools, the strongest area was the absence of false information; the weakest was the evidence supporting the answer. Overall, the best tools landed in the study's "average" range, and every chatbot scored significantly below the pharmacists. The pattern fits our framework: these tools are useful and usually not wrong, but you still need to check the source behind the answer.

De Jesus and colleagues (2025) showed that grounding a model in official drug labeling improved patient instructions and reduced critical errors. The source behind the answer matters as much as the tool.

Here's an AI answer to our case. It's illustrative, written for teaching, but typical of what you'll see. You don't have time to verify every sentence, and you don't need to. Verify the claims that would change what you order.

1. "Apixaban is reasonable." Consistent with the plan the team already chose. Low priority.
2. "CHA₂DS₂-VASc is 5." A quick calculator check confirms it (age ≥75 = 2, female = 1, hypertension = 1, diabetes = 1). Either way she's well above the treatment threshold, so it doesn't change management.
3. "Over 80 with CrCl about 36, so 2.5 mg twice daily." This is the one that matters, and it's wrong. The US label for AF reduces the dose only when at least two of three criteria are met: age ≥80, weight ≤60 kg, serum creatinine ≥1.5 mg/dL. With the charted data she meets one (age). Creatinine clearance isn't one of the criteria. Check: the label's dosing section or a drug database.
4. "Diltiazem inhibits CYP3A4, another reason to reduce." Misleading. The label's interaction dose adjustment is for combined strong CYP3A4 and P-gp inhibitors such as ketoconazole or ritonavir. Diltiazem is a moderate inhibitor. It raises apixaban levels modestly and doesn't call for a dose change. Check: the label's interaction section or an interaction checker.
5. "With or without food." True, and low stakes.

Notice that both errors are in kinetics and interactions, exactly where Stroop found the gaps. And the answer said "per guidelines" with nothing you could open, which matches Ipema's finding that evidence support was the weakest area.

Bottom line with the data as charted: the label supports 5 mg twice daily.

Apply asks two things: does this answer fit the patient in front of me, and am I ready to own it?

The AI answered the patient we described. Before signing, use the second-reader move: tell the AI your plan and what you know, and ask what you're missing. Here (an illustrative response) it points out that the weight is three months old and she has reported a poor appetite. If she now weighs 60 kg or less, she meets two criteria and the dose changes.

So weigh her. Today she's 59 kg. She now meets two criteria (age ≥80 and weight ≤60 kg), and the label dose is 2.5 mg twice daily.

Two things to notice. First, the AI could only flag this because you told it about her appetite. It only knows what you typed. Second, the AI's flawed reasoning on the Appraise slide happened to land on today's dose, but for the wrong reasons. That's not something you want to depend on.

Socrates and colleagues (JMIR Aging, 2025) found exactly this division of labor. An LLM was good at flagging which deprescribing criteria might apply, but poor at deciding whether a criterion actually applied to a given patient. Many of its errors came from complex inclusion and exclusion criteria or from missing clinical information, and its confidence didn't match its accuracy. Here the AI flagged the possibility, and you confirmed it by weighing her.

Clinically, the weight loss itself deserves follow-up, and her dose should be rechecked as her weight and renal function change.

Assess closes the loop. Compare your Anticipate answer with where you ended up.

• If you picked 5 mg twice daily: you were right on the charted data, but the stale weight changed today's answer.
• If you picked 2.5 mg because she's elderly: you got today's dose right for the same flawed reason the AI did. Age alone isn't a criterion. Worth noticing.
• If you picked "avoid apixaban": a creatinine clearance around 36 mL/min isn't a reason to avoid it.
• If you picked 2.5 mg because of diltiazem: diltiazem is a moderate inhibitor and doesn't trigger a label dose change. Today's 2.5 mg comes from age plus weight.

Then ask: What did the AI change? In this case it did both. One answer tried to talk you into a dose change for the wrong reasons, and the second-reader check caught the stale weight. Was your confidence right? Over time, this habit tells you where your own knowledge is reliable and where you lean on the tool.

That's the second-reader model from the Ong study: you think first, AI checks you, you verify what matters, and you decide.

Here are six moves that make AI a better second reader. Several turn the weaknesses we saw earlier into techniques.

• "Here's my plan and what I know. What am I missing?" This is the second-reader move from the Ong and Everett studies, and it's what caught the stale weight in our case. Do your own thinking first, then let the AI check you.
• "Argue against this plan." Models tend to agree with the premise of a question. Asking for the counterargument pushes against that.
• "What would change your answer?" This surfaces the criteria that drive the decision and the data you didn't provide.
• Ask again, or regenerate. In van Nuland's study, only 10% of answers stayed the same on regeneration. Use that as a check: if the answer changes, trust neither version until you verify it.
• Start a new chat for each patient. Stroop and colleagues noted that context can carry over into follow-up questions. A fresh chat keeps one patient's details out of another's answer.
• "Rewrite this for a patient at a 6th-grade reading level." Patient-facing explanations are one of AI's best uses. Review the result before it reaches the patient.

Pair these with the standing instructions from the Ask slide pop-up, so every answer cites its source and flags what's missing or uncertain.

Four guardrails.

• Protect privacy. Don't put identifiable patient information into any tool your institution hasn't approved for it. De-identify, and use approved enterprise tools when patient data is involved.
• Be transparent. The AAMC's Principles for the Responsible Use of AI in and for Medical Education start with responsible, transparent use, including disclosure and teaching learners to explain to patients how technology informed their care.
• Own the order. Your name is on it. "The AI said so" isn't a defense, clinically, legally, or ethically.
• Document what you verified. When AI helped, record what you checked and where (the label, the interaction database), not just that a chatbot suggested it.

Check your program's and hospital's current AI policy. These policies are changing quickly.

Four responsibilities that are specific to being a resident.

• Loop in your supervisor. Tell your attending when AI shaped a plan. Supervision depends on seeing your reasoning, and they can't supervise reasoning they can't see.
• Build your own judgment. Residency is when clinical judgment forms. A 2026 review of the state of clinical AI (Worth and colleagues) flags automation bias and clinician deskilling as emerging risks. The Anticipate step is the fix: commit to your own answer first, so AI sharpens your thinking instead of replacing it.
• AI is already in your EHR. AI scribes, which listen to a visit and draft the note, and drafted replies to patient messages are AI too. Scribes are worth trying. In three randomized trials, clinicians using them reported less burnout and a lighter workload (Lukac 2025, Chowdhury 2026, Shin 2025). Time savings were modest and varied by product, and clinicians reported occasional clinically significant errors. So learn how your site's tool works before you start, including how to tell the patient and get their OK before recording, and read every draft before you sign. Whatever you sign, you own.
• Watch for bias. A 2026 scoping review of ambient documentation (Wolfe and Welsh) found that the speech recognition behind these tools makes more errors across racial, ethnic, and linguistic groups, driven largely by dialect and accent, with documented higher error rates for Black speakers than for White speakers. More broadly, models can repeat outdated or biased practices from their training data. Be most careful where those errors would fall on patients who are already underserved.

Here's the one-slide version.

• Anticipate: commit to an answer and a confidence level first.
• Ask: give the numbers that drive the decision, ask neutrally, and name the source.
• Acquire: pick the right tool for each part: label, calculator, database, or evidence-grounded AI.
• Appraise: verify the claims that would change the order, especially kinetics and interactions, and open the citations.
• Apply: check that the inputs are true for this patient today, and own the decision.
• Assess: compare with your first answer and check your calibration.

Where AI shines: as a second reader on your plan, flagging interactions or problems you might miss, explaining mechanisms, and drafting label-grounded patient instructions that you then review. If you remember one prompt, make it "What am I missing?"

Where it needs verification every time: doses, renal and hepatic adjustments, interactions, and anything that changes what you order.

And the bottom line: use it like a sharp consultant. Often helpful, sometimes wrong. You're still the attending. AI can make you a better prescriber, if you stay in charge.

These references cover AI working with clinicians: the medication-safety, management, diagnosis, and ambient-scribe trials; the study showing that access alone isn't enough; and the reviews on deskilling and bias. Links are on the slide.

That's the end of the session. If you have questions, bring them to your next noon conference.

The references are listed on the slide with links. The case is fictional, and the AI responses labeled "Illustrative" were written for teaching to show common patterns. They aren't outputs from a specific tool.

A good next step: the next time you use AI for a medication question, run the 6 A's once, start to finish. Commit first, write a strong prompt, ask what you're missing, check the claims that would change the order, and compare at the end.