이미지: X — 뉴스 앰프 화면 갈무리
Summary
- Google used a reinforcement learning technique called "ResidencyRL" to train Gemini 3.5 Flash and improve its clinical consultation abilities
- The training drew on 49,870 simulated telehealth consultations covering 81 conditions, using a GRPO-based reward structure
- In a human evaluation involving 97 raters, the model won 88% on overall impression and 91% on completeness of information gathering, but scored lower on hallucination-free responses (12%) and prescription safety (42%)
- 훈련 방식
- ResidencyRL (강화학습, GRPO)
- 대상 모델
- Gemini 3.5 Flash
- 훈련 규모
- 81개 질환, 4만9870건 가상 원격진료 상담
- 시나리오 구성
- 일반 진료 83%, 병력 청취 15%, 까다로운 환자 대응 2%
- 사람 평가
- N=97, 전반적 인상 88% / 정보 수집 완결성 91% 우세
Gemini trained through a simulated residency
Google used a reinforcement learning technique called "ResidencyRL" to train Gemini 3.5 Flash and improve its clinical consultation abilities. The training involved 49,870 simulated telehealth consultations covering 81 different conditions. Much like a hospital resident going through training, the AI accumulated repeated consultation experience with simulated patients.
The training pipeline builds scenarios by combining demographic information such as age, race, location, and gender with the DDXPlus differential diagnosis dataset, along with personality traits like neuroticism, agreeableness, and extraversion. The generated scenarios were divided into three types for training: general consultations (83%), history-taking (15%), and difficult patient interactions (2%). Each consultation episode consists of up to 68 actions and reasoning steps, with the AI scored across six categories — diagnosis, prescription, questioning, communication, documentation, and tone — while being penalized for overly long responses, hallucinations, or contraindicated prescriptions. Based on this reward structure, the model was refined through repeated reinforcement learning using GRPO (Group Relative Policy Optimization).
What it means to teach clinical conversation through RL
Reinforcement-learning-based fine-tuning is especially useful for conversational tasks that don't have a single correct answer. Medical consultations need to be evaluated not just on diagnostic accuracy, but also on how thoroughly necessary information is gathered from the patient and how easy the explanations are to understand. This approach differs from conventional fine-tuning that simply mimics reference text, in that it combines these multiple criteria into a single reward score and lets the model find its own balance among them.
On August 11, Google also published research adding real-time voice and video consultation capabilities to AMIE, its medical AI research system. AMIE is a separate project focused on reading non-verbal cues such as facial expressions and breathing, while ResidencyRL is the result of training a distinct model, Gemini 3.5 Flash, specifically for text-based telehealth scenarios. The two projects show that Google is refining AI for medical consultations along multiple fronts at once.
Human evaluation shows an edge, but safety remains a work in progress
In a comparative evaluation involving 97 raters, the model trained with ResidencyRL won 88% of the time on overall impression. It was also preferred 91% of the time for completeness of information gathering, and scored higher than the baseline model on diagnostic assessment (66%) and prescription appropriateness (75%). However, its win rate for hallucination-free responses was only 12%, and for prescription safety it was a relatively low 42%. This suggests that while diagnostic and consultation abilities improved noticeably, safety-related improvements still have further to go.
These results show medical AI moving beyond simple information retrieval toward mimicking the act of clinical consultation itself. However, the low win rates on hallucination and prescription safety in the human evaluation should be read as a sign that such models still have hurdles to clear before they can be deployed in real clinical settings.
