Google Research finds limits of transfer learning in genetic risk prediction

Google Research shared results in a post on its own blog evaluating methods for improving the accuracy of genetic risk prediction across populations. The core finding was a bit different from what one might expect. Transfer learning — carrying over a model trained on European-ancestry cohort data to another population — did boost predictive power when the target population's sample was small, but as that sample grew larger, accuracy actually got worse.

To unpack this a bit: a polygenic risk score is a statistical model that combines information from a huge number of genetic variants to estimate someone's probability of developing a particular disease or carrying a particular trait. Most large-scale genomic studies to date have focused on European-ancestry populations, which is why these models have historically predicted less accurately for other ancestry groups. Transfer learning is meant to address that gap by carrying over what a European-ancestry model has learned and applying it to populations with less available data.

What was tested

The evaluation used two cohorts: the UK Biobank (UKB), which contains European-ancestry population data, and Biobank Japan (BBJ), which contains East Asian population data. The traits under study were HDL cholesterol, LDL cholesterol, and blood glucose. Researchers scaled the UKB sample up to as many as 150,000 people and the BBJ sample up to 56,571 (HDL), 58,356 (LDL), and 74,431 (blood glucose) people, depending on the trait, tracking how prediction accuracy changed across different combinations of the two sample sizes.

The paradox that emerged as samples grew

Here's the accuracy range recorded for each of the three traits:

Trait	Minimum	Median	Maximum
HDL cholesterol	0.000	0.272	0.289
LDL cholesterol	0.000	0.161	0.182
Blood glucose	0.000	0.087	0.101

All three traits followed the same trend: accuracy was close to zero when both the UKB and BBJ samples were small, and it clearly rose as both samples grew together. But Google Research pointed to something else worth noting. Looking at the BBJ — the target population — sample size on its own, the benefit from transfer learning drawn out of UKB was large when that sample was small. Once the BBJ sample crossed a certain threshold, though, accuracy for models that included transfer learning actually started to drop.

Why this happens

Transfer learning is a method that carries skills learned in one domain over to another, and it's widely used to fill in gaps in data-scarce situations. The catch is that things change once the target population itself has accumulated enough data of its own. As the target population's sample grows, the model has more of its own material to directly learn that population's specific genetic characteristics — and at that point, information carried over from European-ancestry data can start to actively get in the way. It's already well known that genetic risk prediction models, most of which are built on large-scale genomic studies centered on European-ancestry populations, lose predictive power when applied directly to other populations. This result shows that even attempts to close that gap can flip from helpful to harmful depending on sample size.

Google Research has recently been putting out a string of studies applying computational methods to biological data, including a complete map of all 160,000 neurons in the male fruit fly brain.

Editor's take

What makes this result interesting is that it overturns, with data, the conventional assumption that transfer learning is always a good thing. Up to now, the problem of cross-ancestry gaps in genetic risk prediction has generally been approached on the premise that even European-ancestry data is better than nothing. This result shows that once a target population has accumulated a certain amount of its own data, that premise no longer holds. Similar patterns turn up often in other foundation model domains, too. When fine-tuning a large pretrained model on a small amount of domain-specific data, pretrained knowledge helps when domain data is scarce — but once domain data becomes plentiful, that same pretrained knowledge can end up dragging performance down. It seems the same principle is at work with genomic data.

There's a lesson here for research institutions and hospital data teams in Korea working with biobank data. Rather than sticking with an approach borrowed from another population's model even after your own sample grows, it's worth designing in advance the point at which you scale back reliance on transfer learning — or switch to your own model — once your own data reaches a certain size. Without setting thresholds for when to change strategy based on sample size, you risk ending up with the paradox of gathering more data while your predictive power actually gets worse.

As efforts to bring genetic risk prediction models into real-world clinical use keep growing, the question of exactly when to cut off transfer learning looks set to become a central part of the next round of discussion.