Contradiction between RFD3 design and AF3 prediction

[nmbiss747-netizen]

I used RFD3's hotspot-guided binder generation, but in the subsequent AlphaFold structure prediction, the binder did not appear near the expected binding site. Has anyone else encountered the same problem? I'm wondering if this means the RFD3 design is flawed, or if there's an issue with the AlphaFold prediction results.

[roccomoretti]

I'll mention that while RFD3 now does produce sequences, in practice it's observed they tend to be lower quality than dedicated design models like ProteinMPNN and LigandMPNN. If you haven't already, you may want to add a redesign step with one of those to your pipeline.

It's also observed that not all RFD/MPNN generated models successfully forward fold with AlphaFold. Forward folding actually is one of the main filtering steps used to pick designs. Generally you'll generate thousands of binder designs, and throw out the (substantial) fraction of designs which don't properly forward fold. If you've only tested a few designs with AlphaFold, you just may have gotten unlucky with which designs you obtained.

On the question of whether it's RFD3 or AlphaFold which has issues, one approach might be to use other prediction models, such as Boltz or Chai or RosettaFold. If they all dislike the structure, then it's probably RFD3 that is generating poor structures. If only AlphaFold thinks it bad, it might be that you've hit upon a weak spot of AlphaFold prediction. (This latter is probably unlikely. The most likely default assumption would be that RFD3 is generating backbones with issues.)

[nmbiss747-netizen]

I also considered the issue of the sequences generated by RFD3 itself, but I tried regenerating the sequences for the binder using ProteinMPNN, and the results were no different. It seems that AlphaFold believes the binder will inevitably bind at a specific location, regardless of which binding site I use in the RFD3 design stage. I tried designing the binder using the tail of the IgG Fc domain and the CH2-CH3 junction as hotspots, but AlphaFold's predictions almost always placed the binder in the central cavity of the Fc dimer.By the way, I also tried other structure prediction models, and the results were similar to AlphaFold's. Placing the binder in the central cavity of the Fc dimer, which is a clear pocket, does make sense, even though this wasn't the binding site I specified during the design stage. Does this mean that RFD, even with clear hotspot constraints, still cannot generate a binder specific to that particular site?

[roccomoretti]

There is some memorization bias in ML structure prediction models. While we hope that they generalize to new systems, there's evidence that the models have varying levels of bias toward things they've seen in the training set. That is, if there's a crystalized homolog of your system with a binder at a specific location (particularly if there's consistently a binder at that location for a large number homologs), the ML models will have a bias toward those structures. Basically, they go "oh, this is an XYZ protein! I know where XYZ binding proteins bind!" and stick the partner there, regardless of whether that's where this protein actually binds. That doesn't always happen, but there's indications that it may be happening more than it should.

One approach is to treat it as a negative design problem. You have two states: the desired binding site and the alternate site AF3 finds. Examine the two states and come up with hypotheses about what you can alter to favor the desired state versus the alternate state. (e.g. putting in extra bumps or loops which might clash with the alternate state. Or perhaps changing the charge or hydrogen bonding pattern). If you impose these sorts of negative design constraints onto your designs, you might be able to push the alternative state into an unfavorable enough position for the ML structure predictors to no longer think the alternate state is a good one.

Another option, if you think the ML structure predictors are biased, is to use a more physics-based approach to examine the two states. You can use something like (classic) Rosetta or MD simulation or some other protein-protein docking program to examine the energy of interaction between the binders in both the designed state and the alternate, AF3-predicted state. If you resample/redock the two states, and the physics-based docking program thinks the designed state is much better energy than the alternate state, that leads further credence to the concept that the ML prediction is a spurious one. This might give you enough confidence to order designs.

Because ultimately, that's what matters -- what the design does in the test tube/cell. AlphaFold refolding is simply a method for increasing the likely success rate of the designs. If the ML refolding is uninformative for your system (e.g. because it consistently places designs in spot which is spurious), then you can safely ignore it and rely on alternate metrics to pick designs which are likely to be successful.