Redesigning error control in cross-linking mass spectrometry enables more robust and sensitive protein-protein interaction studies

In order to evaluate the sensitivity and specificity of different data grouping schemes used for FDR filtering, we made use of our recently developed XL-MS benchmarking standard. This dataset consists of 256 purified proteins that have been mixed and cross-linked according to a defined scheme (Clasen et al, 2024) prescribing which inter-protein connections will occur and which ones cannot occur (Fig. 1A). Following database search, we can accordingly assign cross-link (that is, unique residue pair) identifications as true or false, and assess the proportion of false-positive and false-negative identifications at any given target-decoy based FDR cutoff (Fig. 1B,C).

We initially tested the two most established strategies in XL-MS: a concatenated target-decoy FDR filtering strategy considering intra-links and inter-links either as one combined (intra-inter combined) or two separate (intra-inter separate) groups (Fig. 1D). In accordance with previous reports (Walzthoeni et al, 2012; Lenz et al, 2021), we find that the combined strategy fails to appropriately control error among inter-links, leading to many false-positive identifications (Fig. 1E). While this problem can be adequately controlled by the intra-inter separate approach, this strategy results in a considerable loss of sensitivity signified by an increase in false-negative identifications (Fig. 1F). For example, approximately half of the available true matches remain unidentified at 1% FDR. These shortcomings of the traditional data grouping schemes in XL-MS highlight the need for a workflow that improves sensitivity toward inter-links while retaining accurate error estimates.

Previous work in shotgun proteomics found that peptide identifications can be increased by subgrouping strategies (Marx et al, 2013; Cox and Mann, 2008; Fu and Qian, 2014). This concept is based on the assumption that different subsets of matches have different error probabilities. Adapted to XL-MS, subgroups may be defined based on various properties of the detected inter-linked peptides, including peptide length, charge state, missed cleavages or the presence of other cross-links supporting the inter-link. We tested the latter possibility by separating inter-links depending on the extent to which other cross-links support the existence of the protein or PPI (context-rich vs context-poor inter-links). We evaluated two definitions of context-richness for their suitability as subgrouping criteria: the first depends on the presence of contextual intra-links (“intra-dependent”) and has been proposed previously in a similar manner (Chen et al, 2022; Sailer et al, 2022), the second depends on the presence of contextual inter-links (“inter-dependent”) and has been devised in this study (Fig. 2A). In both subgrouping procedures, inter-links are separated in the two subgroups prior to FDR analysis, which is then performed separately on both subsets before aggregating them into a combined list (see “Methods”).

We tested intra-dependent and inter-dependent subgrouping combined with concatenated target-decoy searches on our XL-MS benchmarking data (Fig. 2B,C). Compared to the established intra-inter separate strategy, our inter-dependent subgrouping yields approximately the same fraction of false positives while substantially lowering the fraction of false negatives. Thus, the inter-dependent strategy appears to be an effective way to increase identification sensitivity without compromising specificity. An increase in sensitivity (i.e., reduction of false negatives) is also observed upon intra-dependent subgrouping (Fig. 2C). However, intra-dependent subgrouping leads to a massive increase in false positives, suggesting that error rate cannot be effectively controlled when using this subgrouping approach together with concatenated target-decoy databases.

To assess whether these observations hold true in a real-world scenario, we analyzed a deep XL-MS dataset from HEK293T cells that was recently generated in our lab (Clasen et al, 2024). This dataset was created by cross-linking with the enrichable cross-linker DSBSO (Wheat et al, 2021; Matzinger et al, 2020) (“HEK293T”), which yielded 12,216 target and 2003 decoy inter-links when no FDR cutoff was applied. When intra-dependent or inter-dependent subgrouping was applied, we observed divergent score distributions between context-rich and context-poor matches (Fig. EV1A,B), which was more pronounced when inter-dependent grouping was applied. This reinforces our initial assumption of different error probabilities between context-poor and context-rich subgroups and suggests that inter-link identification in the HEK293T dataset can benefit from context-sensitive subgrouping.

We first separated inter-links according to the grouping criteria and assessed the overall fraction of targets and decoys prior to imposing any FDR cutoff (Fig. 3A). We observed that both inter-link and intra-link subgrouping strategies result in very few decoys among context-rich subgroups. For inter-dependent subgrouping, this observation agrees with the results from our ground-truth dataset. However, for intra-dependent subgrouping, the complete lack of decoys among the context-rich inter-links is in stark contrast to the explosion of false positives we observed in the benchmarking dataset. Following the herein-applied FDR filtering strategy using target-decoy concatenated database, more than 10,000 inter-links could be added to the identification without accepting any false match. The inconsistency to our benchmarking dataset indicates that combining intra-dependent subgrouping and concatenated target-decoy searches might fail to recognize the false positives among the context-rich inter-links.

Since the true number of false positives in the HEK293T data cannot be known, we simulated the theoretically expected distribution of true and false positives in the context-rich and context-poor subgroups based on a set of simple assumptions and controlled parameters (see “Methods”). We assume that wrong matches are randomly assigned to any protein in the database, while true matches have the tendency to frequently match to a restricted set of proteins. Following the placement of correct and incorrect matches on the proteins, we grouped context-poor and context-rich inter-links and evaluated the fraction of correct and incorrect matches in these subgroups. This is advantageous, as a simulation allows direct counting of the number of correct and incorrect matches (Fig. EV2). For intra-dependent subgrouping, the simulation gives ~1300 false positives representing ~11.9% of the identifications in the context-rich subgroup (Fig. 3B), which is in stark contrast to the experimentally observed numbers of decoys in this subgroup (see Fig. 3A). Interestingly, the simulated distribution of false positives for the inter-dependent strategy closely resembles the distribution of decoys. Altogether, this suggests that the error rate can be properly controlled for inter-dependent subgrouping, but not for intra-dependent subgrouping.

The problem with intra-dependent subgrouping is that a cross-link group with a low error rate (intra-links) is used to categorize a cross-link group with a substantially higher error rate (inter-links). This becomes intuitively apparent when considering how intra-dependent filtering is impacted by having high numbers of intra-links in settings of restricted database size: When more and more intra-links are identified, many (if not most) proteins in a database will eventually contain an intra-link. Randomly assigned false-positive inter-links will then more frequently contain proteins supported by these intra-links (Fig. 3C). This issue is avoided in inter-dependent subgrouping because this strategy categorizes inter-links based on information from the same cross-link group. Inter-dependent subgrouping builds on the fact that inter-links belonging to PPIs, which are additionally targeted by different inter-links are very unlikely to occur by chance.

Based on the explosion of false-positive identifications in the benchmarking and simulated datasets, we reason that the robustness of target-decoy concatenation strategies can be compromised whenever higher-level information (e.g., proteins supported by intra-links) is used to classify lower-level matches (e.g., specific inter-links). Integration of different information levels can also occur in conventional bottom-up proteomics and has been addressed by the introduction of fused target-decoy databases (Savitski et al, 2015; Zhang et al, 2012). In this setup, each decoy entry is fused to its respective target entry, creating one fused protein sequence (Fig. 3D). In an ungrouped setting, the distribution of target and decoy hits is not different when using a concatenated or fused decoy database. However, upon context-sensitive subgrouping, fused databases ensure that once a target protein is assigned to a context-rich subgroup its corresponding decoy part is automatically assigned to the same subgroup.

We hypothesized that a fused decoy strategy will also be beneficial for context-sensitive subgrouping in XL-MS. Repeating our HEK293T dataset analysis with this strategy resulted in a realistic decoy count for all subgrouping strategies including the intra-dependent approach (Fig. 3E). Similarly, reanalysis of our benchmarking data shows that target-decoy fusion can reduce the false-positive identifications for intra-dependent subgrouping to similar levels as those observed for inter-dependent subgrouping and the inter-intra-separate strategy (Fig. 4A). Comparing all strategies, we find that inter-dependent subgrouping is best suited for minimizing false positives (Fig. 4A) and false negatives (Fig. 4B).

Next, we sought to explore the benefits of context-sensitive subgrouping and fused target-decoy searches when studying biologically relevant interactomes. To this end, we analyzed interactome coverage in intact human cells using the HEK293T dataset described above, intact human mitochondria (“mito”) (Zhu et al, 2024), and intact herpesviral particles (“virion”) (Bogdanow et al, 2023). We first used the simulation approach described above to confirm the utility of target-decoy fusion databases and context-sensitive subgrouping also for the mito and virion datasets (Fig. EV3A–G). In all cases, only target-decoy fusion accurately reflects simulated error rates. Context-rich subgroups, particularly the inter-dependent subgroup, display low error rates in all systems, which also holds true at the level of cross-link spectrum matches (Fig. EV3H–J) The inter-dependent context-rich subgroup further contained overall relatively few unique proteins, suggesting that the majority of the cross-links within this subgroup match to a restricted set of confident proteins (Fig. EV4A–C). As discussed above, this can be explained by the low probability of two different random (i.e., false) inter-links coincidentally matching to the same two protein sequences, which becomes exceedingly rare as the search space expands.

To assess intra-dependent and inter-dependent subgrouping for their potential to increase sensitivity (and thereby interactome coverage), we counted the number of identified inter-links that could be assigned at any FDR. We compared these results to those obtained when grouping all inter-links together (intra-inter-separate). In all tested datasets, we observed a pronounced increase in statistical power when inter-dependent grouping is applied (Fig. 4C and EV5A,B). The improvement was smaller (for “mito” and “virion” datasets) or non-existent (for “HEK293T”) upon intra-dependent grouping. In all cases, inter-dependent grouping performed best, particularly for stringent FDR cutoffs (29–76% increase in sensitivity at FDR = 1%) (Fig. 4D). These data show that context-sensitive subgrouping on a fused decoy database is a universally applicable strategy to increase sensitivity while maintaining accurate FDR estimates, whereby inter-dependent subgrouping increases sensitivity more robustly and effectively than intra-dependent subgrouping.

Finally, we assessed the utility of our optimized inter-link identification strategy for structural biology. In the HEK293T dataset, compared to the standard inter-intra separate strategy, our inter-dependent filtering strategy increased inter-link counts for 38% of PPIs (Fig. 5A). We then mapped inter-links from both strategies onto proteome-wide predictions of complexes by AF2 multimer (Jumper et al, 2021). While both strategies resulted in a similar fraction of inter-links satisfying the computational model, we find that inter-dependent grouping increased the number of inter-links in models of good (pDockQ >0.5), acceptable (0.23 <pDockQ <0.5) and poor (pDockQ <0.23) quality (Fig. 5B). Focusing only on models of acceptable and good docking quality, we observed particularly robust increases of coverage in regions within the models that are likely disordered (pLDDT <50) (Fig. 5C,D), indicating that increased coverage is particularly prominent for regions of higher structural complexity. Overall, compared to the standard approach, inter-dependent subgrouping resulted in a similar distribution of Cɑ-Cɑ distances in good, moderately and poorly satisfied models (Fig. 5E). The newly identified inter-links provide additional supporting structural information for AF2-multimer models, such as the SMC2-SMC4 dimer (Fig. 5F, 10 additional cross-links) and large cryoEM structures, such as the proteasome (Fig. 5G,H, 17 additional cross-links). Thus, inter-dependent subgrouping increases the coverage of PPI contact sites, providing additional cross-linking data to support computational and experimental structural models across the proteome.

We reanalyzed raw files (.raw) from Clasen et al (Clasen et al, 2024), containing HCD-MS2 spectra of the HEK293T and ground-truth datasets (ProteomeXchange identifier PXD042173). First, we converted into peak lists (.mgf files) in Proteome Discoverer v2.5 (Thermo Fisher). The.mgf files were used as input to identify cross-linked peptides with a stand-alone search engine based on XlinkX v2.0 (Liu et al, 2017). The following settings of XlinkX were used: MS ion mass tolerance, 10 parts per million (ppm); MS2 ion mass tolerance, 20 ppm; fixed modification, Cys carbamidomethylation; variable modification, Met oxidation; enzymatic digestion, trypsin; and allowed the number of missed cleavages, 3; DSSO cross-linker, 158.0038 Da (short arm, 54.0106 Da; long arm, 85.9824 Da), reaction site: lysine, protein N-termini.

The search was performed using a database containing all 256 mixed, 264 additional entrapment, and 520 reversed decoy proteins. All cross-links matching to a database of 123 potential contaminant entries were removed prior to data analysis. True and false were assigned at the peptide level to account for the partial homology of proteins in different groups. True matches were assigned, when in accordance to the mixing scheme and false matches when in violation. It is important to note that, in principle, true matches may occur randomly when a cross-link matches two proteins within the same group. In this case, a true match is falsely assigned. However, the frequency of this is negligible as within group protein count (n = 8) is far lower than that of outgroup proteins (n = 512).

Raw files from HEK293T dataset (dowloaded from ProteomeXchange, identifier: PXD042173) containing HCD-MS2 data were converted into peak lists (.mgf) and searched as described above with DSBSO specificity (DSBSO cross-linker, 308.0388276 Da, short arm, 54.01056 Da, long arm, 236.01770 Da, reaction site: lysine, protein N-termini) with MS2 spectra searched against a concatenated target-decoy databases (randomized decoys) generated based on the corresponding proteome determined by bottom-up proteomics, containing 4860 target sequence entries. The table giving targets and decoy residue-level cross-links at no FDR threshold was used for subgrouping, FDR calculations and input for simulations. Cross-links refer to unique residue pairs, and the top scoring cross-link spectrum (CSM) match was considered as a representative for the residue pair.

For the mitochondrial (DSSO) dataset (Zhu et al, 2024) and the virion (DSSO) dataset (Bogdanow et al, 2023), the XlinkX search output available from the PRIDE deposition was used directly (under ProteomeXchange identifiers PXD032132, PXD031911).

In order to evaluate the performance of a fused decoy database search strategy, we analyzed our datasets as if they were searched using a fused database. To this end, we performed the database search in a concatenated design but then replaced the entries containing the gene names for the cross-linked proteins with their target annotations in the cross-link results table. For example, a cross-link matching the proteins “RPL12-RPS6(decoy)” was re-annotated as “RPL12-RPS6”. Thus, all targets and decoys originating from the same .fasta entry are fused into one entry. Importantly, we kept an additional identifier indicating whether the cross-link originated from the decoy or target part of the fused entry.

We devised only two subgroups for intra- and inter-dependent filtering to assure subgroups are sufficiently large to enable target-decoy competition (Fu and Qian, 2014) and to avoid overfitting to a specific dataset. In the case of intra-dependent subgrouping, we devised two subgroups: One context-rich group containing all inter-links where both the cross-linked proteins are additionally supported by intra-links and a second, context-poor, subgroup containing inter-links where at most one of the proteins contained intra-links. In the case of inter-dependent grouping, we also devised two subgroups. A context-rich subgroup containing the inter-links where each of the proteins in a PPI is supported by at least two inter-linked lysines involving different reactive sites on both ends and a second, context-poor, subgroup containing all other inter-link matches. Groupings were performed at the level of residue–residue pairs, and all residue–residue identifications without FDR control were used for subgrouping. Following subgrouping, we used previously established approaches for calculating posterior error probabilities (PEP) to estimate the probability distributions for correct and incorrect matches within individual subgroups (Cox and Mann, 2008). Briefly, we make two lists per group. One list containing the matches to the decoy part, and one list matching the target part. Then, we generated a histogram for each by gaussian kernel smoothing of the score distributions. We here used the lower (worse) negative decadic logarithmic XlinkX search scores reported for both sides of the residue pair as the overall score for the residue pair. The resulting distributions correspond to approximations to the probability densities for group-dependent correct and incorrect hits:where X signifies a match to a correct or incorrect hit, s is the negative decadic logarithmic XlinkX score and G the placement of the identification into a context-rich or context-poor subgroup.

According to Bayes theorem, we calculated the PEP for each residue pair as given in Eq. (1).

Here, p(s | X = false, G) describes the probability of observing a given score s, considering an incorrect match in subgroup G. p(X = false | G) is the probability of an incorrect match in the subgroup G, calculated as the fraction of decoys in the subgroup, and p(s,G) is the probability of observing the score s in subgroup G, considering both correct and incorrect matches.

The FDR was calculated by combining the PEP values from the different groups and the combined PEPs are sorted starting with the best (lowest). Then residue pairs are successively accepted until a desired FDR cutoff is reached according to Eq. (2)where T denotes a target match, and D is a decoy match on either side of a cross-link, respectively.

The standard inter-intra separate strategy does not require PEP calculation since it does not depend on subgrouping. Instead, all inter-links are assigned to the same group, which was sorted by decreasing search scores. Again, we used the lower (worse) negative decadic logarithmic XlinkX search scores reported for both sides of the residue pair as the overall score for the residue pair. The false-discovery rate for each entry was then calculated based on Eq. (2).

Two simulations were performed to estimate the proportion of false positives when employing two different subgrouping strategies (intra- and inter-dependent). In both cases, we simulated a distribution of false and correct cross-links based on parameters as obtained from the datasets (HEK293T, mito and virion). For intra-dependent subgrouping, we first simulated a distribution of correct matches. This was achieved by calculating the number of cross-links per target protein within the respective dataset, followed by fitting a Zipfian (Furusawa and Kaneko, 2003) distribution through the ranked target cross-link counts per protein. A Zipfian power-law distribution was used to model the correct cross-links, reflecting the nature of real-world protein interaction networks. To simulate false matches, we calculated the number of cross-links per decoy protein and fitted a Poisson distribution through the ranked decoy cross-link counts per protein. Using the Poisson distribution assumes that wrong matches occur randomly, without considering the possibility of strong systematic biases towards specific proteins, such as through misidentified modified peptides (Bogdanow et al, 2016). The fit parameters of both models were utilized to create probability distributions for true and false matches. Then we placed correct intra-links (corresponding in number to the target intra-link count), correct inter-links (corresponding in number to the target inter-link count) and incorrect inter-links (corresponding in number to the decoy inter-link count) according to the protein-dependent match probabilities for correct and false matches onto all proteins considered in the database search. Subsequently, the fraction of incorrect inter-links was evaluated in the subgroups of inter-links where (i) both linked proteins contained at least one intra-link and (ii) all others.

For simulating the frequency of false matches upon inter-dependent subgrouping, the simulation was adapted as follows: (i) intra-links were not considered, and (ii) inter-link matches were split into two parts, reflecting the possibility that each of the two cross-linked peptides could be wrongly matched (target-target, decoy-target, decoy-decoy). Again, a Poisson distribution was fitted to the empirically observed counts of decoy matches for all cross-links per PPI involving the most frequently linked decoy protein. To simulate correct matches, a Zipfian distribution was fitted for all cross-links per PPI involving the most frequently linked target protein. Incorrect and correct matches were then deposited on both sides of the inter-link according to the resulting probabilities for all possible entries in the database (as described in the previous paragraph). The frequency of incorrect matches was evaluated in the subgroups of inter-links where (i) both of the linked proteins contained at least one additional inter-link between the same proteins and (ii) all others.

Hetero-dimers were predicted by AF2 multimer v2.2 (Evans et al, 2022) using the default protocol and sequence search methods. For each model, the pDockQ score was calculated (Bryant et al, 2022). Different model qualities were considered based on the pDockQ score. Mapping of cross-links on AF2-Multimer predictions was performed using the bio3d R package. Therefore, we extracted the C-alpha atom coordinates of inter-linked Lysines in the predicted zero-rank dimeric structure and calculated their distance in three-dimensional space.