Cloud Hosting

Rolls out new AI-as-a-Service Solution

Inuvo, Inc., a leading provider of advertising technology, powered by artificial intelligence (AI) that serves brands and agencies, announced the self-serve availability of IntentKey models within demand-side platforms (DSPs) for advertisers.

This new Artificial Intelligence as a Service gives brand and agency clients direct access to IntentKeys powerful AI-driven audience selection and targeting recommendations. The frictionless access via Deal ID is customized to the marketing strategies associated with a clients products, services, or brands.

Recommended AI News: Box Expands its Collaboration with Microsoft with New Azure OpenAI Service Integration

Our self-serve solution puts control into the hands of our clients, allowing them to activate campaigns tailored to their brands using their programmatic campaign system of choice, said Rich Howe, Inuvo CEO. With a dedicated insights dashboard offering real-time visibility into the targeting rationale behind why audiences are interested, they can feel confident knowing IntentKey is maximizing performance.

Inuvo offers two service plans for clients accessing IntentKeys AI to inform their media buys. The first plan leverages the companys Managed Services team. The second plan, the new self-serve solution, empowers advertisers to independently activate IntentKeys proprietary models optimized for their unique brands and KPIs. After providing campaign details and goals, clients gain access to a Deal ID for PMP (private marketplace) activation across their preferred buying platforms.

Recommended AI News: Horizon3.ai Unveils Pentesting Services for Compliance Ahead of PCI DSS v4.0 Rollout

Unlike look-a-like models built on third-party cookies or stale offline data, IntentKeys self-serve solution uses real-time signals associated with online content to reach audiences as their interests change. This allows media to be dynamically placed across the open web. The IntentKey Insights Dashboard provides unmatched transparency revealing insights contributing to model optimization refreshed every five minutes.

With unparalleled transparency, independence, and customization, you simply cant compete with IntentKeys self-serve integration in todays privacy-centric landscape, added Howe. At a minimum, clients should immediately be setting up an IOS Deal ID so they can reach the Apple users their DSP is not currently capable of targeting.

If marketers can describe their audiences contextually, IntentKey can find and target them. Weve defined what the next generation of advertising technology looks like, concluded Howe.

Recommended AI News: OurCrowd AI Fund to Collaborate with NVIDIA Inception

[To share your insights with us as part of editorial or sponsored content, please write to sghosh@martechseries.com]

Original post:
Inuvo Puts the Power of its AI in the Hands of Agencies and Brands - AiThority

The TITAN ground system solution will provide multi-domain integrated data directly to the front lines

AZUSA, Calif. March 7, 2024 Northrop Grumman Corporation (NYSE: NOC) is partnering with Palantir USG, Inc. on the newly awarded Tactical Intelligence Targeting Access Node (TITAN) ground system for the U.S. Army. The program supports one of the Armys key modernization imperatives by using artificial intelligence (AI) and machine learning (ML) to enhance the automation of target recognition and geolocation and integrate data from multiple sensors to reduce sensor-to-shooter timelines.

Northrop Grumman will partner to:

The TITAN ground system will enable faster decision making on the frontlines by providing actionable intelligence to reduce sensor-to-shooter timelines and maximize effectiveness of long-range fires. (Photo Credit: Palantir)

Expert:

Aaron Dann, vice president, strategic force programs, Northrop Grumman: Northrop Grummans extensive experience in large-scale system integration will help enable mission success and provide information superiority for our warfighters in complex operating environments.Our work on TITAN continues our long history of supporting our nations need for actionable intelligence when and where it matters most.

Details:

TITAN is a ground system that has access to space, high altitude, aerial, and terrestrial sensors to provide actionable targeting information for enhanced mission command. TITAN will enable the Army to fuse, correlate, and integrate intelligence data from a rapidly expanding series of sensors providing operational forces a full picture of their surroundings. This robust capability allows real-time decision making that will substantially increase the accuracy, precision, and effects of long-range precision fires.

Northrop Grumman is a leading global aerospace and defense technology company. Our pioneering solutions equip our customers with the capabilities they need to connect and protect the world, and push the boundaries of human exploration across the universe. Driven by a shared purpose to solve our customers toughest problems, our employees define possible every day.

The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the U.S. Government.

Continued here:
Northrop Grumman Partners to Advance Deep Sensing for the US Army | Northrop Grumman - Northrop Grumman Newsroom

At opposite ends of Illinois, two leaders in emerging construction technologies are collaborating to demonstrate how AI, ML (machine learning) and CV (computer vision) can validate constructions digital-data mandate: Save time, save money, save lives.

For the fourteenth consecutive year, U.S. construction valuation has grown and is now at $1.98 trillion, noted Mani Golparvar, professor of civil engineering, computer science and technology entrepreneurship at the Grainger College of Engineering/University of Illinois. But, hes reported, 53% of construction projects are behind schedule, 66% are over budget and nearly all carry cost overruns.

Golparvar estimated that $280 billion in potential added value exists through improved coordination. As an industry, weve significantly improved the way we plan projects. But the way we plan the job, execute, monitor our execution and use whatever we monitor to update a planthese four problems all contribute to why projects are behind schedule and over budget. If youre looking to these issues, 80% are preventable: Project team members do not have visibility early enough to be able to come up with a remedy.

To that end, Golparvar is founder and chief strategy officer of Reconstruct, a Visual Command Center that uploads schedules, 2D and 3D models, and reality capture so users can track progress as well as coordination and communications problems. Reconstruct says its benefits include a 30% reduction in time reporting field progress, and 25%-40% in improved schedule management and proactive risk mitigation. Eighty-five companies and several universities now partner with it.

Burcin Kaplanoglu mirrored Golparvars remarks. Speaking from the Oracle Industry Lab he heads just outside Chicago, he noted that U.S. infrastructure is graded as a C- by the American Society of Civil Engineers. At the same time, he said, we [would] need to hire 500,000 people a year, on top of regular hiring, to meet demand.

Consequently, he added, we need to automate processes, gain efficiency and improve safety with technology. Talking to your data is going to have tremendous benefits for the construction engineer. Getting that response really quickly is going to change how we interact with technology.

Golparvar and Kaplanoglu recently joined forces around a Reality Mapping Experiment (see Constructing the Future sidebar, pg. 46) at the Oracle lab to explore what combination of tools, processes and teams can best conduct reality capture and mapping on-site, reducing time and cost and ensuring safety.

Thats of great value to an industry facing an ever-growing tech stack, as Jennifer Suerth, senior vice president of Pepper Construction, which built the lab, said in an Oracle TV video. Autonomy allows us to continue to get the work done but be efficient with the resources we have.

Golparvar and Kaplanoglu offered quick definitions of ML, CV and AI.

Machine learning allows computers to learn from data to improve and streamline processes. Machine learning is continuing to improve the construction industry, Kaplanoglu said. We need to figure out how to optimize and how to forecast better.

Computer vision is a form of AI, Golparvar said. It involves the process of analyzing pictures and videos and generating actionable insights from them, doing image processing and understanding the geometry of the scene. Recognizing objects by automatically analyzing them can track progress, detect and recognize anomalies, and coordinate and optimize operations.

To solve that coordination issue requires everyone to be on the same page in terms of what was there versus what should be there, Golparvar continued. In construction, weve really taken advantage of design information that we call BIM, building information models, 3D representation of design. What if you tie them against the schedule? At any snapshot in time, you can click on any two points, you can measure the length, area and volume. The picture shows you what was promised to be done; the picture in the background shows you what has been achieved on the job. And the delta between the twowhat needs to be coordinated in terms of quality, safety and progresscan be color-coded in red and green: what is on the schedule, what is behind.

Generative AI is the third leg of this digital triad. AI involves machines mimicking human cognitive functions, but theres more to that than large language models. GenAI is going to help us do project management and planning, Kaplanoglu said. AI can inform the entire construction cycle, from estimating preliminary costs, to tracking quality and productivity, to the more tactical tasks of ordering and payment.

Kaplanoglu ventured predictions for AI, ML and CV in construction:

Unified, intelligent clouds will unite software, platforms and infrastructure within end-to-end solutions, easing management and accelerating change.

Oracle is a major player in the world of project management and design, and in 2018 Kaplanoglu co-founded the Oracle Industry Lab to advance the companys innovation in construction and engineering. He explained the rationale. When youre running day-to-day operations, its very hard to stop what youre doing and try these new technologies. So, by creating a testbed where we bring the technology and they bring their problems, we are creating a neutral space to try and learn from it.

To realize this, writing a data strategy around machine learning was followed by building an ecosystem because our products do scheduling, safety, risk management, cost. We had almost 2,000 engagements in the first two years, he said. The pandemic only spurred a concentration on autonomy. All of a sudden, we did remote site monitoring, cameras and drone flights, route inspections. We could work with architects and designers remotely; they didnt have to come to the site. Everything was video.

The original sandbox gave way to a much larger, industry-benchmarking construction space for investigatingrealistic job site situations. It opened in April 2022on time, on budget. With analogs now operating in the United Kingdom and Australia, it has passed 13,000 engagements. I think construction really got used to digital imagery, 3D point clouds, using drones, capturing data, Kaplanoglu said.

Construction has some relatively unique characteristics, from a constantly changing environment to wind and weather exposure. Its like building a prototype each time, Kaplanoglu added. Data can bridge that. For example, Komatsu, the massive construction and mining equipment manufacturer, is using Nvidias Jetson edge AI platform to impart intelligence to trucks, excavators and the like. AI and machine learning are critical to providing real value to the construction space, a Nvidia spokesman toldInside Unmanned Systems.

In terms of project management, Kaplanoglu said, theres tons of opportunities when we want to use machine learning, computer vision and GenAI.

The need to test construction innovation, validation and measurement in real-world situations led to a collaboration between Oracles. Smart Construction Platform ecosystem and founding partner tenant Reconstructs visualization tools. The goal was to create guidelines and tool utilizations best suited for particular use cases, improving documentation of and guidelines for progress, quality and assessment of as-built conditions through a structures lifecycle.

The Report explains costs, time spent on different tools, and what kind of results you should be expecting from each, Golparvar explained about the 2023 document.

Pepper Construction and Clayco offered project expertise on the construction side, with leading drone company Skydio and 3D measurement company FARO Technologies providing technical expertise.

Eight different ways of capturing and mapping data were scanned across resolutions, speed, cost and deliverables. These included manual and autonomous drones, 360 and smartphone cameras, and various stationary and mobile LiDAR techniques. Capture data was processed into Oracles platform and then into Reconstruct, which could process them for a consistent viewing experience. Post-processing, photogrammetry and 4D simulations could be viewed for actionable and exportable results.

Construction is said to be the number one civil market for drones, which can take pictures quickly in hard-to-access areas while their sensors allow for obstacle avoidance. For the experiment, a fully autonomous Skydio drone used its Indoor Capture vision-based autonomous software over several iterations to, as Solutions Engineering Manager Colin Romberger put it on Oracle TV, get total coverageto have the best data to put into photogrammetry software for Reconstruction.

Kaplanoglu discussed gains from using autonomous drones. The time savings dropped significantly, because youre already preprogrammed. Human drone flights also improved with repetition. Theres still some autonomy, like it avoids obstacles and you can do flight plans.

But the biggest drop happens when its fully autonomous.

You always want to know about what youre trying to solve, what kind of tools you have and what kind of resources you can leverage to solve that problem, Golparvar said.

He and Kaplanoglu enumerated how AI and associated technologies can self-direct and break down planning and actions.

Site selection already involves machine learning, Kaplanoglu said, because you have certain parameters to make sure you comply with, like height restrictions, zoning. You can use machine learning to pick the optimal location.

Design can use machine learning to investigate bigger datasets and find helpful patterns. Golparvar: You want to capture the architects intent and transform it into a document that can be used as a base of design. GenAI can provide and reconcile design alternatives, incorporating everything from routing mechanical systems to complying with local codes. The way it works, Kaplanoglu added, you sketch things and then people try to visualize it and build models. Learning from them can offer significant time savings. Im talking about doing this in an hour versus doing it in months.

Construction has its own mini cycle. You define your scope, you hire engineers and architects, and now you need a contractor to build it Kaplanoglu said. You send an RFP for a contractor to build it.

RFPs, however, can be limited and schedulers may cut and paste. Kaplanoglu offered a solution: We can upload an RFP document to Oracle Cloud infrastructure. It reads the document and gives you a summary and then it asks you questions. Its going to show you an early-stage prototype, and it builds a schedule for you. Its going to be a good template for you to build your own schedule.

Now something that would have taken you three days is going to take you maybe two to three hours.

Operations can use computer vision. How many times does someone show up at their location and they dont even know what the specifications are or what theyre supposed to do, Kaplanoglu said. You can take a picture or video and then computer vision can tell you, Its this manufacturer, this model, or vision can process the video or image and say, Oh, theres rust in this corner.

Predictive maintenance can significantly reduce costs and downtime. Safety also can be empowered by AI. Many of the AI tools that we have are completely focusing on offering better awareness for our workers, Golparvar said.

Skydio has participated in the Oracle labs work, and Kaplanoglu offered an example of its drones adding value. We have a great relationship with Skydio. Instead of taking images that humans need to tag, locate and input, that data goes to our product, and then our Vision Services recognizes the rust and registers it to our work order system. But it doesnt just register; it tells you the lat[itude] and long[itude], the location, the severity of the damage, it helps you create a ticket.

Basically, we automate the whole thing.

Imprecise capture, incorrect inputting and lack of updating can undermine results. Still, the industry has significantly improved its use of cloud, Kaplanoglu said. The data is more accessible, you can put more line, compute is easier. There are a lot of benefits to it, and our industry really took that to heart.

Golparvar advocates a step-by-step process. We do a maturity assessment on the readiness of a company for adopting and adapting AI-driven products, and what we can do to make sure that these products fit into the existing workflows. Because people are resistant to change.

We also need to demonstrate the return on investment, at the project level and the individual levelquality, speed, time and money saved. So, the best strategy is to kind of introduce one next step to that project team so we can take them from where they are to that future where many of their steps will be fully automated, and make sure they see how AI is verifiable, to have that element of trust.

Kaplanoglu is bullish in terms of technology adoption. Machine learning, computer vision and GenAI are all going to have a big impact in the next three to five years.

Continued here:
Inside AI: Talking to the Data - Inside Unmanned Systems

MHC-Bench

The MHC-Bench dataset consists of 2,232,937 peptides of length 9 and 115 MHC alleles. All the MHC alleles in this dataset are human MHC molecules (i.e. Human Leukocyte Antigens or HLA). Out of the 115 MHC alleles, about half are HLA-B, a third are HLA-A and remaining HLA-C. The MHC-Bench dataset contains 3,464,013 peptide-MHC pairs previously unseen by predictors during training. It is worth noting that the peptides by themselves may have been seen in the training data paired with a different HLA allele. The peptide overlap between training data for investigated predictors and MHC-Bench is shown in Table1. A description of the construction and composition of the dataset is presented in Methods section.

We evaluated the performance of four MHC class I predictors on the MHC-Bench dataset MHCflurry, MHCfovea, NetMHCpan and TransPHLA. The choice of the four predictors was guided by their popularity and performance reported in the literature1,2,4,47. MHCflurry2.047 is an ensemble predictor of 10 neural networks for predicting presentation of a peptide. It supports 14,993 MHC class I alleles. MHCflurry provides three scores, namely Binding Affinity (BA), Processing score and Presentation Score (PS). PS is produced by a logistic regression model combining the binding affinity and processing scores. Processing score captures the antigen probability which combined with binding affinity substantially improves the performance of the predictor47.

NetMHCpan4.11, an ensemble of 50 neural network models, produces Elution Ligand (EL) and Binding Affinity (BA) score and we refer to these modes as NetMHCpan-EL and NetMHCpan-BA respectively. It utilizes NNAlign_MA48, an artificial neural network that predict both BA and EL scores. For both modes, peptides with rank 2% or less are considered as binders and 0.5% or less as strong binders. It supports 11,000 MHC class I alleles.

MHCfovea2, is an ensemble of multiple CNN models that takes MHC allele sequence and peptide sequence as input to predict binding probability. In MHCfovea, ScoreCAM2,49 is applied to identify important positions and corresponding residues in input peptides and MHC allele sequence. It provides the motifs for the first and last 4 positions of the peptide for each allele along with the motif for MHC sequence.

TransPHLA4, is a transformer architecture that predicts binding probability for an input peptide and MHC allele sequence. Using the attention scores, important residues for each position of a 9mer can be obtain to generate a peptide motif for a given allele.

We use Area Under ROC (AUROC) and Area Under Precision-Recall Curve (AUPRC) as benchmark or performance metrics. Since peptide binding is MHC allele specific, we calculated the scores for each allele separately. The benchmark metrics for each allele are reported in Supplementary Data1, 2. The average benchmark metrics for the MHC class I predictors are reported in Table2. We find that all four predictors are comparable in their average performances across all alleles, as seen in Fig.2c. While the differences between the scores among predictors is minimal, we see that for most alleles, performance of MHCflurry-PS is marginally higher. Figure2a displays number of times (or number of alleles) the predictor was top performer based on AUROC and AUPRC score. We provide additional benchmarking analysis of the predictors in theSupplementary material (See Supplemental Fig.1 and Supplementary Note2) which indicates that the performance of the predictors are comparable across various metrics. The predictors achieve a higher performance on the AUROC metric, 0.950.98, as opposed to AUPRC where it is in the range of 0.750.86. For most alleles, there are fewer binding peptide-MHC pairs in comparison to non-binding peptide-MHC pairs. This is evident in the distribution plot in Fig.2b where for most of the alleles, only 110% of the peptides are binding. The %binding peptide-MHC pairs per allele is provided in Supplementary Data3). In imbalanced scenarios with few positive labels, the AUROC metric can be overly optimistic50. In contrast, AUPRC is relatively unaffected by this imbalance in the dataset owing to its focus on true positives (as true negatives are not into consideration). These two metrics were selected because they are used for benchmarking in the original paper describing the predictors. Recent work by Carrington et al.51 provides a useful discussion on use of AUROC and AUPRC metrics while also introducing a new metric called DeepROC.

a The number of alleles for which predictors exhibited the highest performances based on AUROC and AUPRC scores. b The percentage of binders in the MHC-Bench dataset per allele. Each dot represents a measurement for one allele. c Distribution of AUROC and AUPRC scores for the MHC class I predictors.

Explanations can be classified as either Global or Local. Global explanations for a predictor are distribution of input feature values across all inputs in the dataset. This offers a consolidated perspective on how the model utilizes input features for a specific output label. In contrast, local (or instance-based) explanations, focus on a single input - output instance. Local explanations typically show attribution of input features used for the prediction outcome. In the context of MHC class I predictors, binding motifs are examples of global explanations while a vector of attribution values for individual peptide positions forms a local explanation. It is worth noting that our work focuses on post-hoc explanations, i.e., explanations for existing models that have been previously trained. Post-hoc explanations are widely applicable as they can be used over models whose internal structure is not known or is too complex to interpret. Existing MHC class I predictors, like MHCfovea, focus on global explanations by generating binding motifs for MHC alleles. There is limited work on local instance-based explanations for this problem. In the next two sections, we motivate the need for local explanations for MHC class I predictors and discuss the additional information it can provide over global explanations.

An explanation for a 9mer peptide in our instance-based approach is represented as a length-9 vector of attribution values, generated through LIME or SHAP. Each positions attribution value can be positive or negative, with a positive (or negative) value indicating that the residue at that position contributes positively (or negatively) to the prediction. Our MHCXAI framework facilitates the generation of explanations for any MHC class I predictor by simply substituting the predictor module while keeping the LIME and SHAP modules unchanged. Using the MHCXAI framework, we successfully generated LIME and SHAP explanation vectors for input peptides from the MHC-Bench dataset across all examined predictors.

In this study, our main emphasis is on providing explanations for input peptides rather than allele sequences, given that not all predictors can process allele input in the form of an amino acid sequence. For instance, NetMHCpan and MHCflurry accept allele inputs as HLA-A02:01 and HLA-A0201, respectively. However, it is important to highlight that our framework has the capability to generate explanations for allele sequences in addition to peptides. This feature can be particularly beneficial for predictors like TransPHLA, which accepts the allele as an input sequence. An illustration of an allele explanation is presented in Supplementary Fig.S2 of Supplementary Note3.

Figure3a illustrates LIME and SHAP explanations generated for all examined predictors for a specific peptide-MHC allele pair (LLVEVLREIHLA-A*02:01). To visualize the explanations, heatmaps are constructed using the attribution values, with lighter colors indicating positive contributions and darker colors indicating little or negative contributions to the binding class. The peptide LLVEVLREI is a binding peptide for the HLA-A*02:01 allele, correctly classified by all MHC class I predictors. However, it is worth noting that the explanations from SHAP and LIME exhibit slight differences. For example, in Fig.3a, both LIME and SHAP attribute high importance to peptide position P2, but SHAP also recognizes peptide position P9 as significant. P9 is typically considered important for binding based on existing literature52,53.

a Examples of SHAP and LIME explanations for all investigated MHC class I predictors for the LLVEVLREI--HLA-A*02:01 pair. To visualize the explanations, the attribution values of positions are used to create heatmaps. For each explanation, a lighter color indicates a positive contribution, while a darker color indicates a smaller or negative contribution to the positive class. SHAP explanations for all the predictors highlight peptide positions P2 and P9 as the most important for binding, while LIME explanations highlight only peptide position P2 as the most important. b LIME and SHAP explanations for NetMHCpan-EL and -BA for the peptide KVAQKQFQL binding to HLA-A*02:01. NetMHCpan-EL classifies KVAQKQFQL correctly, but NetMHCpan-BA does not. SHAP captures these differences in performance and produces different explanations for the two predictor modes. c LIME and SHAP explanations for MHCflurry-PS and -BA for the peptide RVMAPRALL binding to HLA-C*06:02. MHCflurry-PS classifies RVMAPRALL correctly, but MHCflurry-BA does not. Similar to the example in b, SHAP captures these differences in performance and produces different explanations for the two predictor modes. In both NetMHCpan and MHCflurry examples, LIME explanations are unable to indicate positions leading to the difference in prediction outcome.

In Fig.3b, the peptide KVAQKQFQL binds to the MHC allele HLA-A*02:01. However, within NetMHCpan, NetMHCpan-EL correctly predicts it as a binder, while NetMHCpan-BA classifies it as a non-binder. SHAP and LIME explanations were generated for both modes, revealing that SHAP can identify the features (or peptide positions) responsible for the predictions made by NetMHCpan-EL and -BA. For instance, at peptide position P1, the amino acid K positively contributes to the prediction outcome in NetMHCpan-EL, whereas it negatively contributes in NetMHCpan-BA. LIME, on the other hand, produces similar attribution values for both predictors and is unable to highlight the cause for the difference in prediction between the two NetMHCpan modes.

In Fig.3c, the peptide RVMAPRALL is a binder to the MHC allele HLA-C*06:02, classified as a binder by MHCflurry-PS but not by MHCflurry-BA. SHAP and LIME were employed to explore the difference in predictions. SHAP identified that for MHCflurry-PS, peptide positions P1 and P9 play an important role. For MHCflurry-BA, while peptide position P9 is crucial, P1 is not deemed important (refer to the heatmaps in Fig.3c). This distinction between the two explanations helps identify and understand the reasons for different predictions.

These examples demonstrate that XAI techniques can generate explanations for MHC class I predictors. However, explanations produced by distinct XAI techniques for the same predictor may not align. Consequently, we evaluate the validity and quality of LIME and SHAP explanations against XAI metrics.

The average time required to generate an explanation for a single instance is influenced more by the choice of the predictor than the XAI technique itself. For example, generating an explanation (either LIME or SHAP) for MHCfovea takes twice as long as generating a corresponding explanation for MHCflurry.

As stated earlier, global explanations for MHC class I predictors manifest as binding motifs. Determining the binding preference for an MHC allele involves examining the most frequently occurring amino acids at anchor positions, specifically positions 2 and 9 in a 9mer peptide, which are the primary sites responsible for binding to an MHC molecule52,53. This can be extended to other peptide positions, forming a binding motif for an MHC allele.

Biological binding motifs for MHC alleles are generated using experimentally validated strong binders54. With recent MHC class I predictors, binding motifs are derived from peptides predicted as strong binders for a particular allele. These peptides serve as the basis for generating position-specific scoring matrices (PSSM), which are then visually represented as binding motifs. This approach is applied to generate binding motifs for MHCflurry, NetMHCpan, and MHCfovea. By comparing these predictor-generated motifs against the biological motif, it becomes possible to assess whether the predictor has effectively learned the correct binding patterns for a given allele. In this study, we utilize binding motifs from the MHC Motif Atlas database55.

Global explanations may overlook deviations observed in specific inputs. Consider a binding peptide that diverges from the typical biological binding motif pattern. In cases where a predictor correctly classifies this peptide as a binder, it becomes valuable to examine the specific features used by the predictor for this classification. Understanding the input features employed by the predictor for a particular peptide requires a local explanation rather than a binding motif. Specialized patterns like these are difficult to infer with a binding motif.

We illustrate the necessity for local explanations to capture specialized patterns that deviate from biological binding motifs with specific examples. In Fig.4a, the motif for HLA-A*02:01 suggests a preference for amino acids L, I, and, M at anchor position P2, binding to the super hydrophobic B pocket of the MHC molecule. However, despite the unfavorable nature of water-soluble Glutamine (Q) for such a pocket, solved peptide-bound HLA molecule structures indicate that many peptides with Q do bind strongly to HLA-A*02:0156. An example is the peptide FQFICNLLL (see Fig.4a), correctly classified as a binder by the MHCflurry-PS predictor. We generated a local explanation for this peptide using SHAP (see Fig.4a). The highest attribution values were assigned to peptide positions P1, P2, and P9. While the high importance of peptide positions P2 and P9 aligns with their roles as anchor positions, the elevated attribution value for peptide position P1 is rationalized by its crucial role in stabilizing the bound structure, as observed in refs. 53,56,57. It is worth noting that the amino acid Q in position P2 does not appear in the biological binding motif (global explanation) prominently which fails to capture the specialized pattern in this instance.

For a and b, biological binding motifs for HLA-A*02:01 and HLA-A*24:03 are obtained from the MHC Motif Atlas55. For each MHC allele, there are four heatmaps, which are SHAP explanations generated for true positive, true negative, false positive, and false negative peptides predicted using MHCflurry-PS. The peptides in both a and b defy the reasoning for binding based on biological motifs. However, the SHAP explanations are able to highlight the cause behind unexpected outcomes. A lighter color in the explanation heatmap indicates a positive contribution, while a darker color indicates a smaller or negative contribution to the positive class.

In Fig.4a, the true negative instance HQKKNEISF also contains the amino acid Q in position P2, similar to the true positive instance discussed. However, in this case the local explanation shows low attribution values for all other peptide positions. This indicates lack of strong binding signal from those positions, explaining the negative classification.

Figure4b is another example of explanations generated for true positive, true negative, false positive and false negative predictions made by MHCflurry-PS for HLA-A*24:03. In this example, peptide conforming to binding motif is correctly classified as non-binder whereas peptide not conforming to binding motif is correctly classified as binder.

In summary, instance-based explanations are particularly useful in explaining scenarios where binder peptides do not conform to motifs, misclassifications, and understanding a peptide specific pattern used for prediction. Additionally, we show that global explanations can be created using instance-based SHAP and LIME explanations in Supplementary Fig.S3 (in Supplementary Note4) which can be useful for quick comparison across predictors.

To achieve trust in a predictor, the generated explanations for the predictions must be reliable. Assessing the reliability of explanations involves comparing them to ground truth about the input-output relationship, as suggested by prior work such as58,59,60. The ground truth in our case are the residues in the peptide that genuinely contribute to the binding. The contribution of residues (hotspots) can be estimated experimentally using Alanine-scanning mutagenesis61, a resource intensive45 technique. Computationally, this can be achieved using BAlaS45,46 which calculates the difference between the free-energy of binding of original bound complex and mutated bound complex where just one residue of ligand peptide is replaced with alanine. This difference in free-energy of binding is indicated as G and G4.184kJ/mol is considered hot or important residue for binding46. G4.184kJ/mol indicates alanine enhances binding relative to the original residue46. Any value between denotes neutral substitution46. As it is difficult to obtain ground truth for all peptides, we use this G as an independent way of highlighting important residues in the peptide.

First, we compile all the available PDB structures featuring bound peptide-MHC allele complexes as documented in the MHC Motif Atlas55. Subsequently, we refine the list to encompass bound peptides with a length of 9 and narrow down the selection to structures that were consistently classified as binders by all examined MHC class I predictors. The resulting list comprises 250 PDB structures, encompassing 40 distinct MHC alleles (as listed in Supplementary Data5).

We compared the G for these 250 peptide-MHC pairs with the LIME and SHAP explanations generated for all the predictors. The LIME and SHAP values can be positive or negative, similar to G, which indicates residue contribution to the prediction. For each of these 250 peptide-MHC pair we calculated Pearson correlation coefficients between LIME/SHAP explanations and G, for each of the investigated predictors.

In Fig.5a, consider the instance of the peptide ITDQVPFSV bound to HLA-A*02:01, which is correctly classified. BAlaS identifies peptide positions P1, P2, P7, and P9 as hot residues (with G4.184kJ/mol), which are highlighted in red within the peptide-MHC complex. The SHAP explanations, feature red arrows pointing to the positions identified as important by BAlaS. Generally, the models consistently prioritize these positions when making predictions. However, despite having a high G, peptide position P7 is not deemed important by most of the predictors. This suggests that the information from the other three residues is sufficient for the predictors to infer the classification outcome. The distribution of correlation coefficients between SHAP-G and LIME-G (depicted in Fig.5b) indicates a positive correlation between the explanations and the important positions identified by BAlaS. Overall, it is observed that SHAP explanations exhibit a closer correlation compared to LIME explanations.

It is done by comparing the attribution values to the difference in free-energy of binding between wild-type protein-protein interaction and mutated protein-protein interaction, known as G. a For the ITDQVPFSV--HLA-A*02:01 complex, BAlaS highlights that peptide positions P1, P2, P7, and P9 are crucial for binding. Replacing the residues at these positions with alanine leads to an increase in G, indicating instability. This peptide is correctly classified by all the investigated MHC class I predictors, and SHAP explanations are generated for each of them. The explanations mostly match the ground truth, as P1, P2, P7, and P9 (indicated by red arrows) are rightly highlighted as the factors influencing the prediction. b G was calculated for each peptide position in 250 PDB structures containing peptide-MHC allele bound complexes that were correctly classified by all the investigated predictors. The SHAP and LIME explanations correlated positively for most complexes, indicating that the explanations mostly align with the ground truth and can be trusted. The correlation coefficient values are reported in Supplementary Data4. A lighter color in the explanation heatmap indicates a positive contribution, while a darker color indicates a smaller or negative contribution to the positive class.

The observed variance in the distribution of correlation coefficients is not surprising, given that BAlaS G serves as only an approximation of the actual positions involved in binding, and the approach is subject to certain limitations. Notably, the accuracy of the G calculation is influenced by the resolution of the PDB structure (refer to Supplementary Fig.S4 and Supplementary Note5). To address this, we selectively choose PDB structures with the highest resolution when multiple structures are available. Additionally, since G is computed by substituting a residue with alanine, it is challenging to ascertain the contribution of alanine, if present (refer to Supplementary Fig.S5).

Consistency refers to similarity in the explanations produced for two similarly performing predictors on a given input. We assess consistency of an XAI technique by comparing explanations for a given peptide between two similarly performing MHC class I predictors (Fig.1c).

To select two predictors with comparable performance, we choose the top two predictors from our results in Section 2 (see Fig.2a), namely MHCflurry-PS and MHCfovea. Additionally, the AUROC scores for these two predictors exhibit a high correlation, as indicated in Fig.6c, demonstrating substantial similarity in their performance.

a For an input peptide, explanations were generated for MHCflurry-PS and MHCfovea using SHAP and LIME. Pearson correlation coefficients were calculated between these explanations, and the process was repeated for 200 input peptides for each of the alleles presented in the plot. The distribution of Pearson correlation coefficients is closer to one, indicating high similarity between the two explanations for the two predictors on the same input. The correlation coefficient values are reported in Supplementary Data6. b In addition to the Pearson correlation coefficient, Euclidean distances were calculated between two explanations for two predictors on the same input. For Euclidean distance, values closer to zero indicate high similarity and high consistency. c Correlation heatmap for AUROC scores between investigated MHC class I predictors. MHCflurry-PS and MHCfovea are highly correlated in their performances.

We selected 9 alleles (3 each from HLA-A, B and C) and for each allele, we randomly selected 200 peptides from our MHC-Bench dataset to generate local explanations, independently using each of SHAP and LIME, for MHCfovea and MHCflurry-PS. For both LIME and SHAP, to compare the similarity between the explanations from the two predictors, we computed Pearson correlation and Euclidean distance.

In Fig.6a, the distribution of Pearson correlations between explanations generated individually for MHCflurry-PS and MHCfovea using LIME and SHAP is presented for all nine alleles. Overall, the majority of the SHAP and LIME explanations exhibited high correlation. In Fig.6b, the distribution of Euclidean distances between the explanations of the two predictors is presented. Explanations that are similar will have a Euclidean distance closer to zero. It is noteworthy that the Euclidean distance distribution for LIME has a narrow range and tends to be closer to zero compared to SHAP. This observation suggests that LIME produces more consistent explanations compared to SHAP.

We also created a baseline distance between explanations for the two predictors using the following procedure. First, we generated 100 random explanations for each original MHCfovea explanation by randomly permuting the attribution values. Next, we calculated the distance between each of these 100 random explanations and the original MHCflurry-PS explanation. The baseline distance was then computed by averaging these 100 distances. This process was repeated for all 200 peptides chosen per allele. Consequently, we obtained 200 Euclidean distances between the original MHCflurry-PS and MHCfovea explanations, along with their corresponding baseline distances. We compared these two distributions for each allele. We confirmed that the two distributions for both SHAP and LIME were statistically different using Kruskal-Wallis test at 5% significance level. The p-value, H-statistics and effect size are reported in Supplementary Tables1 and 2 for SHAP and LIME respectively. It is also worth noting that the Euclidean distance for both LIME and SHAP explanations were smaller than the corresponding average baseline Euclidean distance for nearly all the input peptides (99% input peptides).

We confirmed that the two distributions for both SHAP and LIME were statistically different using Kruskal-Wallis test at 5% significance level. The p-value, H-statistics, and effect size are reported in Supplementary Tables1 and 2 for SHAP and LIME, respectively. Additionally, it is noteworthy that the Euclidean distances for both LIME and SHAP explanations were smaller than the corresponding average baseline Euclidean distances for nearly all input peptides (99% of input peptides).

Stability of an explanation technique refers to the extent to which explanations for similar inputs (with same output) over a given predictor are close. We use the MHCflurry-PS predictor to asses stability of the LIME and SHAP techniques, independently. To identify input peptides that are similar, we perform clustering over a subset of peptide sequences for HLA-A*02:01. Using GibbsCluster-2.062,63, we cluster the peptides into 110 clusters. The number of clusters that yields highest average KullbackLeibler Distance (KLD) is considered to be the optimum number of clusters. We found choosing 10 clusters has the highest KLD with cluster size ranging between 7001000 peptides. The plot showing KLD distribution and cluster motifs generated from GibbCluster is provided in Supplementary Fig.S6. Peptides within a cluster are considered similar.

From each of these clusters, we sampled 100 peptides that are binders and generated explanations for these peptides. We calculated the Euclidean distance between all pairs of peptides within each cluster, and this is referred to as the intracluster distance distribution. As a comparison, we also computed the distance between explanations for peptides from different clusters, referred to as Intercluster distance. We show results for the top six most unrelated cluster pairs (c2, c5), (c3, c5), (c3, c8), (c5, c6), (c5, c9), (c5, c10), based on the similarity of their position-specific scoring matrix, in Fig.7.

a Euclidean distance distribution for the top six cluster pairs using LIME. For each pair, there are three distributions - IntraclusterL, Intercluster, and IntraclusterR. For any two pairs (e.g., c2, c5), the intracluster explanation distance distribution for the left cluster (c2) and right cluster (c5) are IntraclusterL and IntraclusterR, while Intercluster is the distribution of explanation distances between the two clusters. b Euclidean distance distribution for the top six cluster pairs using SHAP. For each pair, there are three distributions - IntraclusterL, Intercluster, and IntraclusterR. For any two pairs (e.g., c2, c5), the intracluster explanation distance distribution for the left cluster (c2) and right cluster (c5) are IntraclusterL and IntraclusterR, while Intercluster is the distribution of explanation distances between the two clusters.

For each cluster pair in Fig.7, we have three distributions: IntraclusterL, Intercluster, and IntraclusterR. Consider the pair (c2, c5), where IntraclusterL represents the intracluster Euclidean distance distribution for cluster c2 (or the left cluster), Intercluster is the intercluster Euclidean distance distribution between c2 and c5, and IntraclusterR is the intracluster Euclidean distance distribution for cluster c5 (or the right cluster). The notation of left-right for Intracluster is arbitrary. It is worth noting that intracluster distances are lower than the intercluster distances, indicating that LIME and SHAP explanations for peptides within the same cluster are more similar, suggesting stability of explanations. We confirmed that the differences between intracluster and intercluster distance distributions in Fig.7 are statistically significant (KruskalWallis test). The p-value, H-statistics, and effect size are reported in Supplementary Table3.

Read more:
Building trust in deep learning-based immune response predictors with interpretable explanations | Communications ... - Nature.com

Key Takeaways:

Researchers from Carnegie Mellon University and Los Alamos National Laboratory have used machine learning to create a model that can simulate reactive processes in a diverse set of organic materials and conditions.

Subscribe to our free Lab Manager Monitor newsletter.

"It's a tool that can be used to investigate more reactions in this field," said Shuhao Zhang, a graduate student in Carnegie Mellon University'sDepartment of Chemistry. "We can offer a full simulation of the reaction mechanisms."

Zhang is the first author on the paper that explains the creation and results of this new machine learning model, "Exploring the Frontiers of Chemistry with a General Reactive Machine Learning Potential," which was published in Nature Chemistry on March 7.

Though researchers have simulated reactions before, previous methods had multiple problems. Reactive force field models are relatively common, but they usually require training for specific reaction types. Traditional models that use quantum mechanics, where chemical reactions are simulated based on underlying physics, can be applied to any materials and molecules, but these models require supercomputers to be used.

This new general machine learning interatomic potential (ANI-1xnr) can perform simulations for arbitrary materials containing the elements carbon, hydrogen, nitrogen, and oxygen and requires significantly less computing power and time than traditional quantum mechanics models. According to Olexandr Isayev, associate professor of chemistry at Carnegie Mellon and head of the lab where the model was developed, this breakthrough is due to developments in machine learning.

"Machine learning is emerging as a powerful approach to construct various forms of transferable atomistic potentials utilizing regression algorithms. The overall goal of this project is to develop a machine learning method capable of predicting reaction energetics and rates for chemical processes with high accuracy, but with a very low computational cost," Isayev said. "We have shown that those machine learning models can be trained at high levels of quantum mechanics theory and can successfully predict energies and forces with quantum mechanics accuracy and an increase in speed of as much as 6-7 orders of magnitude. This is a new paradigm in reactive simulations."

Researchers tested ANI-1xnr on different chemical problems, including comparing biofuel additives and tracking methane combustion. They even recreated the Miller experiment, a famous chemical experiment meant to demonstrate how life originated on Earth. Using this experiment, they found that the ANI-1xnr model produced accurate results in condensed-phase systems.

Zhang said that the model could potentially be used for other areas in chemistry with further training.

"We found out it can be potentially used to simulate biochemical processes like enzymatic reactions," Zhang said. "We didn't design it to be used in such a way, but after modification it may be used for that purpose.

In the future, the team plans to refine ANI-1xnr and allow it to work with more elements and in more chemical areas, and they will try to increase the scale of the reactions it can process. This could allow it to be used in multiple fields where designing new chemical reactions could be relevant, such as drug discovery.

- This press release was originally published on the Carnegie Mellon University website

Originally posted here:
Advancing Chemistry with AI: New Model for Simulating Diverse Organic Reactions - Lab Manager Magazine