This section presents the experimental results and comprehensive evaluations of BayesCovid. We will explicitly discuss the results of the algorithms applied to the clinical datasets to uncover the hidden patterns of COVID-19 symptoms.

We set up a Spark on Hadoop Yarn cluster consisting of 4 EC2 machines, 1 master and 3 workers in AWS to deploy BayesCovid. We chose Ubuntu Server 20.04 LTS as the operating system for all the machines and installed Hadoop version 3.3.2 and Spark 3.3.1. All the nodes have 4 cores and 16 GB of memory.

The dataset, prepared by Carbon Health and Braid Health14, was obtained through RT-PCR tests from 11,169 individuals, approximately 3% of patients living in the United States who had COVID-positive, and 97% had COVID-negative tests. This dataset, which began to be collected by Carbon Health in early April 2020, was collected under the anonymity standard of the Health Insurance Portability and Accountability Act (HIPAA) privacy rule. This dataset covers multiple physiognomies, including Epidemiological (Epi) Factors, comorbidity, vital signs, healthcare worker-identified, patient-reported, and COVID-19 symptoms. In addition, information about patients, such as heart rate, temperature, diabetes, cancer, asthma, smoking and age, is also available. The Carbon Health team gathered the Braid Health team datasets, which collected radiological information, including CXR information. This dataset includes data from patients with one or more symptoms and no symptoms, and we only used the COVID-19 symptom information indicated in Fig. 1. Radiological information was not included in the analysis. Table 2 shows the statistical information of the COVID-19 dataset. We have 18,538 test results of 11 different COVID-19 symptoms and COVID severity values, belonging to 11,169 individuals. Moreover, Table 3 demonstrates the number of false (negative) and true (positive) values for each symptom.

Cross-validation is an important step in assessing the predictive power of models while mitigating the risk of overfitting15. To rigorously evaluate our models, we implemented ten-fold cross-validation by dividing the dataset into ten equal parts. During each iteration, one part was the validation/test set, while the remaining nine were used for model training. This process was repeated ten times, and the resulting accuracies were averaged across all folds to assess each models performance comprehensively. Importantly, using ten-fold cross-validation ensures that every instance in the dataset is precisely used once as a testing and training sample, which minimises the risk of overfitting16.

This subsection explains three distinct Bayesian networks: Nave Bayesian, Tree-Augmented Nave Bayesian, and Complex Bayesian models. These models have unveiled intricate and concealed patterns within COVID-19, offering valuable insights into the complex dynamics and relationships underlying the disease.

Figure 3a depicts the dependencies for the Nave Bayesian algorithm where the class variable, COVID severity, is the only parent associated with each symptom, and there is no link between symptoms. Figure 4 and 5 show the probability percentages of the symptoms for their positive and negative values. For example, in Fig. 4, while the probability of diarrhea is around 3% for COVID severity level 1, the probability of this symptom for level 3 is about 95%. Moreover, the probabilities of shortness of breath for levels 1, 2, 3, and 4 are very low, about 5%, and the likelihood of having this symptom is very high for levels 5 and 6. In short, the distribution of symptoms differs according to the severity levels of COVID-19, and the probability of some increases as the COVID-19 severity level rises. When we compare Figs. 4 and 5, it is seen that there is an inverse relationship between the incidence and absence of symptoms.

Conditional probability of symptoms with COVID-19 severity if symptoms are positive for Nave Bayes.

Conditional probability of symptoms with COVID-19 severity if symptoms are negative for Nave Bayes.

The dependency network built using the Tree-augmented Nave Bayesian network is depicted in Fig. 3b. COVID severity is the class variable similar to Nave Bayesian network, but the connections between the symptoms (features) are also available. As seen from the figure, for example, cough has an effect on both headache and fever while muscle sore is affected by headache and affects fatigue. For the probabilities, Tables 4 and 5 show some results of the Conditional Probability Table (CPT). In Table 4, when shortness of breath and fever are negative (F), the probability of COVID severity level 1 is 92.95%. In contrast, when shortness of breath is positive (T), and fever is negative, then the probability of COVID severity level 1 is 2.04%. When headache is positive, but cough is negative, then the probability of COVID severity level 4 is 65.92% (see Table 5).

Figure 3c shows the dependencies between all the symptoms (features) and COVID severity (class variable). Cough is most affected by different symptoms and does not affect any features. While the class variable, COVID severity, has impacts on shortness of breath, fever, fatigue, and sore throat, interestingly, it is affected by diarrhea. Another interesting pattern different from the Tree-augmented Nave Bayesian network is that fever affects muscle sore. While Table 6 shows the CPT for three variables, namely COVID severity, shortness of breath, and fever, Table 7 displays the probabilities for four variables, namely diarrhea, fatigue, muscle sore, and headache. When shortness of breath is negative, and fever is positive, the probability of COVID severity level 4 is 98.92%. In contrast, when shortness of breath is positive, but fever is negative, the probability of COVID severity level 6 is 48.18% (see Table 6). For the probabilities based on the situation of four symptoms (see Table 6), for instance, when all three symptoms, diarrhea, fatigue, and muscle sore, are positive, the probability of having headache symptom is 73.18%. Another remarkable finding in Table 7 is that if an individual has fatigue, muscle sore, and headache, the probability of not having diarrhea is 58.43%.

In this study, we have also investigated and implemented three distinct Bayesian models, each representing a unique intersection of deep learning and Bayesian inference. The first model, Deep Learning-based Nave Bayes (DL-NB), is a deep learning-based Nave Bayes structure that capitalises on the capacity of deep neural networks to refine the traditional Nave Bayes model for enhanced feature learning and dependency representation. Additionally, we extended our exploration to traditional Bayesian network structures by implementing Deep Learning-based Tree-Augmented Nave Bayes (DL-TAN), where deep learning principles are integrated to augment the classic Tree-Augmented Nave Bayes algorithm, providing richer feature representations. Furthermore, our investigation includes Deep Learning-based Complex Bayesian (DL-CB), a model designed to overcome the limitations of traditional Complex Bayesian structures in modelling intricate relationships within high-dimensional data. This comprehensive analysis and implementation of DL-NB, DL-TAN, and DL-CB contribute to the broader understanding of the synergies between deep learning and Bayesian techniques in various Bayesian network architectures. Figure 6 demonstrates the network dependencies of deep learning-based Bayesian network algorithms which uncover the complex and hidden relationships between COVID symptoms. As illustrated in Fig. 6ac, our Bayesian deep learning models, namely DL-NB, DL-TAN, and DL-CB, reveal a richer web of relationships among features compared to their traditional counterparts. The Bayesian Deep Learning models exhibit a higher density of connections, which indicates a more nuanced understanding of inter-feature dependencies. This heightened connectivity means the enhanced capacity of Bayesian Deep Learning to capture complex relationships within the data that provide a comprehensive and informative modelling of the underlying dynamics.

Bayesian deep learning dependency networks.

Figure 7 demonstrates the accuracys for the three different algorithms proposed in our system, namely Nave Bayesian Network, Tree-Augmented Nave Bayesian Network, and Complex Bayesian Network. Although the general accuracy of the algorithms is close to each other, there are apparent differences in the accuracy of the symptoms. The algorithms perform between 60% and 68% poorly for cough symptoms, while they show high accuracys for COVID severity ranging from 94% to 97%. The overall accuracys of these three algorithms are 83.52%, 87.35%, and 85.15%, respectively.

Total accuracys of the algorithms.

In the evaluation of the accuracy of deep learning-based Bayesian network algorithms, the results, as depicted in Fig. 8, showcase the performance of three distinct models: DL-NB, DL-TAN, and DL-CB. The overall accuracies reveal nuanced differences among the algorithms. DL-TAN emerges with the highest cumulative accuracy of 95.21%, which indicates its superior predictive capabilities across a spectrum of symptoms. DL-NB and DL-CB follow closely, exhibiting overall accuracies of 91.04% and 92.81%, respectively. These results underscore the efficacy of deep learning-based Bayesian approaches in capturing complex relationships within the dataset.

The comparative analysis of Bayesian deep learning algorithms against traditional Bayesian network algorithms elucidates a discernible advantage favouring the former. Notably, the Bayesian deep learning models, such as DL-NB, DL-TAN, and DL-CB, exhibit superior predictive performance across various symptoms.

Total accuracys of the deep learning-based Bayesian algorithms.

We have developed a web interface for BayesCovid decision support system that can be used by any clinical practitioner or other users. It utilises Python libraries, concerning probabilistic graphical models, data manipulation, network analysis, and data visualization. Additionally, tkinter is adopted for the graphical user interface, and PyMuPDF (fitz) is leveraged for PDF file handling. All the source code and accompanying documentation for BayesCovid decision support system are available as open-source on GitHub (https://github.com/umitdemirbaga/BayesCovid). A demonstration is also available online on YouTube (https://youtu.be/7j36HuC9Zto). The designed user interface provides dual functionality highlighted below.

Dependency analysis: This component of application ensures efficient and accurate relationship analysis between the symptoms and severity assessment, enhancing the decision-making process in clinical settings. Figure 9a depicts the user-friendly interface where a data file can be uploaded using Select CSV button. After the data file is uploaded, six radio buttons are provided for users to select one of the following Bayesian models: (a) Nave Bayesian Network, (b) Tree-Augmented Nave Bayesian Network, (c) Complex Bayesian Network, (d) Nave Bayes Deep Learning, (e) Tree-Augmented Bayes Deep Learning, and (e) Complex Bayes Deep Learning. An Analyse button that starts the processing of the selected model with the selected CSV file. A progress bar populates to show the processing status. After the model is processed, the dependency network plot is generated (see Fig. 9b) and the CPT output is saved as a file.

Severity analysis: This component of the application assists clinical staff in calculating the severity of COVID-19. This feature assists in selecting the detected symptoms that the patient exhibits and subsequently determines the severity of COVID-19. As depicted in Fig. 9c a clinician or user can select the visible symptoms and calculate severity. This will output the COVID-19 severity level based on the input symptoms as shown Fig. 9c.

See the rest here:
Uncovering hidden and complex relations of pandemic dynamics using an AI driven system | Scientific Reports - Nature.com

Overview of RMSF-net procedure

We propose a deep learning approach named RMSF-net to analyze protein dynamics based on cryo-electron microscopy (cryo-EM) maps. The primary objective of this method is to predict the RMSF of local structures (residue, atoms) within proteins. RMSF is a widely used measure to assess the flexibility of molecular structures in MD analysis and defined by the following equation:

$${RMSF}=sqrt{frac{1}{T}{sum }_{t=1}^{T}{left(xleft(tright)-widetilde{x}right)}^{2}}$$

where(x) represents the real-time position of atoms or residues, (t) represents time and (widetilde{x}) represents the mean position over a period of time ({T}). In addition to the experimental cryo-EM maps, RMSF-net incorporates fitted PDB models, which represent the mean structures of fluctuating proteins. A schematic overview of RMSF-net is depicted in Fig.1a. The cryo-EM map and PDB model are initially combined to create a dual feature pair. The PDB models are converted into voxelized density maps using the MOLMAP tool in UCSF Chimera23 to facilitate seamless integration with cryo-EM maps. Subsequently, both the density grids of the cryo-EM maps and the PDB simulated maps are divided into uniform-sized density boxes (404040) with a stride of 10. The corresponding density boxes from the mapping pair are concatenated to form a two-channel feature input for the neural network (RMSF-net) to predict the RMSF of atoms within the central subbox (101010). RMSF-net is a three-dimensional convolutional neural network comprising two interconnected modules. The primary module employs a Unet++(L3) architecture24 for feature encoding and decoding on the input density boxes. The other module utilizes 1-kernel convolutions for regression on the channels of the feature map generated by the Unet++ backbone. A center crop is then applied to the regression module output to obtain the central RMSF subboxes, where the voxel values correspond to the RMSF of the atoms contained within them. Finally, the RMSF sub-boxes are spatially merged into an RMSF map using a merging algorithm.

a Overview of RMSF-net. The data preparation and RMSF inference for RMSF-net are illustrated in the upper section. Cryo-EM maps and their fitted atomic structure models were obtained from the EMDB and PDB databases. The PDB models were simulated as density maps resembling cryo-EM maps. Both the cryo-EM map and PDB simulated map were then segmented into 40-cubic density boxes. The density boxes with matching positions from the pair were concatenated into a two-channel tensor and input to the 3D CNN of RMSF-net to infer the RMSF for atoms in the central 103 voxels (subboxes). The RMSF prediction across the entire map was obtained by combining predictions from these subboxes. The lower section depicts the RMSF-net supervised training process. The RMSF-net neural network architecture is shown in the lower right, with the number of channels indicated alongside the hidden feature maps. With Unet++ (L3) as the backbone, a regression head and crop operation were added. The ground truth RMSF for training RMSF-net was derived from MD simulations, as illustrated in the lower left. b Data processing of maps in RMSF-net.

RMSF-net incorporates several data processing strategies for maps, as illustrated in Fig.1b. First, to ensure a consistent spatial scale, all the cryo-EM maps were resampled using the ndimage.zoom module from SciPy25 to obtain a uniform voxel size of 1.5, which is approximately the size of a C atom. Second, a screening algorithm is applied to retain only those boxes that encompass atoms within the central subbox to avoid processing unnecessary boxes located in structure-free regions. This strategy significantly improves the efficiency of RMSF-net when dealing with large cryo-EM maps. The retained box indices are recorded for the subsequent merging algorithm. In addition, the voxel densities within the boxes are normalized to a range of [0, 1] before being input into the network, thus mitigating density distribution variations across cryo-EM maps. Voxel density normalization is achieved through the following process: within each box, any density values less than 0 are set to 0, and then divided by the maximum density value within the box, thus scaling the voxel density to a range from 0 to 1.

We created a high-quality protein dataset with 335 entries for training and evaluating RMSF-net. The dataset was constructed by selecting data from EMDB26 and PDB27. As of November 2022, EMDB contained over 23,593 deposited entries, with more than half being high-resolution maps with resolutions ranging from 24. We focused on maps within this range. Initially, we included a high-resolution cryo-EM map dataset from EMNUSS12 which underwent rigorous screening in EMDB and PDB prior to October 2020 and consisted of 468 entries. In addition, we performed data selection on deposited cryo-EM maps and PDB models from October 2020 to November 2022 to incorporate newly deposited data. The selected data had to meet specific criteria, including a well-fitting cryo-EM map and PDB model with a fitness above 0.7 (measured by the correlation between the PDB simulated map and cryo-EM map); the proteins had to contain at least one alpha helix or beta-strand, with no missing chains or nucleic acids. We further filtered these data by applying a clustering procedure to remove redundancy. Using the K-Medoids28 algorithm with a k value of 50, we defined the distance between two proteins as the maximum distance between any two chains from each protein, where chain distances were determined by sequence identity. After clustering, we selected the 50 medoids and added them to the dataset. Finally, out of the remaining 518 entries, 335 were successfully subjected to MD simulations, resulting in the RMSF-net dataset.

RMSF-net employs a supervised training approach, requiring labeled RMSF values derived from MD simulations19. We conducted MD simulations on the PDB models of the dataset following a standardized procedure using Assisted Model Building with Energy Refinement (AMBER)20, which consists of four stages: energy minimization, heating, equilibration, and production runs. To focus on local structure fluctuations around specific protein conformations, we configured the production run for 30 nanoseconds. Specifically, the initial atomic coordinates of the proteins were set to the original PDB model coordinates. Small molecule ligands in all complexes were removed to purely study the characteristics of proteins. Each system was immersed in a truncated octahedron box filled with TIP3P29 water molecules (at least a 12 buffer distance between the solute and edge of the periodic box). Based on the charge carried by the protein, Na+ or Cl ions were placed randomly in the simulation box to keep each system neutral. An additional 150mM NaCl solution was added to all systems according to the screening layer tally by the container average potential method30 to match the experimental conditions better. All MD simulations were performed using the AMBER 20 software package20,31 on NVIDIA Tesla A100 graphics cards. The parameters for Na+ and Cl- ions were derived from the previous work by Joung et al.32. The parameters used for the protein structure were AMBER ff14SB force field33. Each system was energy minimized using the conjugate gradient method for 6000 steps. Then, the systems were heated using the Langevin thermostat34 from 0 to 300K in 400ps using position restraints with a force constant of 1000kcal mol1 2 to the protein structure (NVT ensemble, T=300K). Subsequently, each system was gradually released in 5ns (spending 1ns each with position restraints of 1000, 100, 10, 1, and 0kcal mol1 2) using the NPT ensemble (P=1bar, T=300K) before a production run. Afterward, the final structure of each system was subjected to a 30ns MD simulation at constant temperature (300K) and pressure (1bar) with periodic boundary conditions and the particle mesh Ewald (PME) method35 We used the isotropic Berendsen barostat36 with a time constant of 2ps to control constant pressure. The protein structure was completely free in the solutions during the equilibration and production process. Simulations were run with an integration step of 2fs, and bond lengths for hydrogen atoms were fixed using the SHAKE algorithm37. PME electrostatics were calculated with an Ewald radius of 10, and the cutoff distance was also set to 10 for the van der Waals potential.

After simulation, the trajectories were processed and analyzed using the built-in Cpptraj module of AMBER Tools package38. We first removed the translational and rotational motion of all protein molecules to ensure a rigorous comparison between different trajectories. Then, the average structure of each protein (only heavy atoms) was calculated as a reference structure. Afterward, each conformation in the trajectory was aligned to the reference structure and RMSF of the protein molecule was output. These computed RMSF values were subsequently mapped onto voxels of cryo-EM maps to serve as the ground truth for training and evaluating RMSF-net.

For the training of RMSF-net, we utilized a masked mean squared error (MSE) loss function to compute the loss between the predicted RMSF and ground truth RMSF on labeled voxels of the output subboxes. The training spanned 100 epochs with a batch size of 32, and we employed the Adam optimizer39 with a learning rate of 0.004. Several techniques were implemented to mitigate overfitting, including Kaiming weight initialization40, learning rate decay, and early stopping. If the validation loss did not decrease for 10 consecutive epochs, the learning rate was halved. If it did not decrease for 30 epochs, training was terminated, and the model with the minimum validation loss was saved. We applied rotation and mirroring augmentation to the training set to account for the lack of rotational and mirror symmetry in convolutional networks, increasing the training data eightfold. The training of RMSF-net was conducted on two NVIDIA Tesla A100 graphics cards, typically lasting 58h. Following training, we conducted RMSF predictions on the test set to evaluate the performance of RMSF-net.

We employed a five-fold cross-validation approach to assess the performance of this method. The dataset was randomly divided into five equal partitions, with one partition used as the test set each time, and the remaining four partitions served as the training and validation sets. In particular, the division was based on the maps rather than the segmented boxes in order to ensure independence between these sets. The training and testing process was repeated five times, and every data entry was tested once to obtain the methods performance on the entire dataset. To prevent overfitting during model training, the training and validation sets were set at a ratio of 3:1. During testing, the correlation coefficients between the predicted RMSF and the ground truth (RMSF values derived from MD simulations) were computed as the evaluation metric. The correlation coefficients were computed at two levels: voxel level, corresponding to RMSF on the map voxels, and residue level, corresponding to RMSF on the PDB model residues (obtained by averaging RMSF on the corresponding atoms). We defaulted to using the correlation coefficient at the voxel level when analyzing and comparing model performance unless otherwise specified. In addition, we employed the correlation coefficient at the residue level when discussing the protein test cases.

As a baseline, we initially used only the cryo-EM map intensity as the single-channel input to the neural network, referred to as RMSF-net_cryo. Cross-validation using RMSF-net_cryo on the dataset revealed an average correlation coefficient of 0.649 and a bias of 0.156. We also performed cross-validation using the prior method DEFMap method for comparison. DEFMap reported a test correlation of approximately 0.7 on its dataset. However, its dataset includes only 34 proteins and the dataset used in our study is more diverse and significantly larger. Therefore, we applied the DEFMap pipeline to our dataset to ensure fair comparisons. Notably, DEFMap employed different data preprocessing strategies and neural networks. During its preprocessing, a low-pass filter was adopted to standardize the resolution of the cryo-EM maps. In addition, the neural network it used took 10-cubic subvoxels as input and outputted the RMSF of the central voxel. We strictly followed DEFMaps procedures and network for training and testing. The results showed an average correlation coefficient of 0.6 and a bias of 0.171. Through comparison, it is evident that RMSF-net_cryo exhibits superior performance compared to DEFMap.

Although RMSF-net_cryo performed better than DEFMap with our designed network and data processing strategies, it still relies on neural networks to directly establish patterns between cryo-EM maps and flexibility. What role the structural information plays in this process remains unknown. This prompted us to divide dynamic prediction via cryo-EM maps into two sequential steps: first, structural information extraction, and second, dynamic prediction based on the extracted structural information.

To accomplish the extraction of structural information, as depicted in Fig.2a, we introduced an Occ-net module. This module predicts the probabilities of structural occupancy on cryo-EM map voxels using a 3D convolutional network. Both input and output dimensions were set to 403. For training and evaluating Occ-net, we utilized PDB models to generate structure annotation maps as the ground truth, where voxels were categorized into two classes: occupied by structure and unoccupied by structure. The details of this network and data annotation process are provided in the Supplementary Information (section Structure of Occ-net and the data annotation process). Cross-entropy loss function was employed during training, with class weights set to 0.05:0.95 to address class imbalance. Once this training stage was completed, Occ-net parameters were fixed, and the second stage of training commenced. In the second stage, the two-channel classification probabilities output by Occ-net were input into the dynamics extraction module to predict the RMSF for the central 103 voxels, which is consistent with the RMSF-net approach.

a Overview of Occ2RMSF-net. In the first stage, the cryo-EM density (403) is input into Occ-net to predict the probabilities (403) of structure occupancy on the voxels, with Pu denoting the probabilities of voxels being occupied by the protein structure and Po denoting that of not being occupied by the structure. Then in the second stage, the two-channel probabilities are input into RMSF-net to predict the RMSF on the center 103 voxels. b Test performance of Occ2net. For six classification thresholds from 0.3 to 0.8, the precisions, recalls and F1-scores of the positive class (structure occupied) on the test set were computed and are shown in the plot. c Comparision of RMSF prediction performance between Occ2RMSF-net and RMSF-net_cryo on the dataset. CC is an abbreviation for correlation coefficient. d Count distribution of test correlation coefficients for DEFMap, RMSF-net_cryo, and RMSF-net on the dataset. e Data distribution of correlation coefficients for RMSF-net_cryo and RMSF-net_pdb relative to RMSF-net on the dataset. f Count distribution of test correlation coefficients for RMSF-net_pdb, RMSF-net_pdb01, and RMSF-net on the dataset. g Data distribution of correlation coefficients for RMSF-net and RMSF-net_pdb relative to RMSF-net_cryo on data points where the test correlation coefficients with RMSF-net_cryo are above 0.4. The color for each method in d, e, f and g is shown in the legend.

This model is referred to as Occ2RMSF-net, and cross-validation was conducted on it. After training, we first assessed the performance of Occ-net by calculating the precision, recall, and F1-score at the voxel level for the positive class (structure class) on the test set. This evaluation involves six classification thresholds, ranging from 0.3 to 0.8. As depicted in Fig.2b, achieving high precision and recall simultaneously was challenging due to severe class imbalance and noise. A relative balance was achieved at the threshold of 0.7, where the F1 score reached its highest value of 0.581. Regarding the final output RMSF, the correlation between the Occ2RMSF-net predictions and the ground truth on the dataset is 0.6620.158, showing a slight improvement compared to RMSF-net cryo. Figure2c displays the scatter plot of the test correlation of data for Occ2RMSF-net and RMSF-net cryo. The two models exhibited similar performance on most of the data points, with Occ2RMSF-net slightly outperforming RMSF-net_cryo overall. This highlights the critical role of structure information from cryo-EM maps for predicting the RMSF in the network and enhances the interpretability of methods like DEFMap and RMSF-net_cryo.

Inspired by the above results, we incorporated PDB models representing precise structural information and integrated them into our method in a density map-like manner, i.e., simulated density maps generated based on PDB models. We employed two approaches to input the PDB simulated maps into the network. First, the PDB simulated map was taken as a single-channel feature input into the neural network, referred to as RMSF-net_pdb. Second, the PDB simulated map was transformed into a binary encoding map representing occupancy of the structure to highlight tertiary structural features: a threshold of 3 ( represents r.m.s.d of the PDB simulated map density) was chosen, where voxels with densities above the threshold were encoded as 1 and the others as 0. This encoding map was then converted into a two-channel one-hot input to the network, known as RMSF-net_pdb01. The same cross-validation was applied to these two models. Results showed that RMSF-net_pdb achieved a test correlation coefficient of 0.7230.117, and RMSF-net_pdb01 achieved 0.7120.112. These two approaches demonstrated significantly better performance than the above cryo-EM map-based methods, further demonstrating the strong correlation between protein structure topology and flexibility.

We further combined the information from the PDB structure and cryo-EM map and input them into the network, which is the main method proposed in this work. We refer to this method, along with the neural network it employs, simply as RMSF-net. As outlined in the Overview of RMSF-net procedure section, RMSF-net takes the dual-channel feature of density from the cryo-EM map and PDB simulated map at the same spatial position as input, while the main part of the network remains the same as RMSF-net_cryo and RMSF-net_pdb. Conducting the same cross-validation on RMSF-net, the results revealed an average correlation coefficient of 0.746, with a median of 0.767 and a standard deviation of 0.127.

RMSF-net demonstrated an approximate 10% improvement compared to the baseline of RMSF-net_cryo, and a 15% enhancement over DEFMap. Figure2d presents a comparison of the distribution of data quantities at different test correlation levels for the three methods. Overall, the two cryo-EM map-based methods (DEFMap and RMSF- net_cryo) exhibit similar distribution shapes, while the distribution of RMSF-net is more concentrated, focusing on the range between 0.6 and 0.9. Nearly half of the data points cluster around 0.7 to 0.8, and close to one-third fall between 0.8 and 0.9. In comparison, the two PDB-based methods in Fig.2f exhibit similar distributions with RMSF-net. This suggests that the structure information from PDB models plays a primary role in the ability of RMSF-net to predict flexibility. On the other hand, RMSF-net further outforms the PDB-based methods through combination with information from cryo-EM maps, indicating image features related to structural flexibility in the cryo-EM map make an effective auxiliary. Regarding the robustness of these approaches, Table1 demonstrates that RMSF-net_pdb and RMSF-net_pdb01 exhibited less deviation on the test set compared to RMSF-net, while RMSF-net_cryo displayed the highest deviation. This indicates the flexibility-related information in cryo-EM maps is unstable compared to that in PDB models, which might be caused by noise and alignment errors in cryo-EM maps.

The experimental results above prove that the combination of cryo-EM map and PDB model results in the superior performance of RMSF-net. As shown in Fig. 2e, the prediction of RMSF-net is better in most cases, comparing models utilizing only the cryo-EM map or only the PDB model. Because the PDB models are built from the corresponding cryo-EM maps, their spatial coordinates are naturally aligned, and their structural information is consistent. Moreover, the PDB model built from the cryo-EM map corresponds precisely to the average position of the structure, and the cryo-EM map reconstructed from multiple particles in the sample corresponds to the information of multiple instantaneous conformations. By combining the expectation and conformational variance from the two sources, we believe that this structural consistency and complementarity create an alignment effect, and promote the superior performance of RMSF-net. However, structural deviations may exist between the PDB model and the cryo-EM map in some instances, or the PDB model may only partially occupy the cryo-EM map. These anomalies might lead to subpar performance of RMSF-net_cryo compared to RMSF-net and RMSF-net_pdb. To exclude the influence of these factors, we performed dataset filtering by excluding data points with test correlations below 0.4 for RMSF-net_cryo and compared the three models on the filtered dataset. Figure2g shows that RMSF-net and RMSF-net_pdb still demonstrated better performance overall compared to RMSF-net_cryo on the filtered dataset. The test correlations for the three models were 0.7600.084, 0.7330.083, and 0.6840.1, respectively. When the filtering threshold was increased to 0.5, the correlations for the three models were 0.7610.08, 0.7340.08, and 0.6980.084, respectively, showing consistent results.

Figure3a showcases RMSF-net predictions for three relatively small proteins: the bluetongue virus membrane-penetration protein VP5 (EMD-6240/PDB 3J9E)41, African swine fever virus major capsid protein p72 (EMD-0776/PDB 6KU9)42 and C-terminal truncated human Pannexin1 (EMD-0975/PDB 6LTN)43. Among these, 3J9E displays an irregular shape composed of loops and alpha helices, while 6KU9 and 6LTN exhibit good structural symmetry with beta sheets and alpha helices, respectively. The predictions by RMSF-net exhibit strong agreement with the ground truth for these proteins, yielding correlation coefficients of 0.887, 0.731, and 0.757, respectively, as depicted in Fig.3b. Predictions by RMSF-net_cryo and RMSF-net_pdb are supplied in Supplementary Figs.S1S3. On 3J9E and 6KU9, both RMSF-net_cryo and RMSF- net_pdb perform well, achieving correlations of 0.82, 0.69, and 0.881, 0.7, respectively. However, on 6LTN, RMSF- net_cryo only exhibits a correlation of 0.3 with the ground truth, possibly due to ring noise in the intermediate region of EMD-0975, leading to model misjudgment. In contrast, RMSF-net_pdb achieves a higher correlation of 0.767 on this protein, even surpassing RMSF-net, suggesting that instability factors in cryo-EM maps have a slight impact on RMSF-nets inference.

a, b Show RMSF-net performance on three small proteins (EMD-6240/3J9E, EMD-0776/6KU9, EMD-0975/6LTN). a Visual comparison of RMSF-net predictions and ground truth. The first column shows the cryo-EM map overlaid on the PDB model. The second and third columns depict the PDB model colored according to the normalized RMSF values, using colors indicated by the color bar on the right. The second column represents the RMSF predictions by RMSF-net, and the third column represents the ground truth RMSF values from MD simulations. b Correlation plots between normalized RMSF-net predicted values and normalized ground truth values at residue levels for each protein. The normalized RMSF for residues is calculated as the average normalized RMSF of all atoms within that residue. Normalization is achieved by subtracting the mean and dividing by the standard deviation of RMSF in the PDB model. CC is an abbreviation for correlation coefficient. c RMSF-net performance on large protein complexes (PDB entry 6FBV, 6KU9, and 6LTN). The first and second rows display the PDB models of three protein complexes, with colors corresponding to the normalized RMSF values, indicated by the color bar on the right. The first row is colored based on the RMSF predictions by RMSF-net. The second row is colored according to the ground truth RMSF values from MD simulations. The third row demonstrates the profiles of normalized RMSF-net predicted values and normalized ground truth values along residue sequences for three proteins, where residue IDs correspond to the sequence order in the PDB models. CC is an abbreviation for correlation coefficient. d The dynamic change of NTCP protein from the inward-facing conformation to the open-pore conformation (PDB entry 7PQG/7PQQ). From left to right, the first cartoon illustrates the conformational transition of 7PQG to 7PQQ. The second, third, and fourth cartoons depict the dynamic changes in the conformational transition of this protein from the front, back, and top-down perspectives, respectively, where the RMSF difference is calculated and colored on the 7PQQ using the color bar provided in the upper right corner. The RMSF visualization was generated using PyMOL59 and UCSF ChimeraX60.

In addition to small proteins, Fig.3c presents test examples of large protein complexes, including Mycobacterium tuberculosis RNA polymerase with Fidaxomicin44 (EMD-4230/PDB 6FBV), RSC complex45 (EMD-9905/PDB 6K15), and coronavirus spike glycoprotein trimer46 (EMD-6516/PDB 3JCL). RMSF-net also excels on these complex structures, achieving correlation coefficients of 0.902, 0.819, and 0.804, respectively. Remarkably, these proteins are associated with human diseases and drug development, emphasizing the potential value of RMSF-net in facilitating drug development efforts. Predictions by RMSF-net_cryo and RMSF-net_pdb for these proteins are provided in Supplementary Figs.S4S6, with correlation coefficients of 0.759 and 0.859 for 6FBV, 0.661 and 0.774 for 6K15, and 0.635 and 0.784 for 3JCL, respectively. Comparing model predictions in the Supplementary Figs. shows that RMSF-net aligns more closely with RMSF-net_pdb, supporting the previous argument that information from the PDB model plays a primary role in RMSF-nets feature processing.

We further applied RMSF-net to investigate dynamic changes in the NTCP protein during its involvement in biochemical processes. NTCP (Na+/taurocholate co-transporting polypeptide)47 is a vital membrane transport protein predominantly located on the cell membrane of liver cells in the human body. It is responsible for transporting bile acids from the bloodstream into liver cells and plays a crucial role in the invasion and infection processes of liver viruses such as the hepatitis B virus. Therefore, structural and functional analysis of NTCP is crucial for liver disease treatment. In a previous study, two NTCP conformations, the inward-facing conformation (EMD-13593/PDB 7PQG) and the open pore conformation (EMD-13596/PDB 7PQQ), were resolved using cryo-electron microscopy. We performed dynamic predictions using RMSF-Net on these two conformations and revealed dynamic changes during the transition from the inward-facing to the open pore state, as shown in Fig.3d. Compared to the inward-facing state, the open pore conformation displayed increased dynamics in the TM1 and TM6 regions of the panel domain, the TM7 and TM9 regions of the core domain, and the X motif in the center. Other regions maintained stability or exhibited enhanced stability. We hypothesize that the increased flexibility in these regions is associated with the relative motion between the panel and core domain in the open pore state, facilitating the transport of bile acids and the binding of preS1 of HBV in this conformation.

Despite exhibiting high-performance, RMSF-net is trained and tested based on relatively short-term (30ns) MD simulations. In order to determine whether the structural fluctuation patterns obtained through the 30ns simulation are stable enough for the model training, we performed longer simulations on three proteins46,48,49 (PDB 3JCL, 6QNT, and 6SXA). The detailed setup and results are provided in the Supplementary Information (section MD simulations over longer time periods). In these cases, the results from the previous 30ns simulation showed strong correlations with the results up to 500ns, indicating that the 30ns simulation can effectively capture stable structural fluctuation mode, thus qualified to serve as the foundation for our model training. The RMSF-net predictions also maintain high correlations with the long-term MD simulations, proving that the trained model has effectively absorbed the structural fluctuation patterns in MD simulations.

In addition, to make large-scale simulations feasible, we removed small molecules and ionic ligands during MD simulations, but ligands are included in the input density of RMSF-net. Therefore, the simulation results may be inaccurate regarding the flexibility of the ligand-containing protein, especially the flexibility near the ligands. To assess the impact of this treatment, we performed MD simulations for two protein systems containing ligands: the structure of the cargo bound AP-1:Arf1:tetherin-Nef closed trimer monomeric subunit50 (EMD-7537/PDB 6CM9) and spastin hexamer in complex with substrate51 (EMD-20226/PDB 6P07). Configurations of the MD simulations are provided in the Supplementary Information (section MD simulation configurations for ligand-binding proteins and membrane proteins). The simulations with and without ligands exhibited high correlations in terms of RMSF, with correlations of 0.748 and 0.859 for the two proteins, respectively (Table2). The predictions of RMSF-net also maintain comparable correlations with the additional simulations, of 0.757 and 0.859 respectively. This indicates that, overall, small molecule ligands have little impact on protein structural flexibility. However, we observed that near the ligands, the RMSF obtained from simulations without ligands is indeed higher than that obtained from simulations with ligands, as shown in Fig.4. In these regions, the predicted values of RMSF-net are even closer to simulations with ligands, i.e., the predicted RMSF is lower. Our understanding of this phenomenon is that on one hand, the structure of the ligands is relatively small compared to the protein structure, so the scope of influence is limited and does not greatly affect the global distribution of structural flexibility. On the other hand, the local structures near the ligands became relaxed without the ligands in the original simulations. The deep model utilizes the learned pattern between protein internal structures and flexibility to infer the dynamics of the protein structure binding to the ligands. Although this is only an approximation, it has some correction effect.

a Results for 6CM9. The top panel shows scatter plots of RMSF on residues, with colors corresponding to the three approaches as indicated in the legend. The middle panels present the visualizations of RMSF from the three approaches on the PDB structure, with colors corresponding to the normalized RMSF values, indicated by the color bar on the right. Ligands (GTP) are shown as yellow sticks, residues within 5 of the ligands are shown as surfaces, and black boxes indicate their positions. The bottom panels display the RMSF of structures near the ligands separately, highlighting regions within the black boxes in the middle panels. b Results for 6P07. The top panel shows scatter plots of RMSF on residues, with colors corresponding to the three approaches as indicated in the legend. The middle panels present the visualizations of RMSF from the three approaches on the PDB structure, with colors corresponding to the normalized RMSF values, indicated by the color bar on the right. Ligands (ADP, ATP) are shown as yellow sticks, and residues within 5 of the ligands are shown as surfaces. The bottom panels display the RMSF of structures near the ligands separately.

Another aspect of MD simulation is the simulation of membrane proteins. In the sample preparation of cryo-EM, membrane proteins are purified and separated from membrane structures (Van Heel et al., 2000), which means that the structure and dynamics of membrane proteins in cryo-EM reflect their free state in solution. Correspondingly, our dynamic simulations were also performed in the membrane-free state. Therefore, our model is applicable to proteins in their free state in solution. However, in vivo, membrane proteins are attached to the cell membrane, so considering the simulation environment of the membrane will more accurately simulate their dynamics in biological systems. To explore the differences brought by the membranes, we conducted MD simulations in a membrane environment on two membrane proteins, the cryo-EM structure of TRPV5 (1-660) in nanodisc52 (EMD-0593/PDB 6O1N) and cryo-EM structure of MscS channel, YnaI53 (EMD-6805/PDB 5Y4O). The configurations of the MD simulations are provided in Supplementary Information (section MD simulation configurations for ligand-binding proteins and membrane proteins). The results, as shown in Table3 and Fig.5, demonstrate that the RMSF obtained from MD simulations with and without membranes maintain consistency overall, with correlations of 0.767 and 0.678 on 6O1N and 5Y4O respectively. The correlations between RMSF-net predictions and MD simulations with membranes are 0.804 and 0.675 respectively for these two proteins. As shown in Fig.5b, the presence of the membrane leads to some changes in the flexibility of 5Y4O: in the upper region of 5Y4O, the RMSF obtained from MD simulations with the membrane is lower than that from MD simulations without membrane and RMSF-net. We speculate that this region may be influenced by membrane constraints, resulting in decreased flexibility, but the overall flexibility distribution remains largely unchanged. In addition, we observe that on these two highly symmetrical structures, the predictions of RMSF-net also maintain symmetry similar to MD simulations.

a Results for 6O1N. b Results for 5Y4O. The top panels show scatter plots of RMSF on residues, with colors corresponding to the three approaches indicated in the legend. The bottom panels present the visualizations of RMSF from the three approaches on the PDB structure, with colors corresponding to the normalized RMSF values, indicated by the color bar inthe middle.

The experimental results also indicate that our method exhibits consistent performance across cryo-EM maps with varying resolutions. The resolution of cryo-EM maps signifies the minimum scale at which credible structures are discernible within the map. In our dataset, there are more maps in the resolution range of 34 compared to 23, as shown in Fig.6a. Considering that our method takes cryo-EM maps of various resolutions into network training, concerns arise regarding potential model bias towards specific map resolutions. To address this concern, we conducted an analysis of the test performance of RMSF-net_cryo, Occ2RMSF-net, and RMSF-net compared to RMSF- net_pdb on maps of different resolutions. Results demonstrate that these models exhibit no significant performance differences across different resolution ranges, as shown in Fig.6bd. Only a minor deviation is observed in the range of 22.5, which is statistically insignificant due to the limited number of 7 data points. This underscores that neural networks can fit data indiscriminately within the high-resolution range of 24, without the need to process the maps to a uniform resolution at the preprocessing stage. The similar distributions of RMSF-net and Occ2RMSF-net across different resolutions, shown in Fig.6b, e, further support the conclusion that dynamic inference from cryo-EM maps relies on an intermediate process of structural resolution. Furthermore, Fig.6d demonstrates that, on average, RMSF-net outperforms RMSF-net_pdb across different resolution ranges, indicating that cryo-EM maps have an auxiliary effect on PDB models for dynamic analysis across different resolutions.

a Resolution distribution of cryo-EM maps in the dataset. bd Shows the performance of RMSF prediction methods on different resolution maps in the dataset. For resolution groups 22.5, 2.53, 33.5 and 3.54, the sample sizes N=7, 42, 120, and 166. b Distribution of correlation coefficients (CC) of RMSF-net_cryo on maps across the four resolution ranges. c Distribution of correlation coefficients of Occ2RMSF-net across the four resolution ranges. d Distribution of correlation difference between RMSF-net and RMSF-net_pdb on the four resolution ranges. In bd, the center white line, the lower and upper bound of the box in each violin plot indicate the median, the first quartile (Q1), and the third quartile (Q3), respectively. The whiskers of the boxes indicate Q1-1.5*IQR and Q3+1.5*IQR, with IQR representing the interquartile range. The bounds of the violin plots show the minima and maxima, and the width indicates the density of the data. e, f Show the relationship between RMSF-net run time and data size. e The relationship between the RMSF-net run time on CPUs and the weighted sum of map size and PDB size among data points of the dataset. The map size and PDB size are weighted as 0.0015:0.9985 from linear regression, both taking k voxels as the units. The weighted sum range is set below 1000, which encompasses the majority of the data. The full range is presented in Supplementary Fig.S12a. f The relationship between the RMSF-net run time on GPUs and the map size among data points of the dataset. The map size is set below 3003, which encompasses the majority of the data. The full range is presented in Supplementary Fig.S12b.

In addition to its superior performance, RMSF-net demonstrates rapid inference capabilities and minimal storage overhead, whether running on high-performance GPUs or CPUs. Using a computer equipped with 10 CPU cores and 2 NVIDIA Tesla A100 GPUs, we conducted runtime assessments on the dataset, revealing a strong linear relationship between the execution time of RMSF-net and the data size. Moreover, compared to conventional MD simulations and DEFMap, this approach achieves substantial acceleration in processing speed.

As shown in Fig.6e, Supplementary Figs.S11c and S12a, when executed on CPUs, RMSF-nets runtime is directly proportional to the weighted summation of cryo-EM map size and PDB model size, with a weight ratio of map size to PDB size at 0.0015:0.9985, both measured in units of k voxels. For most data, the weighted sum of map size and PDB size is within 500 k voxels, and processing can be completed in under a minute. When executed on GPUs, since most of the time is spent on preprocessing, the total time is linearly related to the map size, as shown in Fig.6f, Supplementary Figs.S11a and S12b. For most maps with sizes below 3003 voxels, computations can be completed within 30s. Detailed information regarding the RMSF-net processing time is provided in the Supplementary Information (section Details of the RMSF-net processing time).

For comparative analysis, we selected ten relatively smallmaps from our dataset and performed runtime assessments using RMSF-net and DEFMap. As presented in Supplementary TablesS1S3, across these ten data points, DEFMap exhibited processing times of 45.9431.84minutes on CPUs and 37.5110.51minutes on GPUs, concurrently generating data files of size 11.988.30 GB. In contrast, RMSF-net showcased remarkable efficiency, with runtime durations of 16.669.60s and 3.091.45s on CPUs and GPUs, respectively, and yielding data files of 66.3031.06MB. Both in terms of storage occupancy and time consumption, RMSF-net demonstrates significant improvements over DEFMap. Furthermore, in contrast to extended MD simulations, which often require hours or even days to perform simulations of 30ns on individual proteins, RMSF-net delivers predictions with an average correlation of up to 0.75 and saves time and resources significantly, making it an ultra-fast means of performing protein dynamic analysis.

Read this article:
Accurate Prediction of Protein Structural Flexibility by Deep Learning Integrating Intricate Atomic Structures and Cryo ... - Nature.com

Brezis, H. & Browder, F. Partial differential equations in the 20th century. Adv. Math. 135, 76144 (1998).

Article MathSciNet Google Scholar

Dissanayake, M. & Phan-Thien, N. Neural-network-based approximations for solving partial differential equations. Commun. Numer. Methods Eng. 10, 195201 (1994).

Article Google Scholar

Rico-Martinez, R. & Kevrekidis, I. G. Continuous time modeling of nonlinear systems: a neural network-based approach. In Proc. IEEE International Conference on Neural Networks 15221525 (IEEE, 1993).

Gonzlez-Garca, R., Rico-Martnez, R. & Kevrekidis, I. G. Identification of distributed parameter systems: a neural net based approach. Comput. Chem. Eng. 22, S965S968 (1998).

Article Google Scholar

Raissi, M., Perdikaris, P. & Karniadakis, G. Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686707 (2019).

Article MathSciNet Google Scholar

Yu, B. et al. The Deep Ritz method: a deep learning-based numerical algorithm for solving variational problems. Commun. Math. Stat. 6, 112 (2018).

Article MathSciNet Google Scholar

Mller, J. & Zeinhofer, M. Deep Ritz revisited. Preprint at https://arxiv.org/abs/1912.03937 (2019).

Gao, H., Zahr, M. J. & Wang, J.-X. Physics-informed graph neural Galerkin networks: a unified framework for solving PDE-governed forward and inverse problems. Comput. Methods Appl. Mech. Eng. 390, 114502 (2022).

Article MathSciNet Google Scholar

Bruna, J., Peherstorfer, B. & Vanden-Eijnden, E. Neural Galerkin schemes with active learning for high-dimensional evolution equations. J. Comput. Phys. 496, 112588 (2024).

Article MathSciNet Google Scholar

Battaglia, P. W. et al. Relational inductive biases, deep learning and graph networks. Preprint at https://arxiv.org/abs/1806.01261 (2018).

Sanchez-Gonzalez, A. et al. Learning to simulate complex physics with graph networks. In Proc. International Conference on Machine Learning 84598468 (PMLR, 2020).

Burger, M. et al. Connections between deep learning and partial differential equations. Eur. J. Appl. Math. 32, 395396 (2021).

Article Google Scholar

Loiseau, J.-C. & Brunton, S. L. Constrained sparse Galerkin regression. J. Fluid Mech. 838, 4267 (2018).

Article MathSciNet Google Scholar

Cranmer, M. et al. Lagrangian neural networks. Preprint at https://arxiv.org/abs/2003.04630 (2020).

Brunton, S. L., Noack, B. R. & Koumoutsakos, P. Machine learning for fluid mechanics. Annu. Rev. Fluid Mech. 52, 477508 (2020).

Article MathSciNet Google Scholar

Wang, R., Walters, R. & Yu, R. Incorporating symmetry into deep dynamics models for improved generalization. In International Conference on Learning Representations (ICLR, 2021).

Wang, R., Kashinath, K., Mustafa, M., Albert, A. & Yu, R. Towards physics-informed deep learning for turbulent flow prediction. In Proc. 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining 14571466 (ACM, 2020).

Brandstetter, J., Berg, R. V. D., Welling, M. & Gupta, J. K. Clifford neural layers for PDE modeling. In Eleventh International Conference on Learning Representations (ICLR, 2023)

De Haan, P., Weiler, M., Cohen, T. & Welling, M. Gauge equivariant mesh CNNS: anisotropic convolutions on geometric graphs. In International Conference on Learning Representations (ICLR, 2021).

Brandstetter, J., Welling, M. & Worrall, D. E. Lie point symmetry data augmentation for neural PDE solvers. In Proc. International Conference on Machine Learning 22412256 (PMLR, 2022).

Brandstetter, J., Worrall, D. & Welling, M. Message passing neural PDE solvers. Preprint at https://arxiv.org/abs/2202.03376 (2022).

Karniadakis, G. E. et al. Physics-informed machine learning. Nat. Rev. Phys. 3, 422440 (2021).

Article Google Scholar

Brunton, S. L. & Kutz, J. N. Data-Driven Science and Engineering: Machine Learning, Dynamical Systems and Control 2nd edn (Cambridge Univ. Press, 2022).

Bongard, J. & Lipson, H. Automated reverse engineering of nonlinear dynamical systems. Proc. Natl Acad. Sci. USA 104, 99439948 (2007).

Article Google Scholar

Schmidt, M. & Lipson, H. Distilling free-form natural laws from experimental data. Science 324, 8185 (2009).

Article Google Scholar

Brunton, S. L., Proctor, J. L. & Kutz, J. N. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proc. Natl Acad. Sci. USA 113, 39323937 (2016).

Article MathSciNet Google Scholar

Cranmer, M. Interpretable machine learning for science with PySR and SymbolicRegression.jl. Preprint at https://arxiv.org/abs/2305.01582 (2023).

Rudy, S. H., Brunton, S. L., Proctor, J. L. & Kutz, J. N. Data-driven discovery of partial differential equations. Sci. Adv 3, e1602614 (2017).

Article Google Scholar

Schaeffer, H. Learning partial differential equations via data discovery and sparse optimization. Proc. Math. Phys. Eng. Sci. 473, 20160446 (2017).

MathSciNet Google Scholar

Zanna, L. & Bolton, T. Data-driven equation discovery of ocean mesoscale closures. Geophys. Res. Lett. 47, e2020GL088376 (2020).

Article Google Scholar

Schmelzer, M., Dwight, R. P. & Cinnella, P. Discovery of algebraic Reynolds-stress models using sparse symbolic regression. Flow Turbulence Combustion 104, 579603 (2020).

Article Google Scholar

Beetham, S. & Capecelatro, J. Formulating turbulence closures using sparse regression with embedded form invariance. Phys. Rev. Fluids 5, 084611 (2020).

Article Google Scholar

Beetham, S., Fox, R. O. & Capecelatro, J. Sparse identification of multiphase turbulence closures for coupled fluid-particle flows. J. Fluid Mech. 914, A11 (2021).

Article MathSciNet Google Scholar

Bakarji, J. & Tartakovsky, D. M. Data-driven discovery of coarse-grained equations. J. Comput. Phys. 434, 110219 (2021).

Article MathSciNet Google Scholar

Maslyaev, M., Hvatov, A. & Kalyuzhnaya, A. Data-driven partial derivative equations discovery with evolutionary approach. In Proc. Computational ScienceICCS 2019: 19th International Conference Part V 19, 635641 (Springer, 2019).

Xu, H., Zhang, D. & Wang, N. Deep-learning based discovery of partial differential equations in integral form from sparse and noisy data. J. Comput. Phys. 445, 110592 (2021).

Article MathSciNet Google Scholar

Xu, H., Chang, H. & Zhang, D. DLGA-PDE: Discovery of PDEs with incomplete candidate library via combination of deep learning and genetic algorithm. J. Comput. Phys. 418, 109584 (2020).

Article MathSciNet Google Scholar

Xu, H., Zhang, D. & Zeng, J. Deep-learning of parametric partial differential equations from sparse and noisy data. Phys. Fluids 33, 037132 (2021).

Article Google Scholar

Xu, H. & Zhang, D. Robust discovery of partial differential equations in complex situations. Phys. Rev. Res. 3, 033270 (2021).

Article Google Scholar

Chen, Y., Luo, Y., Liu, Q., Xu, H. & Zhang, D. Symbolic genetic algorithm for discovering open-form partial differential equations (SGA-PDE). Phys. Rev. Res. 4, 023174 (2022).

Article Google Scholar

Taira, K. & Colonius, T. The immersed boundary method: a projection approach. J. Comput. Phys. 225, 21182137 (2007).

Article MathSciNet Google Scholar

Colonius, T. & Taira, K. A fast immersed boundary method using a nullspace approach and multi-domain far-field boundary conditions. Comput. Methods Appl. Mech. Eng. 197, 21312146 (2008).

Article MathSciNet Google Scholar

Van Breugel, F., Kutz, J. N. & Brunton, B. W. Numerical differentiation of noisy data: a unifying multi-objective optimization framework. IEEE Access 8, 196865196877 (2020).

Article Google Scholar

Messenger, D. A. & Bortz, D. M. Weak SINDy: Galerkin-based data-driven model selection. Multiscale Model. Simul. 19, 14741497 (2021).

Article MathSciNet Google Scholar

Messenger, D. A. & Bortz, D. M. Weak SINDy for partial differential equations. J. Comput. Phys. 443, 110525 (2021).

Article MathSciNet Google Scholar

Schaeffer, H. & McCalla, S. G. Sparse model selection via integral terms. Phys. Rev. E 96, 023302 (2017).

Article MathSciNet Google Scholar

Fasel, U., Kutz, J. N., Brunton, B. W. & Brunton, S. L. Ensemble-SINDy: robust sparse model discovery in the low-data, high-noise limit, with active learning and control. Proc. R. Soc. A 478, 20210904 (2022).

Article MathSciNet Google Scholar

Gurevich, D. R., Reinbold, P. A. & Grigoriev, R. O. Robust and optimal sparse regression for nonlinear PDE models. Chaos 29, 103113 (2019).

Article MathSciNet Google Scholar

Alves, E. P. & Fiuza, F. Data-driven discovery of reduced plasma physics models from fully kinetic simulations. Phys. Rev. Res. 4, 033192 (2022).

Article Google Scholar

Reinbold, P. A., Gurevich, D. R. & Grigoriev, R. O. Using noisy or incomplete data to discover models of spatiotemporal dynamics. Phys. Rev. E 101, 010203 (2020).

Article Google Scholar

Suri, B., Kageorge, L., Grigoriev, R. O. & Schatz, M. F. Capturing turbulent dynamics and statistics in experiments with unstable periodic orbits. Phys. Rev. Lett. 125, 064501 (2020).

Article Google Scholar

Reinbold, P. A., Kageorge, L. M., Schatz, M. F. & Grigoriev, R. O. Robust learning from noisy, incomplete, high-dimensional experimental data via physically constrained symbolic regression. Nat. Commun. 12, 3219 (2021).

Article Google Scholar

Pope, S. A more general effective-viscosity hypothesis. J. Fluid Mech. 72, 331340 (1975).

Article Google Scholar

Ling, J., Kurzawski, A. & Templeton, J. Reynolds averaged turbulence modelling using deep neural networks with embedded invariance. J. Fluid Mech. 807, 155166 (2016).

Article MathSciNet Google Scholar

Duraisamy, K., Iaccarino, G. & Xiao, H. Turbulence modeling in the age of data. Annu. Rev. Fluid Mech. 51, 357377 (2019).

Article MathSciNet Google Scholar

Ahmed, S. E. et al. On closures for reduced order modelsa spectrum of first-principle to machine-learned avenues. Phys. Fluids 33, 091301 (2021).

Article Google Scholar

Supekar, R. et al. Learning hydrodynamic equations for active matter from particle simulations and experiments. Proc. Natl Acad. Sci. USA 120, e2206994120 (2023).

Article MathSciNet Google Scholar

Read the rest here:
Promising directions of machine learning for partial differential equations - Nature.com

eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

Generative artificial intelligence (AI) is valued for its ability to create new content, including text, images, video, and music. It uses AI algorithms to analyze patterns in datasets to mimic style or structure to replicate different types of content, and can be used to create deep-fake videos and voice messages.

Generative AI is a subset of artificial intelligence, which also includes a broad range of technologies that enable machines to perform tasks that once required human intelligence and judgment. Its often used to build systems with the cognitive capacity to mine data, and it continuously boosts its performance over the course of repeated events. Heres what you need to know about the benefits and logistics of using AI and generative AI as well as the ethical concerns of which to be aware.

Both generative AI and artificial intelligence,sometimes called traditional AI, use machine learning algorithms to obtain their results. However, they have different goals and purposes. Generative AI is intended to create new content, while AI goes much broader and deeperin essence, to wherever the algorithm coder wants to take it. AIs possible deployments include better decision-making, removing the tedium from repetitive tasks, or spotting anomalies and issuing alerts for cybersecurity. The following summary spells out the common differences between generative AI and AI:

To fully understand the relationship between generative AI and AI, its necessary to understand each of these technologies at a deeper level.

Generative AI is an open-ended and rapidly evolving form of artificial intelligence. Its major characteristics include the following:

With its ability to use source data for any number of creative tasks, generative AIs use cases range from product design to software development to fraud detection.

Generative AI helps in creating innovative designs that meet specific performance criteria, from prototyping to optimization of design, while minimizing not only material use but also waste. Additionally, generative AI succeeds at creating highly personalized product experiences by analyzing user data to create products that align with the preferences and needs of individual users. This personalization can help with cheating marketing and sales campaigns.

For the creative industries, generative AI can mimic various artistic styles, compose original music and even generate complete pieces of artwork. This application is expanding the horizons of creative expression and is being used by artists, musicians, and other content creators to increase their output.

Generative AI provides the ability to automate code generation, bug fixes, and optimization. This results in more efficient development cycles and higher-quality software. AI tools can also generate synthetic data for training and testing purposes, which plays an important role in developing robust AI applications.

Generative AI-powered chatbots and virtual assistants provide 24/7 assistance, personalize interactions, and handle complex queries. These tools raise customer satisfaction and operational efficiency by automating routine support tasks and offering faster responses than human operators.

In finance and insurance, generative AI is used to detect fraud and manage risk. It analyzes transaction patterns and identifies anomalies, then helps in creating detailed reports and summaries that aid in decision-making, thereby enhancing the overall security and reliability of financial operations.

Based on the significant advancements that keep enhancing generative AIs capabilities, its future is incredibly promising. Expect to see models becoming larger and more powerful, like GPT-4 and PaLM2, which are revolutionizing content creation and personalized customer communications. Such models enable businesses to generate high-quality, human-like outputs more efficiently, with impact seen across many market sectors.

We can also expect to see generative AI models run on a wider variety of hardware devices, which will open up an array of use cases. A notable trend is the rise of multimodal AI models that can understand and generate content across several forms of data, such as text, images, and audio. The result? Users will get more immersive and natural user experiences, especially in fields like virtual reality and augmented reality.

Additionally, generative AI is driving new levels of personalization by improving how it adapts products and services to individual preferences. Its therefore seen as a particularly aggressive driver of change across retail, marketing and ecommerce sectors.

Although artificial intelligence has enjoyed an enormously higher profile over the last few years, the history of AI stretches back to the 1940s. This traditional AI is the basis for generative AI, and while there are major differences, there is major overlap between these two technologies. To fully understand the topic, heres a deeper look at artificial intelligence itself.

Overall, traditional AI is focused on explicit programming to execute tasks with precision. The following are its core characteristics:

Artificial intelligence can compute exponentially faster than the fastest team of human experts, even as it handles far greater complexity. This capability enables an array of use cases, ranging from business automation to research and development to cybersecurity.

AI-driven automation is streamlining repetitive and manual business operations. Robotic process automation (RPA) uses AI to automate routine administrative tasks, freeing up human workers for more complex activities. AI algorithms are used to optimize supply chain management by predicting demand, managing inventory, and optimizing logistics.

In research and development (R&D), traditional AI accelerates innovation by analyzing huge datasets to identify patterns, predict outcomes, and generate new insights. In pharmaceuticals, AI helps drug discovery by predicting the efficacy of compounds and optimizing clinical trials. In engineering, AI models can be used to optimize product designs, which helps to lower the time and cost associated with bringing new products to market.

AI is increasingly used for predictive maintenance, with use cases like analyzing data from machinery to predict failures before they occur. This proactive approach helps schedule maintenance activities at optimal times. The benefits include lower downtime and extended equipment lifespans. Industries such as manufacturing, energy, and transportation are the biggest beneficiaries of predictive maintenance.

AIs role in cybersecurity and fraud detection includes analyzing network traffic and identifying potential threats in real time. AI algorithms detect anomalies and patterns associated with cyber attacks, which leads to faster and more accurate responses. AI-driven systems can automate responses to a variety of threats and reduce the risk of breaches and enhance overall security.

AI-enabled forecasting models help financial leaders predict future trends. AI systems incorporate variables like mixed economic forecasts and non-traditional data sources. It then allows for more reliable and comprehensive financial scenario planning and more specific revenue projections.

The future of AI involves handling ever more complex and multifaceted real-world scenarios. Innovations will likely focus on enhancing the adaptability of rule-based systems, making them more flexible and capable of dealing with unforeseen situations. Expect to see enhanced flexibility and the rise of multimodal systems capable of processing many data types simultaneously. This will allow AI to tackle more complex enterprise challenges across multiple domains and significantly broaden its impact.

Self-improving AI systems are also emerging. They leverage reinforcement learning and dynamic analysis for autonomous optimization of performance over time. This will further enhance adaptability and efficiency without constant human intervention.

The integration of traditional AI with generative AI is expected to create hybrid systems that deliver an exponentially more powerful combination. Innovations in AI hardware and infrastructure, including specialized AI processors, will support these advanced systems. This will allow traditional AI to provide more sophisticated solutions across an expanding array of use cases.

Generative AI and traditional AI face largely similar challenges in terms of ethics, including biases built into systems, job displacement and potential environmental impact.

AI systems can inadvertently magnify biases that were built into their training data. These biases can lead to unfair outcomes, particularly for marginalized groups. To ensure fairness in AI, whether generative or traditional AI, there needs to be meticulous scrutiny of the training data, implementation of bias mitigation strategies, and continual monitoring of AI systems for biased behavior. Techniques like algorithmic fairness reviews and bias audits are a step toward promoting equity and inclusivity in AI applications.

The security and privacy concerns raised by the deployment of AI technologies are pervasive. AI systems often need vast amounts of data, including personal and sensitive information, to function effectively. Whether generative or traditional, ensuring robust data protection measures and maintaining privacy throughout the AI lifecycle are critical. This includes implementing strong encryption, data anonymization techniques, and complying with regulations such as GDPR. Transparency about data usage and incorporating user consent is also essential in building trust and safeguarding privacy.

For the ever-increasing reach and use cases of AI, we need to be able to trust AI and hold the technology accountableand many users do not trust AI systems. This trust is enabled by transparency in AI systems. Explainable AI (XAI) practices allow users and stakeholders to understand how AI algorithms make decisions. By providing clear and understandable explanations of AI processes, organizations can enhance user trust and facilitate better decision-making. A transparent system makes it easy to identify and address any ethical issues and to ensure AI systems are used responsibly.

One of the greatest concerns about the rise of AI has been job displacement as automated systems replace human roles. Alleviating this issue calls for strategies for transitioning workforces to new or evolved roles, such as reskilling and upskilling programs to prepare employees for roles created by AI advancements. Organizations need to consider the broader social implications of deploying AI solutions and work to implement practices that strike a balance between technological progress and socioeconomic stability.

The deployment and training of large AI models, especially generative AI, requires significant computational resources, which leads to substantial energy consumption and environmental impact. Organizations using AI need to develop and implement energy-efficient AI models. They also need to optimize computational resources to minimize carbon footprints. Encouraging sustainable practices in AI development and operation is a must for reducing the environmental impact and promoting green AI technologies.

Artificial intelligence in all its forms is advancing at a remarkable rate, so its advantageous for tech professionals to be knowledgeable about AI skills and developments. Here are relevant courses to help you use these technologies effectively. Please be aware that while each title below refers to generative AI, these courses all teach fundamental concepts that also cover overall AI technology.

This course provides a solid foundation in generative AI, covering fundamental concepts, model types, and practical applications. Its suitable for those who are new to the field and want to explore the potential of generative AI using Google Cloud tools like Vertex AI.

Andrew Ngs course offers a comprehensive introduction to generative AI. It cuts across the workings, uses, and impact of generative AI in various industries. The course also includes hands-on exercises for applying the concepts you learn practically.

Based on a partnership between AWS and DeepLearning.AI, this intermediate-level course goes into using large language models (LLMs) like GPT-4 for generative AI. It covers the architecture, training processes, practical applications of LLMs, and more. The course is designed for data scientists, AI developers, and anyone interested in mastering LLMs and applying them effectively in their work.

No, conversational AI and generative AI are related but distinct subsets of artificial intelligence. Conversational AI is designed to interact with users through dialogue, often used in chatbots and virtual assistants like Siri, Alexa, or Google Assistant. It focuses on understanding and generating human-like responses to deliver meaningful interactions. Generative AI, on the other hand, refers to AI systems that create new content based on learned patterns from existing data. While conversational AI can use generative AI techniques to give responses, generative AI covers a broader range of creative applications beyond just conversation.

Predictive AI focuses on analyzing existing data to forecast future events or trends. It uses techniques like regression analysis, time series analysis, and machine learning models to predict outcomes such as stock prices, weather conditions, or customer behaviors. Generative AI, however, aims to create new data rather than predict future events. It uses models like Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs) to generate new content that is similar to the training data.

Generative AI has rapidly gained popularity due to several key factors. The development of sophisticated models like GPT-4, GANs, and VAEs has significantly improved the quality and realism of generated content. Increased access to high-performance computing resources such as GPUs and cloud computing has enabled the training of complex generative models. The vast amount of data available for training these models has allowed them to learn from diverse and extensive datasets, enhancing their capabilities. Plus, the wide range of applicationsfrom creative industries like art and music to practical uses such as text generation and synthetic data creationhas driven interest and investment in generative AI.

Generative AI and traditional AI each bring unique strengths and challenges to the table. Generative AI is geared for creativity, generating new and innovative content, and is seeing more integration into fields like art, music, and content creation. In contrast, traditional AI focuses on analyzing existing data to improve efficiency, accuracy, and decision-making, making it invaluable in sectors that value consistency and predictability such as finance, healthcare, and manufacturing.

As both these technologies continue to evolve rapidly, the differences between them will likely lessen, with generative AIs creativity and AIs data crunching strength found side by side in many advanced applications.

Read our guide to the Top 20 Generative AI Tools and Apps 2024 to learn more about what platforms organizations are using to deploy these dynamic technologies across their businesses.

Follow this link:
Generative AI vs. AI: Advantages, Limitations, Ethical Considerations - eWeek

Heres a look at 10 data science and machine learning tools that solution and service providers should be aware of.

Deep Thoughts

Data science and machine learning technologies have long been important for data analytics tasks and predictive analytical software. But with the wave of artificial intelligence and generative AI development in 2023, the importance of data science and machine learning tools has risen to new heights.

One absolute truth about AI systems is that they need huge amounts of data to be effective.

Data science combines math and statistics, advanced analytics, specialized programming and other skills and tools to help uncover actionable insights within an organizations data. The global data science tool market reached $8.73 billion last year and will nearly double to $16.85 billion by 2030, according to 24MarketReports.

Machine learning systems make business-outcome decisions and predictions based on algorithms and statistical models that analyze and draw inferences from huge amounts of data. The worldwide machine learning market is expected to reach $79.29 billion this year, according to Statista, and grow at a 36 percent CAGR to $503.40 billion by 2030

Heres a look at some of the hottest data science and machine learning tools in use today. Some of the following tools are relatively new to the market while others have been around for a while and recently updated. The list also includes both commercial products and open-source software.

Amazon SageMaker

Amazon SageMaker is one of Amazon Web Services (AWS) flagship AI and machine learning software tools and is one of the most prominent machine learning products in the industry.

In November, at the AWS re:Invent extravaganza, AWS expanded SageMakers functionalities with five new capabilities that the company said helps accelerate the building, training and deployment of large language models and other foundation machine learning models that power generative AI.

One new capability enhances SageMakers ability to scale models by accelerating model training time while another optimizes managed ML infrastructure operations by reducing deployment costs and model latency.

The new SageMaker Clarify tool makes it easier to select the right model based on quality parameters that support responsible use of AI. A new no-code feature in SageMaker Canvas makes it possible to prepare data using natural language instructions. And Canvas continues to democratize model building and customization, AWS said, by making it easier to use models to extract insights, make predictions and generate content using an organizations proprietary data.

AWS also offers Amazon Machine Learning, a more highly automated tool for building machine learning models.

Anaconda Distribution for Python

Python has become the most popular programming language overall, but it has long been used by data scientists for development in data analytics, AI and machine learning. Anacondas distribution of the open-source Python system is one of the most widely used data science and AI platforms.

In addition to its distribution of Python, Anaconda offers its Data Science and AI Workbench platform that data science and machine learning teams use for expediting model development and deployment while adhering to security and governance requirements.

Over the last year Anaconda has established alliances with major IT vendors to expand the use of its platform. In April Anaconda announced a partnership to integrate its Anaconda Python Repository with Teradatas VantageCloud and ClearScape Analytics. A collaboration with IBM announced in February provides watsonx.ai users with access to the Anaconda software repository. And in August 2023 the company unveiled the Anaconda Distribution for Python in Microsoft Excel.

ClearML

ClearMLs platform, designed for data scientists and data engineers, automates and simplifies the development and management of machine learning solutions. The system provides a comprehensive lineup of capabilities spanning data science, data management, MLOps, and model orchestration and deployment.

In March startup ClearML added new orchestration capabilities to its platform to expand control over AI infrastructure management and compute costs while maximizing the use of compute resources and improving model serving visibility.

Also in March introduced an open-source fractional GPU tool to help business utilize their GPU utilization by enabling multi-tenancy for all Nvidia GPUs.

Databricks Mosaic AI

At Databricks recent Data + AI Summit the company unveiled a number of new capabilities for its Mosaic AI software for building and deploying production-quality ML and GenAI applications.

Databricks acquired MosaicML in June 2023 in a blockbuster $1.3-billion deal and has been integrating the startups technology with its data lakehouse platform. (Databricks has since rebranded the product as Mosaic AI.)

The latest capabilities in Mosaic AI include support for building compound AI systems, new functionality to improve model quality, and AI governance tools. Databricks said the innovations give users the confidence to build and measure production-quality applications, delivering on the promises of generative AI for their business.

Dataiku

The Dataiku platform offers a comprehensive lineup of data science, machine learning and AI capabilities including machine learning development, MLOps, data preparation, DataOps, visualization, analytical applications and generative AI.

In September 2023, Dataiku launched LLM Mesh, a new tool for integrating large language models within the enterprise that the company called the common backbone for Gen AI applications. LLM Mesh capabilities include universal AI service routing, secure access and auditing for AI services, performance and cost tracking, and safety provisions for private data screening and response moderation.

In April, Dataiku debuted LLM Cost Guard, a new capability within LLM Mesh that creates standards for tracking and optimizing generative AI use cases.

dotData Feature Factory 1.1

dotDatas Feature Factory is an automated feature discovery and engineering platform that helps data scientists find and use data features within large-scale data sets for use in AI and machine learning projects.

In Feature Factory version 1.1, introduced in May, the company provided significant enhancements including new data quality assessment capabilities, support for user-defined features and interactive feature selection, improved support for AutoML through the Python-based PyCaret AutoML library, and preview support for generative AI feature discovery.

Hopsworks MLOps Platform

The Hopsworks platform is used to develop, deploy and monitor AI/ML models at scale.

The core of the serverless system is its machine learning feature store for storing data for ML models running on AWS, Azure and Google Cloud platforms and in on-premises systems. The Hopsworks platform also provides machine learning pipelines and a comprehensive development toolkit.

Hopsworks 3.7, what the company called the GenAI release, became generally available in March with new capabilities to support GenAI and large language model use cases. It also introduced feature monitoring, a new notification service to track changes to specific features, and support for the Delta Lake data storage format.

Founded in Sweden in 2016, Hopsworks has offices in Stockholm, London and Palo Alto, Calif.

Obviously AI

A problem faced by many businesses is the shortage of people with data science and machine learning expertise. Obviously AI looks to close that gap with its no-code AI/ML platform that allows people without technical backgrounds to build and train machine learning models.

The platform helps quickly build models that run predictions on historical data, everything from sales and revenue forecasting to predictions about energy consumption and population growth.

Because data science shouldnt feel like rocket science, the companys web site says.

PyTorch

PyTorch is a powerful open-source framework and deep learning library for data scientists who are building and training deep learning models.

PyTorch is popular for such applications as computer vision, natural language processing, image classification and text generation. It can be used for a variety of algorithms including convolutional neural networks, recurrent neural networks and generative adversarial networks, according to a LinkedIn posting by data scientist and analysis expert Vitor Mesquita.

PyTorch 2.3 was released on April 24.

PyTorch was created out of the Lua-based Torch framework that came out of Facebooks AI research lab in 2017. Today PyTorch is part of the Linux Foundation and is available through the pytorch.org website.

PyTorch and TensorFlow are generally seen as the top alternative even competing open-source data science and machine learning systems, according to a Projectpro.com comparison. PyTorch is often considered better for smaller-scale research projects while TensorFlow is more widely used for production-scale projects.

TensorFlow

TensorFlow is a popular open-source, end-to-end machine learning platform and library for building ML models that can run in any environment. The system handles data preprocessing, model building and model training tasks.

TensorFlow, generally seen as an alternative to PyTorch, was originally developed by the Google Brain team for internal research and production tasks, particularly around machine learning and deep leaning neural networks. It was originally released as open-source software under the Apache License 2.0 in November 2015.

Google continues to own and maintain TensorFlow, which is available through the tensorflow.org community website. A major update, TensorFlow 2.0, was released in September 2019.

TensorFlow and PyTorch are generally seen as the top alternative even competing open-source data science and machine learning systems, according to a Projectpro.com comparison. PyTorch is often considered better for smaller-scale research projects while TensorFlow is more widely used for production-scale projects.

Go here to read the rest:
The 10 Hottest Data Science And Machine Learning Tools Of 2024 (So Far) - CRN