Cloud Hosting

Recent developments in machine learning (ML) have led to increasingly large models, some of which require hundreds of billions of parameters. Although they are more powerful, training and inference on those models require significant computational resources. Despite the availability of advanced distributed training libraries, its common for training and inference jobs to need hundreds of accelerators (GPUs or purpose-built ML chips such as AWS Trainium and AWS Inferentia), and therefore tens or hundreds of instances.

In such distributed environments, observability of both instances and ML chips becomes key to model performance fine-tuning and cost optimization. Metrics allow teams to understand workload behavior and optimize resource allocation and utilization, diagnose anomalies, and increase overall infrastructure efficiency. For data scientists, ML chips utilization and saturation are also relevant for capacity planning.

This post walks you through the Open Source Observability pattern for AWS Inferentia, which shows you how to monitor the performance of ML chips, used in an Amazon Elastic Kubernetes Service (Amazon EKS) cluster, with data plane nodes based on Amazon Elastic Compute Cloud (Amazon EC2) instances of type Inf1 and Inf2.

The pattern is part of the AWS CDK Observability Accelerator, a set of opinionated modules to help you set observability for Amazon EKS clusters. The AWS CDK Observability Accelerator is organized around patterns, which are reusable units for deploying multiple resources. The open source observability set of patterns instruments observability with Amazon Managed Grafana dashboards, an AWS Distro for OpenTelemetry collector to collect metrics, and Amazon Managed Service for Prometheus to store them.

The following diagram illustrates the solution architecture.

This solution deploys an Amazon EKS cluster with a node group that includes Inf1 instances.

The AMI type of the node group is AL2_x86_64_GPU, which uses the Amazon EKS optimized accelerated Amazon Linux AMI. In addition to the standard Amazon EKS-optimized AMI configuration, the accelerated AMI includes the NeuronX runtime.

To access the ML chips from Kubernetes, the pattern deploys the AWS Neuron device plugin.

Metrics are exposed to Amazon Managed Service for Prometheus by the neuron-monitor DaemonSet, which deploys a minimal container, with the Neuron tools installed. Specifically, the neuron-monitor DaemonSet runs the neuron-monitor command piped into the neuron-monitor-prometheus.py companion script (both commands are part of the container):

The command uses the following components:

Data is visualized in Amazon Managed Grafana by the corresponding dashboard.

The rest of the setup to collect and visualize metrics with Amazon Managed Service for Prometheus and Amazon Managed Grafana is similar to that used in other open source based patterns, which are included in the AWS Observability Accelerator for CDK GitHub repository.

You need the following to complete the steps in this post:

Complete the following steps to set up your environment:

The following is our sample output:

COA_AMG_ENDPOINT_URL needs to include https://.

The secret will be accessed by the External Secrets add-on and made available as a native Kubernetes secret in the EKS cluster.

The first step to any AWS CDK deployment is bootstrapping the environment. You use the cdk bootstrap command in the AWS CDK CLI to prepare the environment (a combination of AWS account and AWS Region) with resources required by AWS CDK to perform deployments into that environment. AWS CDK bootstrapping is needed for each account and Region combination, so if you already bootstrapped AWS CDK in a Region, you dont need to repeat the bootstrapping process.

Complete the following steps to deploy the solution:

The actual settings for Grafana dashboard JSON files are expected to be specified in the AWS CDK context. You need to update context in the cdk.json file, located in the current directory. The location of the dashboard is specified by the fluxRepository.values.GRAFANA_NEURON_DASH_URL parameter, and neuronNodeGroup is used to set the instance type, number, and Amazon Elastic Block Store (Amazon EBS) size used for the nodes.

You can replace the Inf1 instance type with Inf2 and change the size as needed. To check availability in your selected Region, run the following command (amend Values as you see fit):

Complete the following steps to validate the solution:

The following screenshot shows our sample output.

The following is our expected output:

The following is our expected output:

The following screenshot shows our expected output.

The following screenshot shows our expected output.

Log in to your Amazon Managed Grafana workspace and navigate to the Dashboards panel. You should see a dashboard named Neuron / Monitor.

To see some interesting metrics on the Grafana dashboard, we apply the following manifest:

This is a sample workload that compiles the torchvision ResNet50 model and runs repetitive inference in a loop to generate telemetry data.

To verify the pod was successfully deployed, run the following code:

You should see a pod named pytorch-inference-resnet50.

After a few minutes, looking into the Neuron / Monitor dashboard, you should see the gathered metrics similar to the following screenshots.

Grafana Operator and Flux always work together to synchronize your dashboards with Git. If you delete your dashboards by accident, they will be re-provisioned automatically.

You can delete the whole AWS CDK stack with the following command:

In this post, we showed you how to introduce observability, with open source tooling, into an EKS cluster featuring a data plane running EC2 Inf1 instances. We started by selecting the Amazon EKS-optimized accelerated AMI for the data plane nodes, which includes the Neuron container runtime, providing access to AWS Inferentia and Trainium Neuron devices. Then, to expose the Neuron cores and devices to Kubernetes, we deployed the Neuron device plugin. The actual collection and mapping of telemetry data into Prometheus-compatible format was achieved via neuron-monitor and neuron-monitor-prometheus.py. Metrics were sourced from Amazon Managed Service for Prometheus and displayed on the Neuron dashboard of Amazon Managed Grafana.

We recommend that you explore additional observability patterns in the AWS Observability Accelerator for CDK GitHub repo. To learn more about Neuron, refer to the AWS Neuron Documentation.

Riccardo Freschi is a Sr. Solutions Architect at AWS, focusing on application modernization. He works closely with partners and customers to help them transform their IT landscapes in their journey to the AWS Cloud by refactoring existing applications and building new ones.

Go here to read the rest:
Open source observability for AWS Inferentia nodes within Amazon EKS clusters | Amazon Web Services - AWS Blog

Article by: Antonio Rubio, Project Engineer, Braking Systems in Applus IDIADA

ReviewPart One| ReviewPart Two | Review Part Three

The field of artificial intelligence (AI) has made significant progress in recent years, with applications ranging from natural language processing to computer vision. In recent years, Applus IDIADA Brakes department has presented several studies about artificial intelligence application for detection of brake noises. In this paper, Applus IDIADA presents the research done in this area, but focusing on the development of an AI model for predicting subjective ratings for squeal brake noises based on objective measurements collected through the instrumentation in a typical Brake Noise Durability programme. Subjective ratings are based on human opinions and can be challenging to quantify. Objective measurements, on the other hand, can be objectively quantified and provide a more reliable basis for prediction.

The first part of the article introduced the data processing, whereas the second and third parts focused on the AI model creation and validation, respectively. This fourth part, on the other hand, summarizes the main results and draws the conclusions.

Other drivers evaluations

Subjective ratings from two different highly skilled drivers were used (different from the reference driver selected for the model trained). With that, the noises and conditions of noises should be similar, but drivers evaluations are different. Dataset per rating used to evaluate other drivers evaluations is shown in table 9.

Using different drivers for validation, we are validating at the same time:

Ideally, model prediction accuracy should be similar to the accuracy result that comes from the validation performed on the model with the reference driver. Differences between accuracy of the model of the reference driver and the accuracy with the data set of other drivers, could be attributed to differences in subjective criteria between reference driver and the driver evaluated.

It can be seen that there are more subjective ratings available in the data set with high ratings than for low ratings.

Similar to the validation of the model for the reference driver, results for each driver are presented in terms of accuracy. Results can be checked in table 10 and accuracy per driver/rating in table 11.

Accuracy is calculated comparing the subjective rating prediction from the model with the actual ones of the drivers, meaning a 100% of accuracy a correct prediction (same as driver) of the model for all subjective ratings. In addition, the % of ratings not correctly assigned with a difference error of 1 rating, 2 rating and 3 rating is calculated.

It can be seen that:

It can be seen that:

Summary results

Regarding reference driver validation, close to 70% of prediction ratings are the same as the reference driver rating. Rating discrepancies between model and driver rating are mainly with a 1 rating error. Rating discrepancies between model and driver rating more than 2 points are minimal. Accuracy for rating 9, rating 8 and rating 7 is around 70%. Accuracy for rating 6 or lower decrease to 50% or lower.

Regarding other drivers evaluations, the accuracy is around 50% for both of them. Same tendency in comparison with reference driver results can be shown. There is an increase of rating discrepancies mainly of 1 rating. The decrease of accuracy can be explained with the difference of subjective criteria of the drivers in comparison with the reference driver.

Conclusion

The goal of the project is to replicate the evaluation of brake noise annoyance performed by an expert driver using a model. Data containing noise samples collected during several years of testing at Applus IDIADA from a reference driver and their corresponding subjective ratings are provided for this purpose.

The data analysis revealed that there is a feasible opportunity to clean and preprocess the dataset by removing variables that do not contribute value to the model. Outliers were removed from the dataset. Data has been split in three parts: 70% noise events for training, 20% for test and 10% for validation.

Two artificial intelligence models were trained with the dataset: a classification and a regression model. According to the test phase results of training, it is shown that models achieve a good knowledge of the dataset. Finally, according to the different trials, the final model involves combining the classification and regression models. A threshold is set to determine when to rely on the classification models prediction and when to prioritize the rounded output from the regression model.

The model underwent validation by comparing its results with evaluations from the reference driver using different vehicles in conditions that were used for training. An accuracy of 68.5% was achieved, with rating discrepancies between model and driver rating mainly with a 1 rating.

In addition, predicted ratings from different drivers with model from the reference driver have been compared. It can be seen that accuracy in comparison with the reference driver decreased, but it can be explained as differences in subjective criteria with the other drivers.

The results of the study were promising, obtaining with the model an important level of accuracy in predicting subjective ratings based on objective measurements, indicating that the models predictions were close to the actual subjective ratings. Actually, it can be seen during models training that characterization of the subjective criteria is learnt by the models. Main rating discrepancies between model and driver rating are mainly with a 1 rating error that it could be explained as some uncertainty in the subjective criteria of the reference driver. This uncertainty in the subjective criteria of the driver could be explained by a variety of uncontrolled variables which can result in different subjective ratings for the same noise event. These differences appear mainly for low rating below rating 6. In addition, dataset contained a smaller number of lower rating 6 or below than above 6.

In conclusion, the development of an AI model for predicting subjective ratings based on objective measurements is an important step towards the understanding of subjective ratings and objective measurements for brake squeal noise. Prediction results from the current artificial intelligence model are based in objective measurements from 20 variables at the same time that characterize the most important features of the noise as frequency, amplitude, duration or corner source. Furthermore, the results of this study demonstrate the potential of AI models to be implemented in the near-to-medium future on autonomous vehicles providing more accurate subjective rating based on objective data. Future work in this area could involve expanding the model to include additional variables or incorporating other machine learning techniques to further improve performance.

About Applus IDIADA

With over 25 years experience and 2,450 engineers specializing in vehicle development, Applus IDIADA is a leading engineering company providing design, testing, engineering, and homologation services to the automotive industry worldwide.

Applus IDIADA is located in California and Michigan, with further presence in 25 other countries, mainly in Europe and Asia.

http://www.applusidiada.com

See the original post:
Brake Noise And Machine Learning (4 of 4) - The BRAKE Report

Heart failure occurs when the heart is unable to pump enough blood. Symptoms can include fatigue, weakness, swollen legs and feet and, ultimately, death.

UVA Health researchers have developed a powerful new risk assessment tool for predicting outcomes in heart failure patients. The researchers have made the tool publicly available for free to clinicians.

The new tool improves on existing risk assessment tools for heart failure by harnessing the power of machine learning (ML) and artificial intelligence (AI) to determine patient-specific risks of developing unfavorable outcomes with heart failure.

Heart failure is a progressive condition that affects not only quality of life but quantity as well. All heart failure patients are not the same. Each patient is on a spectrum along the continuum of risk of suffering adverse outcomes, said researcher Sula Mazimba, MD, a heart failure expert. Identifying the degree of risk for each patient promises to help clinicians tailor therapies to improve outcomes.

Heart failure occurs when the heart is unable to pump enough blood for the bodys needs. Thiscan lead to fatigue, weakness, swollen legs and feet and, ultimately, death.Heart failure isa progressive condition, so it is extremely important forclinicians to be able to identify patients at risk ofadverse outcomes.

Further, heart failure is a growing problem. More than 6 million Americans already have heart failure, and that number is expected to increase to more than 8 million by 2030. The UVA researchers developed their new model, called CARNA, to improve care for these patients. (Finding new ways to improve care for patients across Virginia and beyond is a key component of UVA Healths first-ever10-year strategic plan.)

The researchersdeveloped their model using anonymized data drawn from thousands of patients enrolled in heart failure clinical trialspreviously funded by the National Institutes of Healths National Heart, Lung and Blood Institute. Putting the model to the test, they found it outperformed existing predictors for determining how a broad spectrum of patients would fare in areas such as the need for heart surgery or transplant, the risk of rehospitalization and the risk of death.

The researchers attribute the models successto the use of ML/AI and the inclusion of hemodynamic clinical data, which describe how blood circulates through the heart, lungs and the rest of the body.

This model presents a breakthrough because it ingests complex sets of data and can make decisions even among missing and conflicting factors, said researcher Josephine Lamp, of the University of Virginia School of Engineerings Department of Computer Science. It is really exciting because the model intelligently presents and summarizes risk factors reducing decision burden so clinicians can quickly make treatment decisions.

By using the model, doctors will be better equipped to personalize care to individual patients, helping them live longer, healthier lives, the researchers hope.

The collaborative research environment at the University of Virginia made this work possible by bringing together experts in heart failure, computer science, data science and statistics, said researcher Kenneth Bilchick, MD, a cardiologist at UVA Health. Multidisciplinary biomedical research that integrates talented computer scientists like Josephine Lamp with experts in clinical medicine will be critical to helping our patients benefit from AI in the coming years and decades.

The researchers have made their new tool available online for free athttps://github.com/jozieLamp/CARNA.

In addition, they havepublished the results of their evaluation of CARNA in the American Heart Journal. The research team consisted of Lamp, Yuxin Wu, Steven Lamp, Prince Afriyie, Nicholas Ashur, Bilchick, Khadijah Breathett, Younghoon Kwon, Song Li, Nishaki Mehta, Edward Rojas Pena, Lu Feng and Mazimba. The researchers have no financial interest in the work.

The project was based on one of the winning submissions to the National Heart, Lung and Blood Institutes Big Data Analysis Challenge: Creating New Paradigms for Heart Failure Research. The work was supported by the National Science Foundation Graduate Research Fellowship, grant 842490, and NHLBI grants R56HL159216, K01HL142848 and L30HL148881.

To keep up with the latest medical research news from UVA, subscribe to theMaking of Medicineblog.

See the original post here:
Artificial Intelligence Tool to Improve Heart Failure Care - UVA Health Newsroom

Newswise Scientists used a neural network, a type of brain-inspired machine learning algorithm, to sift through large volumes of particle collision data.

For over two decades, theATLASparticle detector has recorded the highest energy particle collisions in the world within the Large Hadron Collider (LHC) located atCERN, the European Organization for Nuclear Research. Beams of protons are accelerated around theLHCat close to the speed of light, and upon their collision atATLAS, they produce a cascade of new particles, resulting in over a billion particle interactions per second.

Particle physicists are tasked with mining this massive and growing store of collision data for evidence of undiscovered particles. In particular, theyre searching for particles not included in theStandard Modelof particle physics, our current understanding of the universes makeup that scientists suspect is incomplete.

As part of theATLAScollaboration, scientists at the U.S. Department of Energys (DOE) Argonne National Laboratory and their colleagues recently used a machine learning approach called anomaly detection to analyze large volumes ofATLASdata. The method has never before been applied to data from a collider experiment. It has the potential to improve the efficiency of the collaborations search for something new. The collaboration involves scientists from 172 research organizations.

The team leveraged a brain-inspired type of machine learning algorithm called a neural network to search the data for abnormal features, or anomalies. The technique breaks from more traditional methods of searching for new physics. It is independent of and therefore unconstrained by the preconceptions of scientists.

Rather than looking for very specific deviations, the goal is to find unusual signatures in the data that are completely unexplored, and that may look different from what our theories predict. Physicist Sergei Chekanov

Traditionally,ATLASscientists have relied on theoretical models to help guide their experiment and analysis in the directions most promising for discovery. This often involves performing complex computer simulations to determine how certain aspects of collision data would look according to the Standard Model. Scientists compare these Standard Model predictions to real data fromATLAS. They also compare them to predictions made by new physics models, like those attempting to explaindark matterand other phenomena unaccounted for by the Standard Model.

But so far, no deviations from the Standard Model have been observed in the billions of billions of collisions recorded atATLAS. And since the discovery of theHiggs bosonin 2012, theATLASexperiment has yet to find any new particles.

Anomaly detection is a very different way of approaching this search, said Sergei Chekanov, a physicist in Argonnes High Energy Physics division and a lead author on the study.Rather than looking for very specific deviations, the goal is to find unusual signatures in the data that are completely unexplored and that may look different from what our theories predict.

To perform this type of analysis, the scientists represented each particle interaction in the data as an image that resembles aQRcode. Then, the team trained their neural network by exposing it to 1% of the images.

The network consists of around 2 million interconnected nodes, which are analogous to neurons in the brain. Without human guidance or intervention, it identified and remembered correlations between pixels in the images that characterize Standard Model interactions. In other words, it learned to recognize typical events that fit within Standard Model predictions.

After training, the scientists fed the other 99% of the images through the neural network to detect any anomalies. When given an image as input, the neural network is tasked with recreating the image using its understanding of the data as a whole.

If the neural network encounters something new or unusual, it gets confused and has a hard time reconstructing the image, said Chekanov.If there is a large difference between the input image and the output it produces, it lets us know that there might be something interesting to explore in that direction.

Using computational resources at Argonnes Laboratory Computing Resource Center, the neural network analyzed around 160 million events withinLHCRun-2 data collected from 2015 to 2018.

Although the neural network didnt find any glaring signs of new physics in this data set, it did spot one anomaly that the scientists think is worth further study. An exotic particle decay at an energy of around 4.8 teraelectronvolts results in a muon (a type of fundamental particle) and a jet of other particles in a way that does not fit with the neural networks understanding of Standard Model interactions.

Well have to do more investigation, said Chekanov.It is likely a statistical fluctuation, but theres a chance this decay could indicate the existence of an undiscovered particle.

The team plans to apply this technique to data collected during theLHCRun-3 period, which began in 2022.ATLASscientists will continue to explore the potential of machine learning and anomaly detection as tools for charting unknown territory in particle physics.

The results of the study were published inPhysical Review Letters. This work was funded in part by theDOEOffice of Sciences Office of High Energy Physics and the National Science Foundation.

Argonne National Laboratoryseeks solutions to pressing national problems in science and technology. The nations first national laboratory, Argonne conducts leading-edge basic and applied scientific research in virtually every scientific discipline. Argonne researchers work closely with researchers from hundreds of companies, universities, and federal, state and municipal agencies to help them solve their specific problems, advance Americas scientific leadership and prepare the nation for a better future. With employees from more than 60 nations, Argonne is managed byUChicago Argonne,LLCfor theU.S. Department of Energys Office of Science.

The U.S. Department of Energys Office of Scienceis the single largest supporter of basic research in the physical sciences in the United States and is working to address some of the most pressing challenges of our time. For more information, visithttps://energy.gov/science.

Read more:
Machine learning could help reveal undiscovered particles within data from the Large Hadron Collider - Newswise