Page 682«..1020..681682683684..690700..»

Working with Non-IID data part2(Machine Learning 2023) – Medium

Photo by Mostafa Ashraf Mostafa on Unsplash

Author : Yeachan Kim, Bonggun Shin

Abstract : Federated learning algorithms perform reasonably well on independent and identically distributed (IID) data. They, on the other hand, suffer greatly from heterogeneous environments, i.e., Non-IID data. Despite the fact that many research projects have been done to address this issue, recent findings indicate that they are still sub-optimal when compared to training on IID data. In this work, we carefully analyze the existing methods in heterogeneous environments. Interestingly, we find that regularizing the classifiers outputs is quite effective in preventing performance degradation on Non-IID data. Motivated by this, we propose Learning from Drift (LfD), a novel method for effectively training the model in heterogeneous settings. Our scheme encapsulates two key components: drift estimation and drift regularization. Specifically, LfD first estimates how different the local model is from the global model (i.e., drift). The local model is then regularized such that it does not fall in the direction of the estimated drift. In the experiment, we evaluate each method through the lens of the five aspects of federated learning, i.e., Generalization, Heterogeneity, Scalability, Forgetting, and Efficiency. Comprehensive evaluation results clearly support the superiority of LfD in federated learning with Non-IID data

2. Federated PAC-Bayesian Learning on Non-IID data. (arXiv)

Author : Zihao Zhao, Yang Liu, Wenbo Ding, Xiao-Ping Zhang

Abstract : Existing research has either adapted the Probably Approximately Correct (PAC) Bayesian framework for federated learning (FL) or used information-theoretic PAC-Bayesian bounds while introducing their theorems, but few considering the non-IID challenges in FL. Our work presents the first non-vacuous federated PAC-Bayesian bound tailored for non-IID local data. This bound assumes unique prior knowledge for each client and variable aggregation weights. We also introduce an objective function and an innovative Gibbs-based algorithm for the optimization of the derived bound. The results are validated on real-world datasets

Read more here:
Working with Non-IID data part2(Machine Learning 2023) - Medium

Read More..

Amity University Online Collaborates with TCS iON to Offer Machine Learning and Gen AI Certification Program – DATAQUEST

Amity University Online, Indias pioneering online degree program provider authorized by the UGC, has unveiled a Certificate Program in Machine Learning and Generative AI. This initiative is a collaborative effort of Amity University with TCS iON, the strategic arm of Tata Consultancy Services dedicated to Manufacturing Industries (SMB), Educational Institutions, and Examination Boards. The program spans eight months and is designed to empower learners with knowledge and skills in the domains of Machine Learning and Generative AI.

The Certificate Program in Machine Learning and Generative AI harnesses the collaboration between Amity University Onlines e-learning expertise and the industry insights and experienced instructors of TCS iON. Participants of this program will not only engage closely with TCS instructors but will also gain invaluable hands-on experience through TCS projects, allowing them to apply their knowledge to practical scenarios.

This program distinguishes itself through a range of features offered by TCS iON, including master classes, weekend sessions, live interactions with industry experts, and the completion of capstone projects. These elements are designed to provide learners with profound insights into the latest trends and advancements in Machine Learning and Generative AI, backed by updated study materials and live projects.

The curriculum of this certification program ensures that learners master machine learning pipelines, including the deployment of AWS Cloud. Moreover, participants will acquire proficiency in core Python programming for ML and Generative AI, and they will gain advanced skills in Computer Vision and NLP through deep learning techniques. During the course, special focus will be given to topics such as CV & NLP models, ChatGPT, and Dall-E. Learners will also benefit from expert-led live sessions, allowing them to gain real-time insights into industry practices. Additionally, they will also delve into model interpretability using tools like LIME & SHAP, which will enhance their understanding of machine learning algorithms.

The AI landscape is continuously evolving. By 2030, it is estimated to add up to $15.7 trillion to the global economy, unlocking potential job opportunities for skilled professionals. Launching the Machine Learning and Generative AI certification program in collaboration with TCS iON positions us at the forefront of this transformative industry, empowering learners with knowledge and expertise in AI. With TCS iONs industry insights, learners will gain access to real-world and capstone projects, ensuring they are well-equipped to navigate the AI-driven world of tomorrow, said Ajit Chauhan, Spokesperson, of Amity University Online.

We are excited to partner with Amity University Online in launching this innovative program in Machine Learning and Generative AI. TCS iON is committed to promoting the skill development of the nations youth, and this collaboration is a testament to our dedication to empowering learners with the latest advancements in technology. To ensure a strong industry focus, we will bring SMEs and domain experts from TCS for every module of the course apart from enabling learning content andprojects. Students will also be given an opportunity to appear in the TCS iON National Proficiency Test on the subject to prove their expertise and be job-ready, said Venguswamy Ramaswamy, Global Head of TCS iON while commenting on the collaboration.

The collaboration between Amity University Online and TCS iON represents a significant step towards equipping learners with advanced skills in Machine Learning and Generative AI. Participants of this program can stay at the forefront of the rapidly evolving digital world with an immersive eight-month program, hands-on experience, and insights from industry experts.

Read the original post:
Amity University Online Collaborates with TCS iON to Offer Machine Learning and Gen AI Certification Program - DATAQUEST

Read More..

Introduction – Rethinking Clinical Trials

Artificial intelligence (AI) is the theory and practice of designing computer systems to simulate actual processes of human intelligence. AI-powered systems rely on computers that embed machine learning (ML) to analyze large datasets and discover patterns across them. AI/ML thus provide powerful computing tools for pragmatic clinical trial (PCT) investigators. These tools support multimodal data analytics (e.g., data from electronic health records, wearables, and social media), advanced prediction, and large-scale modeling that far exceed the analytic capacities of many existing trial designs. Using AI/ML, a new class of digital PCTs has emerged (Inan et al 2020). It is anticipated that modern digital PCTs will increasingly serve as testbeds for AI/ML systems in clinical decision support (Yao et al 2021). Application of AI/ML could help find new ways to contain healthcare costs and facilitate longitudinal health surveillance. Among their strengths, AI/ML-enabled digital PCTs can help investigators to:

However, AI/ML systems are only as accurate as the data on which they are trained. Multimodal linkages allow researchers to triangulate many sources of data that collectively improve how algorithms iteratively learn and begin to discover patterns indicative of health or hospital trends. Health-related data used for medical AI/ML research and development are often from various sources, including:

Both over- and under-representation of patient populations in these AI/ML training data sources can yield biased results that in turn harm real patients and exacerbate existing health inequities. For example, one study (Obermeyer et al 2019) found that Black patients were given a lower risk score than equally sick White patients based on data input that reflected that more health care dollars are spent on White versus Black patients. The algorithm misunderstood that signal to assume that meant that Black patients needed less healthcare, rather than had less access.

These and other technical limitations of AI/ML have ethical consequences that digital PCT investigators should anticipate and can proactively address at every stage in the research.

See the article here:
Introduction - Rethinking Clinical Trials

Read More..

Inexpensive water-treatment monitoring process powered by machine learning – Tech Xplore

This article has been reviewed according to ScienceX's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:

close

Small, rural drinking water treatment (DWT) plants typically use only chlorine to implement the disinfection process. For these plants, free chlorine residual (FCR) serves as a key performance measure for disinfection. The FCR is stated as the concentration of free chlorine remaining in the water, after the chlorine has oxidized the target contaminants.

In practice, the FCR is determined by plant operators based on their experience. Specifically, operators choose a dose of chlorine to achieve a satisfactory FCR concentration, but often have to make an estimate of the chlorine requirements.

The challenge of determining an accurate FCR has led to the use of advanced FCR prediction techniques. In particular, machine learning (ML) algorithms have proven effective in achieving this goal. By identifying correlations among numerous variables in complex systems, successful ML implementation could accurately predict FCR, even from cost-effective, low-tech monitoring data.

In a new study published in Frontiers of Environmental Science & Engineering, the authors implemented a gradient boosting (GB) ML model with categorical boosting (CatBoost) to predict FCR. GB algorithms, including CatBoost, accumulate decision trees to generate the prediction function.

The input data was collected from a DWT plant in Georgia in the U.S., and included a wide variety of DWT monitoring records and operational process parameters. Four iterations of a generalized modeling approach were developed, including (1) base case, (2) rolling average, (3) parameter consolidation, and (4) intuitive parameters.

The research team also applied the SHapely Additive explanation (SHAP) method to this study. SHAP is an open-source software for interpreting ML models with many input parameters, which allows users to visually understand how each parameter affects the prediction function. We can study the influence of each parameter on the predicted output, by calculating its corresponding SHAP value. For example, the SHAP analysis ranks the channel Cl2 as the most influential parameter.

Of all four iterations, the fourth and final iteration considered only intuitive, physical relationships and water quality measured downstream from filtration. The authors summarized the comparative performance of the four ML modeling iterations. According to them, the key findings are: 1) with a sufficient number of related input parameters, ML models can produce accurate prediction results; 2) ML models can be driven by correlations that may or may not have a physical basis; 3) ML models can be analogous to operator experience.

Looking forward, the research team suggests that future studies should explore expanding the applicability domain. For example, the data set analyzed was limited to only one full year. Therefore, greater data availability is expected to broaden the applicability domain and improve the predictivity.

More information: Wiley Helm et al, Development of gradient boosting-assisted machine learning data-driven model for free chlorine residual prediction, Frontiers of Environmental Science & Engineering (2023). DOI: 10.1007/s11783-024-1777-6. journal.hep.com.cn/fese/EN/10. 07/s11783-024-1777-6

Provided by Frontiers Journals

Read the original:
Inexpensive water-treatment monitoring process powered by machine learning - Tech Xplore

Read More..

Snowflake Accelerates How Users Build Next Generation Apps and Machine Learning Models in the Data Cloud – Yahoo Finance

Snowflake Notebooks unlock data exploration and machine learning development for SQL and Python users with an interactive, cell-based programming environment

Snowflake advances Snowpark to streamline end-to-end machine learning workflows with the Snowpark ML Modeling API, Snowpark Model Registry, Snowflake Feature Store, and more

Hundreds of Snowflake customers including Cybersyn, LiveRamp, and SNP are increasing developer productivity with the Snowflake Native App Framework and unlocking new revenue streams through Snowflake Marketplace

No-Headquarters/BOZEMAN, Mont., November 01, 2023--(BUSINESS WIRE)--Snowflake (NYSE: SNOW), the Data Cloud company, today announced at its Snowday 2023 event new advancements that make it easier for developers to build machine learning (ML) models and full-stack apps in the Data Cloud. Snowflake is enhancing its Python capabilities through Snowpark to boost productivity, increase collaboration, and ultimately speed up end-to-end AI and ML workflows. In addition, with support for containerized workloads and expanded DevOps capabilities, developers can now accelerate development and run apps all within Snowflake's secure and fully managed infrastructure.

This press release features multimedia. View the full release here: https://www.businesswire.com/news/home/20231101058683/en/

Snowflake Accelerates How Users Build Next Generation Apps and Machine Learning Models in the Data Cloud (Graphic: Business Wire)

"The rise of generative AI has made organizations most valuable asset, their data, even more indispensable. Snowflake is making it easier for developers to put that data to work so they can build powerful end-to-end machine learning models and full-stack apps natively in the Data Cloud," said Prasanna Krishnan, Senior Director of Product Management, Snowflake. "With Snowflake Marketplace as the first cross-cloud marketplace for data and apps in the industry, customers can quickly and securely productionize what theyve built to global end users, unlocking increased monetization, discoverability, and usage."

Story continues

Developers Gain Robust and Familiar Functionality for End-to-End Machine Learning

Snowflake is continuing to invest in Snowpark as its secure deployment and processing of non-SQL code, with over 35% of Snowflake customers using Snowpark on a weekly basis (as of September 2023). Developers increasingly look to Snowpark for complex ML model development and deployment, and Snowflake is introducing expanded functionality that makes Snowpark even more accessible and powerful for all Python developers. New advancements include:

Snowflake Notebooks (private preview): Snowflake Notebooks are a new development interface that offers an interactive, cell-based programming environment for Python and SQL users to explore, process, and experiment with data in Snowpark. Snowflakes built-in notebooks allow developers to write and execute code, train and deploy models using Snowpark ML, visualize results with Streamlit chart elements, and much more all within Snowflakes unified, secure platform.

Snowpark ML Modeling API (general availability soon): Snowflakes Snowpark ML Modeling API empowers developers and data scientists to scale out feature engineering and simplify model training for faster and more intuitive model development in Snowflake. Users can implement popular AI and ML frameworks natively on data in Snowflake, without having to create stored procedures.

Snowpark ML Operations Enhancements: The Snowpark Model Registry (public preview soon) now builds on a native Snowflake model entity and enables the scalable, secure deployment and management of models in Snowflake, including expanded support for deep learning models and open source large language models (LLMs) from Hugging Face. Snowflake is also providing developers with an integrated Snowflake Feature Store (private preview) that creates, stores, manages, and serves ML features for model training and inference.

Endeavor, the global sports and entertainment company that includes the WME Agency, IMG & On Location, UFC, and more, relies on Snowflakes Snowpark for Python capabilities to build and deploy ML models that create highly personalized experiences and apps for fan engagement.

"Snowpark serves as the driving force behind our end-to-end machine learning development, powering how we centralize and process data across our various entities, and then securely build and train models using that data to create hyper-personalized fan experiences at scale," said Saad Zaheer, VP of Data Science and Engineering, Endeavor. "With Snowflake as our central data foundation bringing all of this development directly to our enterprise data, we can unlock even more ways to predict and forecast customer behavior to fuel our targeted sales and marketing engines."

Snowflake Advances Developer Capabilities Across the App Lifecycle

The Snowflake Native App Framework (general availability soon on AWS, public preview soon on Azure) now provides every organization with the necessary building blocks for app development, including distribution, operation, and monetization within Snowflakes platform. Leading organizations are monetizing their Snowflake Native Apps through Snowflake Marketplace, with app listings more than doubling since Snowflake Summit 2023. This number is only growing as Snowflake continues to advance its developer capabilities across the app lifecycle so more organizations can unlock business impact.

For example, Cybersyn, a data-service provider, is developing Snowflake Native Apps exclusively for Snowflake Marketplace, with more than 40 customers running over 5,000 queries with its Financial & Economic Essentials Native App since June 2022. In addition, LiveRamp, a data collaboration platform, has seen the number of customers deploying its Identity Resolution and Transcoding Snowflake Native App through Snowflake Marketplace increase by more than 80% since June 2022. Lastly, SNP has been able to provide its customers with a 10x cost reduction in Snowflake data processing associated with SAP data ingestion, empowering them to drastically reduce data latency while improving SAP data availability in Snowflake through SNPs Data Streaming for SAP - Snowflake Native App.

With Snowpark Container Services (public preview soon in select AWS regions), developers can run any component of their app from ML training, to LLMs, to an API, and more without needing to move data or manage complex container-based infrastructure.

Snowflake Automates DevOps for Apps, Data Pipelines, and Other Development

Snowflake is giving developers new ways to automate key DevOps and observability capabilities across testing, deploying, monitoring, and operating their apps and data pipelines so they can take them from idea to production faster. With Snowflakes new Database Change Management (private preview soon) features, developers can code declaratively and easily templatize their work to manage Snowflake objects across multiple environments. The Database Change Management features serve as a single source of truth for object creation across various environments, using the common "configuration as code" pattern in DevOps to automatically provision and update Snowflake objects.

Snowflake also unveiled a new Powered by Snowflake Funding Program, innovations that enable all users to securely tap into the power of generative AI with their enterprise data, enhancements to further eliminate data silos and strengthen Snowflakes leading compliance and governance capabilities through Snowflake Horizon, and more at Snowday 2023.

Learn More:

Read more about how developers are building and deploying ML models with the latest Snowflake and Snowpark advancements in this blog post.

Learn more about how organizations can use Snowpark Container Services, Snowflake Native Apps, and Hybrid Tables to build, distribute, and operate full-stack apps on Snowflake in this blog post.

Read how Snowflake Cortex is providing customers with fast, easy, and secure LLM-powered app development in this blog post.

Explore whats new in Snowpark ML with this quickstart guide, and follow along the Snowpark ML docs page.

Ramp up on all things Snowflake Native Apps by signing up for the Snowflake Native App Bootcamp, and checking out this quickstart guide.

Stay on top of the latest news and announcements from Snowflake on LinkedIn and Twitter.

Forward Looking Statements

This press release contains express and implied forward-looking statements, including statements regarding (i) Snowflakes business strategy, (ii) Snowflakes products, services, and technology offerings, including those that are under development or not generally available, (iii) market growth, trends, and competitive considerations, and (iv) the integration, interoperability, and availability of Snowflakes products with and on third-party platforms. These forward-looking statements are subject to a number of risks, uncertainties and assumptions, including those described under the heading "Risk Factors" and elsewhere in the Quarterly Reports on Form 10-Q and the Annual Reports on Form 10-K that Snowflake files with the Securities and Exchange Commission. In light of these risks, uncertainties, and assumptions, actual results could differ materially and adversely from those anticipated or implied in the forward-looking statements. These statements speak only as of the date the statements are first made and are based on information available to us at the time those statements are made and/or management's good faith belief as of that time. Except as required by law, Snowflake undertakes no obligation, and does not intend, to update the statements in this press release. As a result, you should not rely on any forward-looking statements as predictions of future events.

Any future product information in this press release is intended to outline general product direction. This information is not a commitment, promise, or legal obligation for us to deliver any future products, features, or functionality; and is not intended to be, and shall not be deemed to be, incorporated into any contract. The actual timing of any product, feature, or functionality that is ultimately made available may be different from what is presented in this press release.

2023 Snowflake Inc. All rights reserved. Snowflake, the Snowflake logo, and all other Snowflake product, feature and service names mentioned herein are registered trademarks or trademarks of Snowflake Inc. in the United States and other countries. All other brand names or logos mentioned or used herein are for identification purposes only and may be the trademarks of their respective holder(s). Snowflake may not be associated with, or be sponsored or endorsed by, any such holder(s).

About Snowflake

Snowflake enables every organization to mobilize their data with Snowflakes Data Cloud. Customers use the Data Cloud to unite siloed data, discover and securely share data, power data applications, and execute diverse AI/ML and analytic workloads. Wherever data or users live, Snowflake delivers a single data experience that spans multiple clouds and geographies. Thousands of customers across many industries, including 639 of the 2023 Forbes Global 2000 (G2K) as of July 31, 2023, use Snowflake Data Cloud to power their businesses. Learn more at snowflake.com.

View source version on businesswire.com: https://www.businesswire.com/news/home/20231101058683/en/

Contacts

Kaitlyn HopkinsProduct PR Lead, Snowflakepress@snowflake.com

View original post here:
Snowflake Accelerates How Users Build Next Generation Apps and Machine Learning Models in the Data Cloud - Yahoo Finance

Read More..

Strategic Empathy for North Korea: The Intersection of Machine … – James Martin Center for Nonproliferation Studies

November 3, 2023

Why has the approach of the United States and its allies towards the North Korean nuclear weapons issue remained unfruitful for decades? Is our understanding of North Koreas worldview comprehensive and accurate? If not, what elements have been overlooked in our dealings with North Korea? Can we objectively discern reality and find a glimmer of hope for progress?

To address these questions through the lens of strategic empathy, CNS Research Fellow, Hyuk Kim, provides a unique seminar for policymakers to reevaluate their policies towards North Korea. Utilizing an unsupervised learning technique, Mr. Kim presents a visual representation of the global nuclear political landscape, illustrating the political alignment among United Nations Member States on nuclear issues. The quantitative analysis reveals unexpected outcomes, including a potential area for diplomatic cooperation with North Korea and surprising findings a reality some policymakers may find challenging to their assumptions. To interpret such anomalies, Hyuk Kim provides a qualitative analysis to help policymakers understand North Koreas worldview, drawing from Pyongyangs narratives at multilateral diplomatic venues. The seminar concludes with policy implications that paint a somewhat ambivalent picture of the complete denuclearization of the Korean Peninsula.

Chapters

00:00:00 Moderator: Robert Shaw, Director, Export Control and Nonproliferation Program of the James Martin Center for Nonproliferation Studies, Middlebury Institute of International Studies

00:05:30 Speaker: Hyuk Kim, Research Fellow, Export Control and Nonproliferation Program of the James Martin Center for Nonproliferation Studies, Middlebury Institute of International Studies

01:03:15 Q&A

On November 2nd, the Research Fellow of CNS, Mr. Hyuk Kim, delivered an insightful and though-provoking discourse on Strategic Empathy. Mr. Kim commenced the seminar by critically scrutinizing the inherent bias in policymakers worldviews, which are inevitably shaped by their environments. This predisposition often leads them to conflate their personal perspectives on global peace with the universally accepted viewpoint.

Throughout the seminar, Mr. Kim endeavored to find uniqueness from generalization in his analysis of the global nuclear political landscape. Through the quantitative analysis, Mr. Kim discerned the alignment of United Nations Member States on nuclear issues at large. Concurrently, his qualitative analysis unveiled the nuanced variations in the positions of seemingly aligned states on specific issue areas.

Mr. Hyuk Kim concluded the seminar by elucidating how the insights gleaned from the analytical process could illuminate our understanding of the Korean Peninsula. He underscored the significance of understanding North Koreas security concerns, thereby casting a new light on the crucial aspect of reassessing the prevailing policy towards North Korea.

Link:
Strategic Empathy for North Korea: The Intersection of Machine ... - James Martin Center for Nonproliferation Studies

Read More..

A runoff prediction method based on hyperparameter optimisation of … – Nature.com

VMD-CEEMD decomposition algorithm

Variational Modal Decomposition (VMD) is an adaptive signal decomposition algorithm. It can decompose the signal into multiple components, and its essence and core idea is the construction and solution of the variational problem. VMD is commonly used to process non-linear signals and can decompose complex raw data to obtain a series of modal components16.

It can effectively extract the features of runoff data and reduce the influence of its nonlinearity and non-stationarity on the prediction results. The main steps of the VMD algorithm are: (1) The original signal is passed through the Hilbert transform to obtain a series of modal functions u, which are calculated to obtain the unilateral spectrum; (2) Transform the spectrum into the fundamental frequency band and construct the corresponding constrained variational problem by estimating the bandwidth; (3) Converting a constrained variational problem into an unconstrained variational problem17.

The calculated equations are as follows:

$$ L = left( {left{ {u_{k} } right},left{ {omega _{k} } right},lambda } right) = alpha mathop sum limits_{{k - 1}}^{K} left| {partial _{t} left[ {left( {delta left( t right) + frac{j}{{pi t}}} right)*u_{k} left( t right)} right]e^{{ - jomega _{t} t}} } right|_{2}^{2} + left| {fleft( t right) - mathop sum limits_{{k = 1}}^{K} u_{k} left( t right)} right|_{2}^{2} + left[ {lambda left( t right),fleft( t right) - mathop sum limits_{{k = 1}}^{k} u_{k} left( t right)} right], $$

(1)

where ({u}_{k}left(tright)) and ({omega }_{k}) are the modal components and the corresponding center frequencies, respectively, is the penalty function and is the Lagrange multiplier. The results of several experiments show that the decomposition results are better when is taken as 2000, so in this paper, is set to 2000. The k modal components of the VMD are solved by using the alternating direction method of multiplicative operators to find the saddle points of the unconstrained variational problem.

There are some potential features of the VMD decomposed runoff residual sequence. The CEEMD decomposition method is a new adaptive signal processing method. Compared with the commonly used EEMD method, its decomposition efficiency and reconstruction accuracy are higher, and it better exploits the potential features of residual sequences.

The EMD method is a method proposed by Huang et al. for signal time-domain decomposition processing, which is particularly suitable for the analysis of nonlinear and non-stationary time series18. In order to cope with the modal confusion problem of the EMD method, Wu et al.19 proposed an overall average empirical modal decomposition. The EEMD method effectively suppresses the modal aliasing caused by the EMD method by adding white noise to the original signal several times, followed by EMD decomposition, and averaging the EMD decomposed IMFs as the final IMFs20.

CEEMD by adding two Gaussian white noise signals with opposite values to the original signal, which are then subjected to separate EMD decompositions. In ensuring that the decomposition effect is comparable to that of EEMD, CEEMD reduces the reconstruction error induced by the EEMD method. After the original signal x(t) is decomposed by CEEMD, the reconstructed signal can be represented as

$$xleft(tright)=sum_{i=1}^{n}IM{F}_{i}left(tright)+{r}_{n}left(tright)$$

(2)

In Eq.(2), (IM{F}_{i}left(tright)) is the intrinsic modal function component; ({r}_{n}(t)) is the residual term; and n is the number of intrinsic modal components when ({r}_{n}(t)) becomes a monotonic function. The original sequence is finally decomposed into a finite number of IMFs.

In order to accurately predict the runoff sequence, this paper establishes a kernel limit learning machine prediction model based on the kernel function optimised by the nature-inspired BOA algorithm.

In Fig.1, the ELM input weights (omega in {R}^{XY}) (X and Y are the input and hidden layer neural networks, respectively) and biases are randomly generated21. Extreme learning machines require less manual tuning of parameters than BP neural networks, and can be trained on sample data in a shorter period of time, with fast learning rate and strong generalisation ability.

Structure of the KELM model.

Its regression function with output layer weights is:

$$left{begin{array}{c}fleft(xright)=h(x)beta =Hbeta \ {{varvec{H}}}^{T}{left(frac{1}{C}+{varvec{H}}{{varvec{H}}}^{T}right)}^{-1}Tend{array}right.$$

(3)

where: (fleft(xright))-model output; (x) -sample input ({varvec{h}}({varvec{x}})) and ({varvec{H}})-hidden layer mapping matrix; (beta ) -regularisation parameter; T-sample output vector.

Conventional ELM prediction models (solved by least squares) tend to destabilise the output when there is potential covariance in the sample parameters. Therefore, Huang et al.22 used the Kernel Extreme Learning Machine (KELM) with kernel function optimisation. Based on the kernel function principle, KELM can project covariant input samples into a high-dimensional space, which significantly improves the fitting and generalisation ability of the model. In addition, this model does not need to set the number of hidden layer nodes manually, reducing the number of spatial training bits and training time. The model output equation is:

$$fleft(xright)={left[begin{array}{c}K(x,{x}_{1})\ vdots \ K(x,{x}_{N})end{array}right]}^{T}{left(frac{1}{C}+{{varvec{Omega}}}_{ELM}right)}^{-1}$$

(4)

where: K(({x}_{i},{x}_{j}))-kernel function; ({{varvec{Omega}}}_{ELM})-kernel matrix, which is calculated as:

$$left{begin{array}{c}{{varvec{Omega}}}_{ELM}=H{{varvec{H}}}^{T}\ {{{varvec{Omega}}}_{ELM}}_{i,j}=hleft({x}_{i}right)hleft({x}_{j}right)=Kleft({x}_{i},{x}_{j}right)end{array}right.$$

(5)

where: ({x}_{i}) and ({x}_{j})-sample input vectors, i and j are taken as positive integers within [1,N]; K(({x}_{i},{x}_{j}))-kernel function.

KELM determines the implicit layer mapping kernel function in the form of an inner product by introducing a kernel function, and the number of implicit layer nodes does not need to be set; The result is faster model learning and effective improvement of the generalisation ability and stability of the KELM-based runoff prediction model.

Butterfly optimisation algorithm is an intelligent optimisation algorithm derived by simulating butterfly searching for food and mating behaviour23. In the BOA algorithm, each butterfly emits its own unique scent. Butterflies are able to sense the source of food in the air and likewise sense the scent emitted by other butterflies and move with the butterfly that emits a stronger scent, the scent concentration equation is:

where (f)Concentration of scent emitted by the butterfly, (c)Perceived morphology, (l)Stimulus intensity, (a)Power index, taken between [0,1]. When a=1, it means that the butterfly does not absorb the scent, meaning that the scent emitted by a specific butterfly is perceived by the same butterfly; This case is equivalent to a scent spreading in an ideal environment, where the butterfly emitting the scent can be sensed everywhere in the domain, and thus a single global optimum can be easily reached.

In order to prove the above with the search algorithm, the following hypothetical regulations were set up to idealise the characteristics of butterflies: (i) All butterflies can give off some scent, and butterflies attract and exchange information with each other by virtue of the scent. (ii) Butterflies undergo random movements or directional movements towards butterflies with strong scent concentrations.

By defining different fitness functions for different problems, the BOA algorithm can be divided into the following 3 steps:

Step 1: Initialisation phase. Randomly generate butterfly locations in the search space, calculate and store each butterfly location and fitness value.

Step 2: Iteration phase. Multiple iterations are performed by the algorithm, in each iteration the butterflies are moved to a new position in the search space and then their fitness values are recalculated. The adaptation values of the randomly generated butterfly population are sorted to find the best position of the butterfly in the search space.

Step 3: End Phase, In the previous phase, the butterflies move and then use the scent formula to produce a scent in a new location.

The penalty parameter C and the kernel function parameter K in the kernel-limit learning machine are chosen as the searching individuals of the butterfly population, and the BOA-KELM model is constructed to achieve the iterative optimisation of C and K. The specific steps are as follows:

Step 1: Collect runoff data and produce training and prediction sample sets.

Step 2: Initialise the butterfly population searching individuals i.e. penalty parameter C and kernel function parameter K.

Step 3: Initialise the algorithm parameters, including the number of butterfly populations M, the maximum number of iterations .

Step 4: Calculate the fitness value of the individual butterfly population and calculate the scent concentration f. Based on the fitness value, the optimal butterfly location is derived.

Step 5: Check the fitness value of the butterfly population searching individuals after updating their positions, determine whether it is better than before updating, and update the global optimal butterfly position and fitness value.

Step 6:Judge whether the termination condition is satisfied. If it is satisfied, exit the loop and output the prediction result; otherwise, bring in the calculation again.

Step 7:Input the test set into the optimised KELM and output the predictions.

According to the above steps, the corresponding flowchart is shown in Fig.2.

BOA Optimisation KELM Model Flowchart.

In order to improve the accuracy of runoff prediction, this paper designs a runoff prediction framework based on the idea of "decompositionmodeling predictionreconstruction", as shown in Fig.3, and the specific prediction steps are as follows:

VMD-CEEMD-BOA-KELM prediction model framework.

Step 1: Data pre-processing. Anomalies in the original runoff series were processed using the Lajda criterion.

Step 2: VMD-CEEMD decomposition. The raw runoff series was decomposed using the VMD algorithm, and then the data was decomposed quadratically using the CEEMD algorithm to obtain k components.

Step 3: Data preparation. Each component is normalised and divided into a training data set and a test data set.

Step 4: Modelling prediction. A BOA-optimised KELM model is built based on the training dataset for each component and predicted for the test dataset.

Step 5: Reconstruction. The predictions of all components are accumulated to obtain the prediction of the original runoff sequence.

In order to reflect the error and prediction accuracy of the model prediction results more clearly, four indicators, RMSE, MAE, R2, and NSE are used for the analysis, and the equations are calculated as follows:

$${varvec{R}}{varvec{M}}{varvec{S}}{varvec{E}}=sqrt{frac{1}{{varvec{N}}}cdot {sum }_{{varvec{i}}=1}^{{varvec{N}}}{left({{varvec{y}}}_{{varvec{i}}}-{{varvec{y}}}_{{varvec{c}}}right)}^{2}}$$

$${varvec{M}}{varvec{A}}{varvec{E}}=frac{1}{{varvec{N}}}cdot {sum }_{{varvec{i}}=1}^{{varvec{N}}}left|{{varvec{y}}}_{{varvec{i}}}-{{varvec{y}}}_{{varvec{c}}}right|$$

$${{varvec{R}}}^{2}={left[frac{sum left({{varvec{y}}}_{{varvec{i}}}-overline{{{varvec{y}} }_{{varvec{i}}}}right)left({{varvec{y}}}_{{varvec{c}}}-overline{{{varvec{y}} }_{{varvec{c}}}}right)}{sqrt{sum {left({{varvec{y}}}_{{varvec{i}}}-overline{{{varvec{y}} }_{{varvec{i}}}}right)}^{2}}sum {left({{varvec{y}}}_{{varvec{c}}}-overline{{{varvec{y}} }_{{varvec{c}}}}right)}^{2}}right]}^{2}$$

$${varvec{N}}{varvec{S}}{varvec{E}}=1-frac{{sum }_{{varvec{t}}=1}^{{varvec{T}}}{left({{varvec{y}}}_{{varvec{i}}}-{{varvec{y}}}_{{varvec{c}}}right)}^{2}}{{sum }_{{varvec{t}}=1}^{{varvec{T}}}{left({{varvec{y}}}_{{varvec{i}}}-overline{{{varvec{y}} }_{{varvec{i}}}}right)}^{2}}$$

This paper does not contain any studies with human participants or animals performed by any of the authors.

Read the original here:
A runoff prediction method based on hyperparameter optimisation of ... - Nature.com

Read More..

Probabilistic source classification of large tephra producing … – United States Geological Survey (.gov)

Alaska contains over 130 volcanoes and volcanic fields that have been active within the last 2 million years. Of these, roughly 90 have erupted during the Holocene, with many characterized by at least one large explosive eruption. These large tephra-producing eruptions (LTPEs) generate orders of magnitude more erupted material than a typical arc explosive eruption and distribute ash thousands of kilometers from their source. Because LTPEs occur infrequently, and the proximal explosive deposit record in Alaska is generally limited to the Holocene, we require a method that links distal deposits to a source volcano where the correlative proximal deposits from that eruption are no longer preserved. We present a model that accurately and confidently identifies LTPE volcanic sources in the Alaska-Aleutian arc using only in situ geochemistry. The model is a voting ensemble classifier comprised of six conceptually different machine learning algorithms trained on proximal tephra deposits that have had their source positively identified. We show that incompatible trace element ratios (e.g., Nb/U, Th/La, Rb/Sm) help produce a feature space that contains significantly more variance than one produced by major element concentrations, ultimately creating a model that can achieve high accuracy, precision, and recall on predicted volcanic sources, regardless of the perceived 2D data distribution (i.e., bimodal, uniform, normal) or composition (i.e., andesite, trachyte, rhyolite) of that source. Finally, we apply our model to unidentified distal marine tephra deposits in the region to better understand explosive volcanism in the Alaska-Aleutian arc, specifically its pre-Holocene spatiotemporal distribution.

See the rest here:
Probabilistic source classification of large tephra producing ... - United States Geological Survey (.gov)

Read More..

Watch: Peter Jackson discusses the machine learning technology … – TVBEurope

A 12-minute documentary provides insight into Now and Then's creation, and includes both archive and current-day footage

By Jenny Priestley

To mark the release of whats being called the last song from The Beatles, the band has also unveiled a documentary about how the song was put together using 21st-century technology.

The documentary, directed by Oliver Murray with sound design by Alastair Sirkett, provides insight into Now and Thens creation, and includes both archive and current-day footage.

It features an interview with director Peter Jackson, whose WingNut Films used MAL software to extract John Lennons voice from the original cassette recording.

During the course of Get Back we were paying a lot of attention to the technical restoration, explains Jackson.

That ultimately led us to develop a technology which allows us to take any soundtrack and split all the different components into separate tracks based on machine learning.

Jenny has worked in the media throughout her career, joining TVBEurope as editor in 2017. She has also been an entertainment reporter, interviewing everyone from Kylie Minogue to Tom Hanks; as well as spending a number of years working in radio. She continues to appear on radio every week and occasionally pops up on TV.

Continued here:
Watch: Peter Jackson discusses the machine learning technology ... - TVBEurope

Read More..

Predicting Conversion to Psychosis Using Machine Learning: Are … – Am J Psychiatry

In the present issue of the American Journal of Psychiatry, Smucny et al. suggest that predictive algorithms for psychosis using machine learning (ML) methods may already achieve a clinically useful level of accuracy (1). In support of this perspective, these authors report on the results of an analysis using the North American Prodrome Longitudinal Study, Phase 3 (NAPLS3) data set (2), which they accessed through the National Institute of Mental Health Data Archive (NDAR). This is a large multisite study of youth at clinical high risk for psychosis followed up on multiple occasions with clinical, cognitive, and biomarker assessments. Several ML approaches were compared with each other and with Cox (time-to-event) and logistic regression using the clinical, neurocognitive, and demographic features from the NAPLS2 individualized risk calculator (3), with salivary cortisol also tested as an add-on biomarker. When these variables were analyzed using Cox and logistic regression, the model applied to the NAPLS3 cohort attained a level of predictive accuracy comparable to that observed in the original NAPLS2 cohort (overall accuracy in the 66%68% range). However, several ML algorithms produced nominally better results, with a random forest model performing best (overall accuracy in the 90% range). Acknowledging that a predictive algorithm with 90% or higher predictive accuracy will have greater clinical utility than one with substantially lower accuracy, several issues remain to be resolved before it can be determined whether ML methods have attained this utility threshold.

First and foremost, an ML models expected real-world performance can only be ascertained when tested in an independent sample/data set that the model has never before encountered. ML methods are very adept at finding apparent structure in data that predict an outcome, but if that structure is idiosyncratic to the training data set, the model will fail to generalize to other contexts and thus not be useful, a problem known as overfitting (4). Internal cross-validation methods are not sufficient to overcome this problem, since the model sees all the training data at certain points in the process, even if some is left out on a particular iteration (5). Overfitting is indicated by a big drop-off in model accuracy moving from the original internally cross-validated training data set to an external, independent cross-validation test. Smucny et al. (1) acknowledge the need for an external replication test before the utility of the ML models they evaluated using only internal cross-validation methods can be fully appreciated.

Is there likely to be a big drop-off in accuracy of the ML models reported by Smucny et al. (1) when such an external validation test is performed? On one hand, they limited consideration to a small number of features that have previously been shown to predict psychosis in numerous independent samples (i.e., the variables in the NAPLS2 risk calculator [3]). This aspect mitigates the overfitting issue to some extent because the features used in model building are already filtered (based on prior work) to be highly likely to predict conversion to psychosis, both individually and when combined in a regression model. On the other hand, the ML models employed in the study use various approaches to find higher-order interactive and nonlinear amalgamations among this set of feature variables that maximally discriminate outcome groups. This aspect increases the risk of overfitting given that a very large number of such higher-order interactive effects are assessed in model building, with relatively few subjects available to represent each unique permutation, a problem known as the curse of dimensionality (6). Tree-based methods such as the random forest model that performed best in the NAPLS3 data set are not immune from this problem and, in fact, are particularly vulnerable to it when applied on data sets with relatively small numbers of individuals with the outcome of interest (7).

The relatively low base rate of conversion to psychosis (i.e., 10%15%), even in a sample selected to be at elevated risk as in NAPLS3, creates another problem for ML methods; namely, such models can achieve high levels of predictive accuracy in the training data set simply by guessing that each case is a nonconverter. Smucny et al. (1) attempt to overcome this issue using a synthetic approach that effectively up samples the minority class (in this case, converters to psychosis) to the point that it has 50% representation in the synthetic sample (8). Although this approach is very helpful in preventing ML models from defaulting to prediction of a majority class, its use in computing cross-validation performance metrics is likely to be highly misleading, given that real-world application of the model is not likely to occur in a context in which there is a 50:50 rate of future converters and nonconverters. Rather, the model will be applied in circumstances in which new clinical high risk (CHR) individuals likelihoods of conversion are computed, and those CHR individuals will derive from a population in which the base rate of conversion is 15%. It is now well established that the same predictive model will result in different risk distributions (and, thereby, different thresholds in model-predicted risk for making binary predictions) in samples that vary in base rates of conversion to psychosis (9). Given this, a 90% predictive accuracy of an ML algorithm in a synthetically derived sample in which the base rate of psychosis conversion is artificially created to be 50% is highly unlikely to generalize to an independent, real-world CHR sample, at least as ascertained using current approaches.

When developing the NAPLS2 risk calculator, the investigators made purposeful decisions to allow the resulting algorithm to be applied validly in scaling the risk of newly ascertained CHR individuals (3). Key among these decisions was to avoid using the NAPLS2 data set to test different possible models, which would then necessitate an external validation test. Rather, a small number of predictor variables was chosen based on their empirical associations with conversion to psychosis in prior studies, and Cox regression was employed to generate an additive multivariate model of predicted risk (i.e., no interactive or non-linear combinations of the variables were included). As a result, the ratio of converters to predictor variables was 10:1 (helping to create adequate representation of the scale values of each predictor in the minority class), and there was no need to use a synthetic sampling approach given that Cox regression is well suited for prediction of low base rate outcomes. The predictor variables chosen for inclusion are ones that are easily ascertained in standard clinical settings and have a high level of acceptability (face validity) for use in clinical decision making. It is important to note that the NAPLS2 model has been shown to replicate (in terms of area under the curve or concordance index) when applied to multiple external independent data sets (10).

Nevertheless, two issues continue to limit the utility of the NAPLS2 risk calculator. One is that it will generate differently shaped risk distributions on samples that vary in conversion risk and in distributions of the individual predictor variables, making it problematic to apply the same threshold of predicted risk for binary predictions across samples that differ in these ways (9, 11). However, it appears possible to derive comparable prediction metrics across samples with differing conversion risks when considering the relative recency of onset or worsening of attenuated positive symptoms at the baseline assessment (11). A more recent onset or worsening of attenuated positive symptoms defines a subgroup of CHR individuals with a higher average predicted risk and higher overall transition rate and in whom particular putative illness mechanisms, in this case an accelerated rate of cortical thinning (12), appear to be differentially relevant (11).

The second rate-limiting issue for the utility of the NAPLS2 risk calculator is that its performance in terms of sensitivity, specificity, and balanced accuracy, even when accounting for recency of onset of symptoms, is still in the 65%75% range. Although ML methods represent one approach that, if externally validated, could conceivably result in predictive models at the 90% or higher level of accuracy, such models would continue to have the disadvantage of being relatively opaque (black box) in terms of how the underlying predictor variables aggregate in defining risk and for that reason may not be used as readily in clinical practice. Alternatively, it may be possible to rely on more transparent analytic approaches to achieve the needed level of accuracy. It has recently been demonstrated that integrating information on short-term (baseline to 2-month follow-up) change on a single clinical variable (e.g., deterioration in odd behavior/appearance) improves the performance of the NAPLS2 risk calculator to >90% levels of sensitivity, specificity, and balanced accuracy; i.e., a range that would support its use in clinical trial design and clinical decision-making (13). Importantly, although the Cox regression model aspect of this algorithm has been externally validated, the incorporation of short-term clinical change (via mixed effects growth modeling) requires replication in an external data set.

Smucny et al. (1) are to be congratulated on a well-motivated and well-executed analysis of the NAPLS3 data set. It is heartening to see such creative uses of this unique shared resource for our field bear fruit, reinforcing the value of open science. As we move forward toward the time and place in which prediction models of psychosis and related outcomes have utility for clinical decision making, whether those models rely on machine learning methods or more traditional approaches, it will be crucial to insist on external validation of results before deciding that we are, in fact, there.

Clark L. Hull Professor of Psychology and Professor of Psychiatry, Yale University, New Haven, Conn.

Dr. Cannon reports no financial relationships with commercial interests.

1. Smucny J, Davidson I, Carter CS: Are we there yet? Predicting conversion to psychosis using machine learning. Am J Psychiatry 2023; 180:836840 Abstract,Google Scholar

2. Addington J, Liu L, Brummitt K, et al.: North American Prodrome Longitudinal Study (NAPLS 3): methods and baseline description. Schizophr Res 2022; 243:262267Crossref, Medline,Google Scholar

3. Cannon TD, Yu C, Addington J, et al.: An individualized risk calculator for research in prodromal psychosis. Am J Psychiatry 2016; 173:980988Link,Google Scholar

4. Cawley GC, Talbot NLC: On over-fitting in model selection and subsequent selection bias in performance evaluation. J Mach Learn Res 2010; 11:20792107 Google Scholar

5. Arlot S, Celisse A: A survey of cross-validation procedures for model selection. Statist Surv 2010; 4:4079 Crossref,Google Scholar

6. Hughes G: On the mean accuracy of statistical pattern recognizers. IEEE Trans Inf Theor 1968; 14:5563 Crossref,Google Scholar

7. Peng Y, Nagata MH: An empirical overview of nonlinearity and overfitting in machine learning using COVID-19 data. Chaos Solitons Fractals 2020; 139:110055Crossref, Medline,Google Scholar

8. Chawla NV, Bowyer KW, Hall LO, et al.: SMOTE: Synthetic Minority Over-Sampling Technique. J Artif Intell Res 2002; 16:321357 Crossref,Google Scholar

9. Koutsouleris N, Worthington M, Dwyer DB, et al.: Toward generalizable and transdiagnostic tools for psychosis prediction: an independent validation and improvement of the NAPLS-2 risk calculator in the multisite PRONIA cohort. Biol Psychiatry 2021; 90:632642Crossref, Medline,Google Scholar

10. Worthington MA, Cannon TD: Prediction and prevention in the clinical high-risk for psychosis paradigm: a review of the current status and recommendations for future directions of inquiry. Front Psychiatry 2021; 12:770774Crossref, Medline,Google Scholar

11. Worthington MA, Collins MA, Addington J, et al.: Improving prediction of psychosis in youth at clinical high-risk: pre-baseline symptom duration and cortical thinning as moderators of the NAPLS2 risk calculator. Psychol Med 2023:19Crossref, Medline,Google Scholar

12. Collins MA, Ji JL, Chung Y, et al.: Accelerated cortical thinning precedes and predicts conversion to psychosis: the NAPLS3 longitudinal study of youth at clinical high-risk. Mol Psychiatry 2023; 28:11821189Crossref, Medline,Google Scholar

13. Worthington MA, Addington J, Bearden CE, et al.: Dynamic prediction of outcomes for youth at clinical high risk for psychosis: a joint modeling approach. JAMA Psychiatry 2023:e232378Google Scholar

Read the original here:
Predicting Conversion to Psychosis Using Machine Learning: Are ... - Am J Psychiatry

Read More..