Cloud Hosting

Recent achievements in supervised tasks of deep learning can be attributed to the availability of large amounts of labeled training data. Yet it takes a lot of effort and money to collect accurate labels. In many practical contexts, only a small fraction of the training data have labels attached. Semi-supervised learning (SSL) aims to boost model performance using labeled and unlabeled input. Many effective SSL approaches, when applied to deep learning, undertake unsupervised consistency regularisation to use unlabeled data.

State-of-the-art consistency-based algorithms typically introduce several configurable hyper-parameters, even though they attain excellent performance. For optimal algorithm performance, it is common practice to tune these hyper-parameters to optimal values. Unfortunately, hyper-parameter searching is often unreliable in many real-world SSL scenarios, such as medical image processing, hyper-spectral image classification, network traffic recognition, and document recognition. This is because the annotated data are scarce, leading to high variance when cross-validation is adopted. Having algorithm performance sensitive to hyper-parameter values makes this issue even more pressing. Moreover, the computational cost may become unmanageable for cutting-edge deep learning algorithms as the search space grows exponentially concerning the number of hyper-parameters.

Researchers from Tsinghua University introduced a meta-learning-based SSL algorithm called Meta-Semi to leverage the labeled data more. Meta-Semi achieves outstanding performance in many scenarios by adjusting just one more hyper-parameter.

The team was inspired by the realization that the network may be trained successfully using the appropriately pseudo-labeled unannotated examples. Specifically, during the online training phase, they produce pseudo-soft labels for the unlabeled data based on the network predictions. Next, they remove the samples with unreliable or incorrect pseudo labels and use the remaining data to train the model. This work shows that the distribution of correctly pseudo-labeled data should be comparable to that of the labeled data. If the network is trained with the former, the final loss on the latter should also be minimized.

They defined the meta-reweighting objective to minimize the final loss on the labeled data by selecting the most appropriate weights (weights throughout the paper always refer to the coefficients used to reweight each unlabeled sample rather than referring to the parameters of neural networks). The researchers encountered computing difficulties when tackling this problem using optimization algorithms.

For this reason, they suggest an approximation formulation from which a closed-form solution can be derived. Theoretically, they demonstrate that each training iteration only needs a single meta gradient step to achieve the approximate solutions.

In conclusion, they suggest a dynamic weighting approach to reweight previously pseudo-labeled samples with 0-1 weights. The results show that this approach eventually reaches the stationary point of the supervised loss function. In popular image classification benchmarks (CIFAR-10, CIFAR-100, SVHN, and STL-10), the proposed technique has been shown to perform better than state-of-the-art deep networks. For the difficult CIFAR-100 and STL-10 SSL tasks, Meta-Semi gets much higher performance than state-of-the-art SSL algorithms like ICT and MixMatch and obtains somewhat better performance than them on CIFAR-10. Moreover, Meta-Semi is a useful addition to consistency-based approaches; incorporating consistency regularisation into the algorithm further boosts performance.

According to the researchers, Meta-Semi requires a little more time to train is a drawback. They plan to look into this issue in the future.

Check out thePaperandReference Article.All Credit For This Research Goes To the Researchers on This Project. Also,dont forget to joinour 15k+ ML SubReddit,Discord Channel,andEmail Newsletter, where we share the latest AI research news, cool AI projects, and more.

Tanushree Shenwai is a consulting intern at MarktechPost. She is currently pursuing her B.Tech from the Indian Institute of Technology(IIT), Bhubaneswar. She is a Data Science enthusiast and has a keen interest in the scope of application of artificial intelligence in various fields. She is passionate about exploring the new advancements in technologies and their real-life application.

Read the original post:
Researchers From Tsinghua University Introduce A Novel Machine Learning Algorithm Under The Meta-Learning Paradigm - MarkTechPost

Read More..

Lemmatization is one of the most common text pre-processing techniques used in natural language processing (NLP) and machine learning in general. Lemmatization is not that much different than the stemming of words in NLP. In both stemming and lemmatization, we try to reduce a given word to its root word. The root word is called a stem in the stemming process, and its called a lemma in the lemmatization process. But there are a few more differences to the two than that. Lets see what those are.

Lemmatization is a text pre-processing technique used in natural language processing (NLP) models to break a word down to its root meaning to identify similarities. For example, a lemmatization algorithm would reduce the word better to its root word, or lemme, good.

In stemming, a part of the word is just chopped off at the tail end to arrive at the stem of the word. There are different algorithms used to find out how many characters have to be chopped off, but the algorithms dont actually know the meaning of the word in the language it belongs to. In lemmatization, the algorithms do have this knowledge. In fact, you can even say that these algorithms refer to a dictionary to understand the meaning of the word before reducing it to its root word, or lemma.

So, a lemmatization algorithm would know that the word better is derived from the word good, and hence, the lemme is good. But a stemming algorithm wouldnt be able to do the same. There could be over-stemming or under-stemming, and the word better could be reduced to either bet, or bett, or just retained as better. But there is no way in stemming that can reduce better to its root word good. This is the difference between stemming and lemmatization.

More on Machine Learning: An Introduction to Classification in Machine Learning

As you can probably tell by now, the obvious advantage of lemmatization is that it is more accurate than stemming. So, if youre dealing with an NLP application such as a chat bot or a virtual assistant, where understanding the meaning of the dialogue is crucial, lemmatization would be useful. But this accuracy comes at a cost.

Because lemmatization involves deriving the meaning of a word from something like a dictionary, its very time consuming. So most lemmatization algorithms are slower compared to their stemming counterparts. There is also a computation overhead for lemmatization, however, in most machine learning problems, computational resources are rarely a cause of concern.

More on Machine Learning: Multiclass Classification With an Imbalanced Data Set

Well, I cant answer that question. Lemmatization and stemming are both much more complex than what Ive made them appear here. There are a lot more things to consider about both the approaches before making a decision. But Ive rarely seen any significant improvement in efficiency and accuracy of a product that uses lemmatization over stemming. In most cases, at least according to my knowledge, the overhead that lemmatization demands is not justified. So, it depends on the project in question. But I want to put out a disclaimer here. Most of the work I have done in NLP is for text classification, and thats where I havent seen a significant difference. There are applications where the overhead of lemmatization is perfectly justified, and in fact, lemmatization would be a necessity.

Read the original here:
Lemmatization in NLP and Machine Learning - Built In

Read More..

by Tsinghua University Press

Meta-Semi trains deep networks using pseudo-labeled samples whose gradient directions are similar to labeled samples. Algorithm 1 shows the Meta Semi pseudo code. The Meta-Semi algorithm outperforms state-of-the-art semi-supervised learning algorithms. Credit: CAAI Artificial Intelligence Research, Tsinghua University Press

Deep learning based semi-supervised learning algorithms have shown promising results in recent years. However, they are not yet practical in real semi-supervised learning scenarios, such as medical image processing, hyper-spectral image classification, network traffic recognition, and document recognition.

In these types of scenarios, the labeled data is scarce for hyper-parameter search, because they introduce multiple tunable hyper-parameters. A research team has proposed a novel meta-learning based semi-supervised learning algorithm called Meta-Semi, that requires tuning only one additional hyper-parameter. Their Meta-Semi approach outperforms state-of-the-art semi-supervised learning algorithms.

The team published their work in the journal CAAI Artificial Intelligence Research.

Deep learning, a machine learning technique where computers learn by example, is showing success in supervised tasks. However, the process of data labeling, where the raw data is identified and labeled, is time-consuming and costly. Deep learning in supervised tasks can be successful when there is plenty of annotated training data available. Yet in many real-world applications, only a small subset of all the available training data are associated with labels.

"The recent success of deep learning in supervised tasks is fueled by abundant annotated training data," said Gao Huang, associate professor with the Department of Automation at Tsinghua University. However, the time-consuming, costly collection of precise labels is a challenge researchers have to overcome. "Meta-semi, as a state-of-the-art semi-supervised learning approach, can effectively train deep models with a small number of labeled samples," said Huang.

With the research team's Meta-Semi classification algorithm, they efficiently exploit the labeled data, while requiring only one additional hyper-parameter to achieve impressive performance under various conditions. In machine learning, a hyper-parameter is a parameter whose value can be used to direct the learning process.

"Most deep learning based semi-supervised learning algorithms introduce multiple tunable hyper-parameters, making them less practical in real semi-supervised learning scenarios where the labeled data is scarce for extensive hyper-parameter search," said Huang.

The team developed their algorithm working from the assumption that the network could be trained effectively with the correctly pseudo-labeled unannotated samples. First they generated soft pseudo labels for the unlabeled data online during the training process based on the network predictions.

Then they filtered out the samples whose pseudo labels were incorrect or unreliable and trained the model using the remaining data with relatively reliable pseudo labels. Their process naturally yielded a meta-learning formulation where the correctly pseudo-labeled data had a similar distribution to the labeled data. In their process, if the network is trained with the pseudo-labeled data, the final loss on the labeled data should be minimized as well.

The team's Meta-Semi algorithm achieved competitive performance under various conditions of semi-supervised learning. "Empirically, Meta-Semi outperforms state-of-the-art semi-supervised learning algorithms significantly on the challenging semi-supervised CIFAR-100 and STL-10 tasks, and achieves competitive performance on CIFAR-10 and SVHN," said Huang.

CIFAR-10, STL-10, and SVHN are datasets, or collections of images, that are frequently used in training machine learning algorithms. "We also show theoretically that Meta-Semi converges to the stationary point of the loss function on labeled data under mild conditions," said Huang. Compared to existing deep semi-supervised learning algorithms, Meta-Semi requires much less effort for tuning hyper-parameters, but achieves state-of-the-art performance on the four competitive datasets.

Looking ahead to future work, the research team's aim is to develop an effective, practical and robust semi-supervised learning algorithm. "The algorithm should require a minimal number of data annotations, minimal efforts of hyper-parameter tuning, and a minimized training time. To attain this goal, our future work may focus on reducing the training cost of Meta-Semi," said Huang.

More information: Yulin Wang et al, Meta-Semi: A Meta-Learning Approach for Semi-Supervised Learning, CAAI Artificial Intelligence Research (2023). DOI: 10.26599/AIR.2022.9150011

Provided by Tsinghua University Press

See the original post:
'Meta-Semi' machine learning approach outperforms state-of-the-art algorithms in deep learning tasks - Tech Xplore

Read More..

How machine learning is used in portfolio management?

Artificial Intelligence & Machine Learning

Introduction

Modern Portfolio Theory was first introduced by Harry Markowitz in 1952. The groundbreaking investment theory demonstrated that the performance of an individual stock is not as important as the performance of an entire portfolio. The interpretations of his article on Portfolio Selection [1], in The Journal of Finance, taught investors that risk, and not the best price, should be the cornerstone of a portfolio [2]. Once an investors risk tolerance has been established, building a portfolio around it is streamlined.

The goal of portfolio management is to choose the best strategy to maximize returns on investment, ensure portfolio flexibility or optimize risk. Attaining such objectives involve the optimal allocation of investment funds, optimization of risk factors, considering financial ratios (such as the Sharpe Ratio) and much more. [3]

Such models help portfolio managers with trade execution, data parsing, idea generation & pattern recognition, alpha factor design, asset allocation, position sizing, strategy testing, and eliminates human biases.

Our goal of this project is to identify an efficient Machine Learning architecture that will replicate the stock basketstrend, almost perfectly. The selected ML model will have superior prediction capabilities and will adapt to the directional changes in prices accurately. Thus, the ML model will help us predict the expected stock returns. Additionally, using the past data we can calculate the standard deviations and gauge the risk.

Now, these stocks will be parsed through various portfolio optimization techniques. To identify the correct optimization technique, in-depth research on various techniques will be carried out. This will produce an optimized portfolio with weightage of each stock.

Literature Review & Model Implementation

1) Meanvariance portfolio optimization using machine learning-based stock price prediction (Wei Chen, Haoyu Zhang, Mukesh Kumar Mehlawat, Lifen Jia) [5]

The paper presents a hybrid model based on machine learning for stock prediction and mean variance (MV) model for portfolio selection. The model follows two stages:

Stock Prediction: Combines eXtreme Gradient Boosting (XGBoost) with an improved firefly algorithm (IFA). XGBoost is an improved gradient boosted decision tree and is composed of multiple decision trees. It is, therefore, a suitable classifier for the financial markets. The accuracy of prediction is calculated using mean absolute percentage error (MAPE), mean square error (MSE), mean absolute error (MAE), and root mean square error (RMSE).

Portfolio Selection: Stocks with higher potential returns are selected according to each stocks predicted prices, and the MV model is employed for allocating the investment proportion of the portfolio. The portfolio is evaluated on its annualized mean return, annualized standard deviation, annualized Sharpe ratio, and annualized Sortino ratio (variation of Sharpe ratio considering the standard deviation of only negative portfolio returns instead of the total standard deviation.)

The paper concludes that ML portfolio management pricing methods outperform the benchmarks concerning accuracy, potency, and efficiency.

Methodology

We predicted the stock price of three companies using ARIMA, LSTM-RNN, and Transformer architectures. The accuracy of prediction is calculated using root mean square error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE), and Profit & Loss (PNL) rubrics.

a) Terminology

ARIMA[6][7] : AutoRegressive Integrated Moving Average is a time series forecasting model that incorporates autocorrelation measures to model non-stationary data to predict future values. The autocorrelation part of the model measures the dependency of a signal with a delayed copy of itself as a function. ARIMA models are capable of capturing temporal structures in time-series data. This model uses three parameters

LSTM-RNN[8] : Long Short Term Memory based Recurrent Neural Network. RNN is a generalization of feedforward neural network that has an internal memory. As its name suggests, it is recurrent in nature, i.e., it performs the same function for every input of data while the output of the current input depends on the past one computation. Once the output is generated, it is replicated and sent back to the RNN. Decisions are based on the current input and the previous step learnt output. LSTM is a modified version of RNN which makes it easier to remember past data in memory.

Transformer[9] : it is a neural network architecture that uses a self-attention mechanism, allowing the model to focus on the relevant parts of the time-series to improve prediction qualities. This consists of Single-head Attention and Multi-head Attention. The self-attention mechanism searches for inputs sequences, for each input, and adds result to the output sequence. These processes are parallelized allowing acceleration of the learning process.

b) Data

We have incorporated the daily OHLCV (Open, High, Low, Close, Volume) data of three companies Dr Reddys Laboratories Ltd, Wipro Limited, Reliance Steel & Aluminum Co for a time frame of 252 days, from 16Sept20 till 15Sept21. The data has been extracted & stored locally from NSE website. The data set incorporated is divided into a ratio of 7:3, where 70% of it will be the training data and the remaining for testing. For this project, we will ignore the top and bottom 5% of values, as they may be outliers.

c) Predicting Stocks

ARIMA: We initially scaled our features to avoid intensive computation. We imported the ARIMA module from statsmodels.tsa.arima.model to train our model. The ARIMA models uses the first 180 rows of the dataset as training data, with parameters p=2, d=1, q=0. These learnt results are utilized to predict the close price of the further rows of dataset (row number 180 onwards)

LSTM- RNN: We are developing a neural network regressor for continuous value prediction using LSTM. We initialize the same. Now, we add the first layer with the Dropout layer. The LSTM layer has 250 neurons that will capture the model trend. Since we need to add another layer to the model, our return_sequence is set to True.

The input_shape corresponds to the umber of time stamps and indicators. For the Dropout layer, a value of 0.2 denotes that 20% of the 250 neurons will be ignored. This is repeated for layers 2,3, and 4. Finally a one-dimension output layer is created. This layer is one dimensional as we are only predicting one price each time. We now compile our code using Adam optimizer and loss as mean square error. We then fit our model.

Fig 4: LSTM-RNN Implementation (File Name: rnn.py)

Transformer: To encode the notion of time into the model, time encoders have been incorporated in the code (class Time2Vector). The time encoder comprises of two main ideas periodic component (ReLU function) & non-periodic component (linear function). The class Time2Vector encompasses two functions build (initialize matrices) and call (calculates the periodic and linear time features). We now initialize the three Transformer Encoder layers Single Head Attention, Multi Head Attention, and Transformer Encoder layer.

(Note: Snippet of this is not in the report. Kindly refer attached python files to view them.) After these initializations, the training process begins, which is followed by testing the data. The parameters used are Batch Size: 12, Sequence Length: 3, and Size of model input: 5.

The stock_prediction.py imports all these models and returns the final outputs depending on the stock ticker symbol entered.

2) Alternative Approaches to Meanvariance Optimization (Maria Debora Braga) [10] This paper illustrates two examples risk-based asset allocation strategies: the global minimum variance strategy; and the optimal risk parity strategy.

Global Minimum-Variance Strategy the asset manager recommends a portfolio already known by the reader. It is the portfolio at the leftmost end of the efficient frontier, which, given the location, is the portfolio with the smallest attainable ex ante standard deviation. It is not necessary to implement Mean-Variance Optimization to identify the global minimum-variance portfolio (GMVP). It can be replaced with a simplified optimization algorithm with the formula for portfolio variance as an objective function to be minimized and the inclusion of the traditional long-only and budget constraints. Therefore, the optimization problem to be solved to identify the GMVP can be written formally as follows:

Optimal Risk Parity Strategy aims to overcome the problem of risk concentration. The alternative designation of optimal risk parity as equally weighted risk contribution strategy or portfolio suggests the criteria used to structure the portfolio coherently with the goal: to give each asset class a weight so that the amount of risk it contributes to overall portfolio risk is equal to the amount contributed by any other asset class in the portfolio. The above condition can become written formally as equality among component risks. Moreover, defined as the product of asset class weight in the portfolio and its marginal contribution to risk.

The identification of optimal asset class weights for a risk parity portfolio involves solving an optimization problem which is different from mean-variance optimization. More precisely, it includes a new objective function to be minimized together with the traditional constraints on portfolio weights.

portfolio theory that assumes that investors will make rational decisions about investments if they have complete information. A major assumption here is that investors seek low risk and high reward. The two main components of mean-variance analysis are as follows:

Variance Represents how spread out the numbers are in a set

Expected Return Probability expressing the estimated return of the investment in the security

If two different securities have the same expected return, but one has lower variance, the one with lower variance is the better pick. Similarly, if two different securities have approximately the same variance, the one with the higher return is the better pick.

Results & Conclusion

From our results of LSTM-RNN, and the Transformer when compared to the ARIMA benchmark, our conclusion is that the LSTM is better at getting the trends to almost replicate the original trend, but it does not react to directional changes as good as the Transformer which does not score so well when it comes to closeness to the original stock trend.

Any stock prediction model cannot provide greater accuracy without increasing the complexity of prediction operations exponentially. The key to optimizing a stock forecasting system is balancing accuracy and computational expense.

The rationale behind the use of ML is not just about novel architectures that revolutionizes forecasting and time-series prediction, but also the way each use case should be evaluated in terms of the domain of the use case itself. Referring to this specific use case, metrics are surely an evaluation factor, but it is the PNL that gives deeper insight into the accuracy of the model to predict changes based on the trends and patterns that are of significance and perform the best when looked at the big picture.

Identification of correct ML architecture in Active Portfolio Management is cardinal because the stock predicted expected returns will be the base for stock selection for portfolio optimization.

Also, from our research, we conclude that risk-based optimization techniques are superior to the traditional mean-variance technique. When the risk-based strategies become considered, the portfolio selection phase undergoes changes because the strategy comes up to a single risky portfolio proposal. Therefore, an investor does not have to choose an optimal point on the efficient frontier according to their risk tolerance/aversion and investment objective.

The decision-making process is therefore analogous to the Capital Asset Pricing Model. In CAPM, the same risky portfolio is offered to all the investors. A point is then selected on the Capital Market line. This is the line, in the risk-return space, that originates from the risk-free rate and is tangent to the efficient frontier. The tangency portfolio is also the portfolio with the maximum Sharpe ratio.

Written by Varun Chandra Gupta

References & Citations

[1] Portfolio Selection, Wiley Online Library [Online]

Available: https://onlinelibrary.wiley.com/doi/full/10.1111/j.1540-6261.1952.tb01525.x [Accessed Nov. 03, 2022]

[2] Portfolio Management, Investopedia

Available: https://www.investopedia.com/articles/07/portfolio-history.asp [Accessed Nov. 03, 2022]

[3] Portfolio Management, groww.in. [Online].

Available: https://groww.in/p/portfolio-management [Accessed Oct. 07, 2022]

[4] Derek Snow, Machine Learning in Asset ManagementPart 1: Portfolio ConstructionTrading Strategies in The Journal of Financial Data Science Winter 2020, [Online Document], Available: https://jfds.pm-research.com/content/2/1/10 [Accessed Nov. 03, 2022]

[5] Wei Chen, Haoyu Zhang, Mukesh Kumar Mehlawat and Lifen Jia, Meanvariance portfolio optimization using machine learning-based stock price prediction, ScienceDirect, [Online Document], Available:

https://reader.elsevier.com/reader/sd/pii/S1568494620308814?token=072C0F96416E333E05C4C88C 0F3EF956F66CB2B591F5865147EF13701FFC6CEB96D70EA6ADA889440EE8F47A7D90D1E6& originRegion=us-east-1&originCreation=20221007063440 [Accessed Oct. 07, 2022]

[6] ARIMA Model, ProjectPro [Online]

Available: https://www.projectpro.io/article/how-to-build-arima-model-inpython/544#:~:text=Model%20in%20Python%3F-,ARIMA%20Model%2D%20Complete%20Guide%20to%20Time%20Series%20Forecasting%20in% 20Python,data%20to%20predict%20future%20values [Accessed Nov. 03, 2022]

[7] ARIMA Model, TowardsDataScience [Online]

Available: https://towardsdatascience.com/time-series-forecasting-predicting-stock-prices-using-an arima-model-2e3b3080bd70 [Accessed Nov. 03, 2022]

Available: https://towardsdatascience.com/lstm-for-google-stock-price-prediction-e35f5cc84165 [Accessed Nov. 03, 2022]

[9] Stock Predictions with state-of-the-art Transformers and Time Embeddings, TowardsDataScience [Online]

Available: https://towardsdatascience.com/stock-predictions-with-state-of-the-art-transformer-and time-embeddings-3a4485237de6 [Accessed Nov. 03, 2022]

[10] Maria Debora Braga, Alternative Approaches to Traditional Mean-Variance Optimisation, SpringerLink, [Online Document], Available: https://link.springer.com/chapter/10.1007/978-3-319- 32796-9_6 [Accessed Nov. 03, 2022]

[11] Michael Pinelis, David Ruppert, Machine learning portfolio allocation, ScienceDirect, [Online Document], Available:

https://reader.elsevier.com/reader/sd/pii/S2405918821000155?token=E142E1AA4431A9B55ECF1C2 ABA5CC7E52CA69D5E1C728796D64FEC02BCFB0E3ABEEFAA56E0AF1E431BAE0CE7C041C E5E&originRegion=us-east-1&originCreation=20221007052506 [Accessed Oct. 07, 2022]

[12] Jrn Sass, Anna-KatharinaThs, Risk reduction and portfolio optimization using clustering methods, ScienceDirect, [Online Document], Available:https://reader.elsevier.com/reader/sd/pii/S2452306221001416?token=5E952856364B9133BB345373 B3373F4A2A5D62EAC2886F34758D28024D01C053253082DAC3408677BB6E425ECB264C6F& originRegion=us-east-1&originCreation=20221007063710 [Accessed Oct. 07, 2022]

How machine learning is used in portfolio management?

Follow this link:
How machine learning is used in portfolio management? - Rebellion Research

Read More..

A Hong Kong University of Science and Technology- and Samsung Medical Center-led team reporting in Genome Medicine outlines molecular features linked to chemotherapy resistance in forms of glioblastoma (GBM) containing wild-type isocitrate dehydrogenase (IDH-wt) genes. Using exome sequencing, GliomaSCAN targeted panel sequencing, and/or RNA sequencing combined with machine learning (ML), the researchers characterized molecular features found in patient-derived glioma stem-like cells (GSCs) originating from 29 temozolomide chemotherapy-resistant IDH-wild type GBM cases and 40 IDH-wild type GBM cases that were susceptible to the adjuvant chemotherapy treatment. In the process, they came up with a combined tumor signature coinciding with temozolomide chemotherapy response, highlighting a handful of genes with enhanced expression in the temozolomide-resistant cells, along with other chemotherapy resistance- or sensitivity-related features. With multisector temozolomide screening, meanwhile, the authors show that chemotherapy response can vary within tumor samples from the same individual. "We identified molecular characteristics associated with [temozolomide] sensitivity, and illustrate the potential clinical value of a ML model trained from pharmacogenomic profiling of patient-derived GSC against IDH-wt GBMs," the authors write.

View original post here:
Genomic Analysis, Machine Learning Leads to Chemotherapy ... - GenomeWeb

Read More..