Cloud Hosting

The performance of deep neural networks (DNNs) relies heavily on their structures, and designing a good structure (aka architecture) tends to require extensive effort from human experts. The idea of an automatic structure-learning algorithm that can achieve performance on par with the best human-designed structures is thus increasingly appealing to machine learning researchers.

In the paper Learning Structures for Deep Neural Networks, a team from OneFlow and Microsoft explores unsupervised structure learning, leveraging the efficient coding principle, information theory and computational neuroscience to design a structure learning method that does not require labelled information and demonstrates empirically that larger entropy outputs in a deep neural network lead to better performance.

The researchers start with the assumption that the optimal structure of neural networks can be derived from the input features without labels. Their study probes whether it is possible to learn good DNN network structures from scratch in a fully automatic fashion, and what would be a principled way to reach this end.

The team references a principle borrowed from the biological nervous system domain the efficient coding principle which posits that a good brain structure forms an efficient internal representation of external environments. They apply the efficient coding principle to DNN architecture, proposing that the structure of a well-designed network should match the statistical structure of its input signals.

The efficient coding principle suggests that the mutual information between a models inputs and outputs should be maximized, and the team presents a solid Bayesian optimal classification theoretical foundation to support this. Specifically, they show that the top layer of any neural network (softmax linear classifier) and the independency between the nodes in the top hidden layer constitute a sufficient condition for making the softmax linear classifier act as a Bayesian optimal classifier. This theoretical foundation not only backs up the efficient coding principle, it also provides a way to determine the depth of a DNN.

The team then investigates how to leverage the efficient coding principle in the design of a structure-learning algorithm, and shows that sparse coding can implement the principle under the assumption of zero-peaked and heavy-tailed prior distributions. This suggests that an effective structure learning algorithm can be designed based on global group sparse coding.

The proposed structure-learning with sparse coding algorithm learns a structure layer by layer in a bottom-up manner. The raw features are at layer one, and given the predefined number of nodes in layer two, the algorithm will learn the connection between these two layers, and so on.

The researchers also describe how this proposed algorithm can learn inter-layer connections, handle invariance, and determine DNN depth. Finally, they conduct intensive experiments on the popular CIFAR-10 data set to evaluate the classification accuracies of their proposed structure learning method, the role of inter-layer connections, and the role of structure masks and network depth.

The results show that a learned-structure single-layer network achieves an accuracy of 63.0 percent, outperforming the single-layer baseline of 60.4 percent. In an inter-layer connection density evaluation experiment, the structures generated by the sparse coding approach outperform random structures, and at the same density level, always outperform the sparsifying-restricted Boltzmann machines (RBM) baseline. In the teams structure mask role evaluation, the structure prior provided by sparse coding is seen to improve performance. The network depth experiment meanwhile empirically justifies the proposed approach for determining DNN depth via coding efficiency.

Overall, the research proves the efficient coding principles effectiveness for unsupervised structure learning, and that the proposed global sparse coding-based structure-learning algorithms can achieve performance comparable with the best human-designed structures.

The paper Learning Structures for Deep Neural Networks is on arXiv.

Author: Hecate He |Editor: Michael Sarazen, Chain Zhang

We know you dont want to miss any news or research breakthroughs.Subscribe to our popular newsletterSynced Global AI Weeklyto get weekly AI updates.

Like Loading...

View post:
Microsoft & OneFlow Leverage the Efficient Coding Principle to Design Unsupervised DNN Structure-Learning That Outperforms Human-Designed...

Transgender inclusion at work is a topic that has received more attention in recent years, but organisations still have a long way to go to achieve a long-term vision of change.

A proactive approach to inclusion is critical to surmount the serious barriers that stifle transgender equality at work. There is a deep and broad knowledge gap regarding transgender issues even within the HR and L&D ranks.

Indeed, transgender is one of the most misunderstood identity aspects in the social realm, especially the workplace. Transgender individuals present a gender identity that is different from their gender assigned at birth. Yet there are a multitude of articulations in which transgender identity can take shape. As such, transgender is often referred to as an umbrella term, which encompasses a vast spectrum of minority gender identity expressions.

For example, a transgender individual may be someone who undertakes hormonal and/or surgical interventions to achieve a destination gender identity. Equally, some transgender people adopt a new gender identity by means of non-medical steps, such as changes in everyday dress, speech or manners.

In addition, there are transgender individuals who express a non-binary gender identity by altogether eschewing the conventional categories of male and female. Whats more, transgender status does not connote a particular sexual orientation, so transgender people can be gay, lesbian, bisexual, straight or pansexual.

The complexity of transgender is often glossed over by the uncritical use of the label LGBT in both HR practice and academic research, which sometimes conflates not only sexual orientation and gender identity, but the distinct needs of different groups.

While it is true that transgender people may have overlapping concerns with LGB cis-individuals, they also have unique needs that must be addressed in their own right. For example, many transgender people undergo transition the complex process of re-configuring an existing gender identity in ways that more closely match a desired gender identity.

In addition, transgender individuals are much more profoundly stigmatised than LGB cis men and cis women. Transgender is viewed in society as a threat to gender, which operates as the most fundamental organising principle of social and organisation life.

Through conducting research on gender identity inclusion in the workplace, in collaboration with Professor Ahu Tatli from Queen Mary University of London, three key challenges faced by transgender employees came to light.

Our qualitative research based on in-depth interviews with transgender employees in the UK shows disturbing trends that must be addressed swiftly and decisively (Ozturk and Tatli, 2016).

First, transgender employees have a severe under-representation problem in UK workplaces. Many organisations view gender identity diversity as too far a goal to advance. Transgender workers are implicitly understood as incongruent with professional work contexts as well as stereotypically gendered lines of work.

As a result, transgender workers are severely discriminated against at the recruitment and selection stage. The continuing lack of organisational awareness or expertise regarding transgender people exacerbates the under-representation problem.

Disclosure was another key issue highlighted by our research (Ozturk and Tatli, 2016). The ability of a transgender worker to be upfront about their gender identity is often severely compromised due to gender-typical industry norms, as well as unsupportive organisational contexts.

Transgender employees often have to remain silent about their gender identity journeys as well as their pressing needs and requirements as transgender workers, because they fear reprisals from employers, peers and customers. As many organisations are not furnished with adequate know-how to create a trans-supportive environment, disclosure concerns continue to loom large in transgender work lives.

Another significant issue highlighted by our research is transition (Ozturk and Tatli, 2016). Transgender workers, who undertake gender identity transition while employed in the same organisation, are often subjected to invasive personal questions, treated as objects of curiosity and represent a problem that must be managed.

Organisations often worry too much about coordinating potential absences during transition, or how schedules can be managed so as to minimise any performance shortfalls.

Conversely, they worry too little about providing transgender employees with genuine support during this challenging period. As a result, transition often involves forced career breaks and the necessity of job change for transgender employees.

A proactive approach to inclusion is critical to surmount the serious barriers that stifle transgender equality at work. There is a deep and broad knowledge gap regarding transgender issues even within the HR and L&D ranks. As such, HR and L&D professionals, as well as equality and diversity officials need training on how to develop a sensitive and precise understanding of transgender issues.

Without this, they are not fully armed with the requisite knowledge to tackle deep-seated prejudices. In addition, the training for these groups must be followed by broader transgender training provided across the organisation, from the management ranks to entry-level employees. This encompassing roll out of training across different levels and functions is a pre-requisite for securing in-depth organisational awareness and appreciation of transgender issues.

Training is only truly effective, however, when it is deployed to engage people meaningfully by enlisting their active participation and imagination. Therefore, it is important to make training more than a single event, which requires one-time participation in a limited manner. Transgender inclusion training needs to be a series of developmental activities that align the organisational community with a strong appreciation of differences.

Training that cuts ice requires serious resource investment, which involves top management support. Top management support is also necessary to make transgender inclusion a strategic aspect of organisational culture and development. This may come in the form of concrete efforts such as targeted recruitment programmes, positive action at the selection stage, and mentoring and leadership programmes aimed at transgender workers. Such acts of ongoing support must be complemented by discursive steps such as amendment of strategic plans, policy documents and communications to ensure that transgender identities are a visible component of organisational activities.

Deep and meaningful progress to achieve transgender inclusion in work organisations requires significant action based on a long-term vision of change, which should be pursued by L&D, HR and top management.

Interested in this topic? ReadDiversity and inclusion: getting it right on the road to recovery.

See the original post:
Mind the gap: why training is vital to pursuing transgender inclusion - TrainingZone.co.uk