Cloud Hosting

In every election, after the polls close and the votes are counted, there comes a time for reflection. Pundits appear on cable news to offer theories, columnists pen op-eds with warnings and advice for the winners and losers, and parties conduct postmortems.

The 2020 U.S. presidential election in which Donald Trump lost to Joe Biden was no exception.

For Caltech undergrad Sreemanti Dey, the election offered a chance to do her own sort of reflection. Dey, an undergrad majoring in computer science, has a particular interest in using computers to better understand politics. Working with Michael Alvarez, professor of political and computational social science, Dey used machine learning and data collected during the 2020 election to find out what actually motivated people to vote for one presidential candidate over another.

In December, Dey presented her work on the topic at the fourth-annual International Conference on Applied Machine Learning and Data Analytics, which was held remotely and was recognized by the organizers as having the best paper at the conference.

We recently chatted with Dey and Alvarez, who is co-chair of the Caltech-MIT Voting Project, about their research, what machine learning can offer to political scientists, and what it is like for undergrads doing research at Caltech.

Sreemanti Dey: I think that how elections are run has become a really salient issue in the past couple of years. Politics is in the forefront of people's minds because things have gotten so, I guess, strange and chaotic recently. That, along with a lot of factors in 2020, made people care a lot more about voting. That makes me think it's really important to study how elections work and how people choose candidates in general.

Sreemanti: I've learned from Mike that a lot of social science studies are deductive in nature. So, you pick a hypothesis and then you pick the data that would best help you understand the hypothesis that you've chosen. We wanted to take a more open-ended approach and see what the data itself told us. And, of course, that's precisely what machine learning is good for.

In this particular case, it was a matter of working with a large amount of data that you can't filter through yourself without introducing a lot of bias. And that could be just you choosing to focus on the wrong issues. Machine learning and the model that we used are a good way to reduce the amount of information you're looking at without bias.

Basically it's a way of reducing high-dimensional data sets to the most important factors in the data set. So it goes through a couple steps. It first groups all the features of the data into these modules so that the features within a module are very correlated with each other, but there is not much correlation between modules. Then, since each module represents the same type of features, it reduces how many features are in each module. And then at the very end, it combines all the modules together and then takes one last pass to see if it can be reduced by anything else.

Mike: This technique was developed by Christina Ramirez (MS' 96, PhD '99), a PhD graduate of our program now at UCLA. Christina is someone who I've collaborated with quite a bit. Sreemanti and I were meeting pretty regularly with Christina and getting some advice from her along the way about this project and some others that we're thinking about.

Sreemanti: I think we got pretty much what we expected, except for what the most partisan-coded issues are. Those I found a little bit surprising. The most partisan questions turned out to be about filling the Supreme Court seats. I thought that it was interesting.

Sreemanti: It's really incredible. I find it astonishing that a person like Professor Alvarez has the time to focus so much on the undergraduates in lab. I did research in high school, and it was an extremely competitive environment trying to get attention from professors or even your mentor.

It's a really nice feature of Caltech that professors are very involved with what their undergraduates are doing. I would say it's a really incredible opportunity.

Mike: I and most of my colleagues work really hard to involve the Caltech undergraduates in a lot of the research that we do. A lot of that happens in the SURF [Summer Undergraduate Research Fellowship] program in the summers. But it also happens throughout the course of the academic year.

What's unusual a little bit here is that undergraduate students typically take on smaller projects. They typically work on things for a quarter or a summer. And while they do a good job on them, they don't usually reach the point where they produce something that's potentially publication quality.

Sreemanti started this at the beginning of her freshman year and we worked on it through her entire freshman year. That gave her the opportunity to really learn the tools, read the political science literature, read the machine learning literature, and take this to a point where at the end of the year, she had produced something that was of publication quality.

Sreemanti: It was a little bit strange, first of all, because of the time zone issue. This conference was in a completely different time zone, so I ended up waking up at 4 a.m. for it. And then I had an audio glitch halfway through that I had to fix, so I had some very typical Zoom-era problems and all that.

Mike: This is a pandemic-era story with how we were all working to cope and trying to maintain the educational experience that we want our undergraduates to have. We were all trying to make sure that they had the experience that they deserved as a Caltech undergraduate and trying to make sure they made it through the freshman year.

We have the most amazing students imaginable, and to be able to help them understand what the research experience is like is just an amazing opportunity. Working with students like Sreemanti is the sort of thing that makes being a Caltech faculty member very special. And it's a large part of the reason why people like myself like to be professors at Caltech.

Sreemanti: I think I would want to continue studying how people make their choices about candidates but maybe in a slightly different way with different data sets. Right now, from my other projects, I think I'm learning how to not rely on surveys and rely on more organic data, for example, from social media. I would be interested in trying to find a way to study their candidatepeople's candidate choice from their more organic interactions with other people.

Sreemanti's paper, titled, "Fuzzy Forests for Feature Selection in High-Dimensional Survey Data: An Application to the 2020 U.S. Presidential Election," was presented in December at the fourth-annual International Conference on Applied Machine Learning and Data Analytics," where it won the best paper award.

Here is the original post:

Politics, Machine Learning, and Zoom Conferences in a Pandemic: A Conversation with an Undergraduate Researcher - Caltech

Editors note: Startup Spotlight is a regular feature at WRAL TechWire designed to bring attentionn about potential emerging stars in the North Carolina innovation economy.

+++

DURHAM Working as part of the Peace Corps in Cameroon in the early 2000s, Meghan Corroon witnessed first-hand the need for reliable data. Without it, she recalled, critical funding got wasted trying to bring clean water to villages.

Years later, she saw similar challenges while running large-scale projects for The Bill and Melinda Gates Foundation, as well as several national governments in Africa and Asia.

I was increasingly frustrated with the pace of innovation, she told WRAL TechWire.

So she founded Clerdata, formerly Lumen Insights. Its a Durham-based data science startup that aims to bring change this time to the corporate world by ending ineffective marketing strategies.

Rather than a cluttered dashboard that still doesnt answer the most important questions,Corroon said Clerdatas product offers sophisticated modeling that is fast, lean, and automated.

In a survey of 1,000 marketers worldwide by Rakuten Marketing, respondents estimated they waste an average of26%of their budgets on ineffective channels and strategies. And about half of respondents said they misspend at least 20% of their budgets.

Corroon, who interviewed businesses across industries for over a year before launching, said she heard similar testimonies from very small to multi-billion-dollar businesses. In todays world of advanced data science and business insights, she said she couldnt believe this was still the case: We built Clerdata to tackle this problem.

Clerdata founder Meghan Corroon

Clerdata offers a marketing tool that tells customers which digital marketing channels are driving their sales. It operates using a proprietary algorithm that includes statistical modeling to tackle the question of marketing action effectiveness on sales in brick-and-mortar stores, as well as e-commerce. This allows customers to move more quickly from insights to business decisions, Corroon said: We believe thats the future of SaaS data products.

It also avoids using consumer web tracking data.

This is a huge deal as Facebook lost 10 billion in ad revenue just last quarter due to this increasingly faulty data pipeline. This problem affects almost every consumer business in our modern economy.

Clerdata is already reaping the rewards. Since launching in 2018, its team has grown to five employees and projected revenue growth is expected to be over 700% this year.

Clerdata team

It has a growing list of clients, including Raleigh-based Videri Chocolate Factory; nationally distributed Marys Gone Crackers; Peacock Alley, a luxury linens company; and Healthy Living Market, an independent grocery store chain, among others.

To date, the startup remains bootstrapped with no help from outside investors. But Corroon said she wont rule that out in the future.

Were hyper focused on running our own race and serving our customers a product with integrity, she said. Were excited for the future of this adventure.

Originally posted here:

Startup Spotlight: This woman-led Triangle startup is democratizing big data to drive sales for businesses - WRAL TechWire

The majority of teams now work remotely or on a rotating basis. In this situation, it is critical for a company to manage their workforce optimally so that only the necessary resources are available when they are required.

Fremont, CA: As the world struggles to cope with the pandemic's aftermath, businesses are looking for new technologies to help them streamline operations and ensure continuity in the coming days. Business leaders are embracing data analytics, AI, and data science to improve processes, become more scalable, and plan for the future.

Let us look at how data analytics is playing a critical role in ensuring business continuity.

Workforce Management

The majority of teams now work remotely or on a rotating basis. In this situation, it is critical for a company to manage their workforce optimally so that only the necessary resources are available when they are required. Businesses can accurately predict how many workers are needed on a daily, weekly, or monthly basis by using data analytics. They can forecast the volume of work and assign teams accordingly. They can also remotely monitor the performance of the teams to ensure maximum efficiency. In the BPO industry, data analytics for workforce management is widely used.

Data Security

Data security has become a major concern for businesses as teams work remotely and office networks are exposed to external environments. Businesses are using data analytics to monitor system logs and identify anomalies as they transition to new, digital ways of working. IT teams are using analytics to monitor all users and applications in real time and to detect and mitigate system threats in real time. They can set up automated workflows for deviations and trigger the next set of actionswithout requiring any dependencies or waiting time.

Forecasting and Risk Mitigation

Business data is a goldmine of information that can assist business leaders in a variety of ways. Business leaders can detect problems before they occur by using AI and ML-powered data analytics. Enterprises are also devoting time and effort to analyzing historical data from the COVID timeline to determine how deeply and where the business was impacted. They can predict what factors will cause interruptions and take proactive steps to control them by using analytics and data science.

Read the rest here:

The Role Of Data Analytics In Ensuring Business Continuity - CIO Applications

The new Kaggle book The Kaggle Book: Data analysis and machine learning for competitive data science, written by Konrad Banachewicz and Luca Massaron, has been released. This book is suitable for anyone new to Kaggle and veteran users too. Data analysts/scientists who want to perform better in Kaggle competitions and secure jobs with tech giants will find this book useful as well.

First of its kind, The Kaggle Book assembles the techniques and skills a participant would require for success in competitions, data science projects, and beyond. In the book, two Kaggle Grandmasters who have accumulated knowledge along the way will walk you through modelling strategies and these details are not easily found elsewhere.

The book also has Kaggle-specific tips, where the participants will learn more techniques for approaching tasks based on image, tabular, textual data, and reinforcement learning. This book will help design validation schemes and work comfortably with different evaluation metrics. This book will help you climb the ranks of Kaggle, build some data science skills, or improve the accuracy of your existing models.

Read the original here:

New book on Kaggle released with inputs from over 30 experts - Analytics India Magazine

Not all data are created equal. But how much information is any piece of data likely to contain? This question is central to medical testing, designing scientific experiments, and even to everyday human learning and thinking. MIT researchers have developed a new way to solve this problem, opening up new applications in medicine, scientific discovery, cognitive science, and artificial intelligence.

In theory, the 1948 paper, A Mathematical Theory of Communication, by the late MIT Professor Emeritus Claude Shannon answered this question definitively. One of Shannons breakthrough results is the idea of entropy, which lets us quantify the amount of information inherent in any random object, including random variables that model observed data. Shannons results created the foundations of information theory and modern telecommunications. The concept of entropy has also proven central to computer science and machine learning.

The challenge of estimating entropy

Unfortunately, the use of Shannons formula can quickly become computationally intractable. It requires precisely calculating the probability of the data, which in turn requires calculating every possible way the data could have arisen under a probabilistic model. If the data-generating process is very simple for example, a single toss of a coin or roll of a loaded die then calculating entropies is straightforward. But consider the problem of medical testing, where a positive test result is the result of hundreds of interacting variables, all unknown. With just 10 unknowns, there are already 1,000 possible explanations for the data. With a few hundred, there are more possible explanations than atoms in the known universe, which makes calculating the entropy exactly an unmanageable problem.

MIT researchers have developed a new method to estimate good approximations to many information quantities such as Shannon entropy by using probabilistic inference. The work appears in a paper presented at AISTATS 2022 by authors Feras Saad 16, MEng 16, a PhD candidate in electrical engineering and computer science; Marco-Cusumano Towner PhD 21; and Vikash Mansinghka 05, MEng 09, PhD 09, a principal research scientist in the Department of Brain and Cognitive Sciences. The key insight is, rather than enumerate all explanations, to instead use probabilistic inference algorithms to first infer which explanations are probable and then use these probable explanations to construct high-quality entropy estimates. The paper shows that this inference-based approach can be much faster and more accurate than previous approaches.

Estimating entropy and information in a probabilistic model is fundamentally hard because it often requires solving a high-dimensional integration problem. Many previous works have developed estimators of these quantities for certain special cases, but the new estimators of entropy via inference (EEVI) offer the first approach that can deliver sharp upper and lower bounds on a broad set of information-theoretic quantities. An upper and lower bound means that although we don't know the true entropy, we can get a number that is smaller than it and a number that is higher than it.

The upper and lower bounds on entropy delivered by our method are particularly useful for three reasons, says Saad. First, the difference between the upper and lower bounds gives a quantitative sense of how confident we should be about the estimates. Second, by using more computational effort we can drive the difference between the two bounds to zero, which squeezes the true value with a high degree of accuracy. Third, we can compose these bounds to form estimates of many other quantities that tell us how informative different variables in a model are of one another.

Solving fundamental problems with data-driven expert systems

Saad says he is most excited about the possibility that this method gives for querying probabilistic models in areas like machine-assisted medical diagnoses. He says one goal of the EEVI method is to be able to solve new queries using rich generative models for things like liver disease and diabetes that have already been developed by experts in the medical domain. For example, suppose we have a patient with a set of observed attributes (height, weight, age, etc.) and observed symptoms (nausea, blood pressure, etc.). Given these attributes and symptoms, EEVI can be used to help determine which medical tests for symptoms the physician should conduct to maximize information about the absence or presence of a given liver disease (like cirrhosis or primary biliary cholangitis).

For insulin diagnosis, the authors showed how to use the method for computing optimal times to take blood glucose measurements that maximize information about a patients insulin sensitivity, given an expert-built probabilistic model of insulin metabolism and the patients personalized meal and medication schedule. As routine medical tracking like glucose monitoring moves away from doctors offices and toward wearable devices, there are even more opportunities to improve data acquisition, if the value of the data can be estimated accurately in advance.

Vikash Mansinghka, senior author on the paper, adds, We've shown that probabilistic inference algorithms can be used to estimate rigorous bounds on information measures that AI engineers often think of as intractable to calculate. This opens up many new applications. It also shows that inference may be more computationally fundamental than we thought. It also helps to explain how human minds might be able to estimate the value of information so pervasively, as a central building block of everyday cognition, and help us engineer AI expert systems that have these capabilities.

The paper, Estimators of Entropy and Information via Inference in Probabilistic Models, was presented at AISTATS 2022.

Read the original here:

Estimating the informativeness of data | MIT News | Massachusetts Institute of Technology - MIT News