Modern data warehousing and Machine Learning12 min read

No doubt, user retention is a crucial performance metric for many companies and online apps. We will discuss how we can use built-in data warehouse machine learning capabilities to run propensity models on user behaviour data to determine the likelihood of user churn. In this story, I would like to focus on dataset preparation and model training using standard SQL. Modern data warehouses allow this. Indeed, retention is an important business metric that helps understand user behaviours mechanics. It provides a high-level overview of how successful our Application is in terms of retaining users by answering one simple question: Is our App good enough at retaining users? It is a well-known fact that its cheaper to retain an existing user than to acquire a new one.

In one of my previous articles, I wrote about modern data warehousing [1].

Modern DWH has a lot of useful features and components which differentiate them from other data platform types [2].

ML model support seems to be the foundational DWH component when dealing with big data.

In this story, I will use Binary logistic regression, one of the fastest models to train. I will demonstrate how we can use it to predict user propensity to churn. Indeed, We dont need to know every machine-learning model.

We cant compete with cloud service providers such as Amazon ang Google in machine learning and data science but we need to know how to use it.

I previously wrote about it in my article here [3]:

In this tutorial, we will learn how to transform raw event data to create a training dataset for our ML

See the rest here:

User Churn Prediction. Modern data warehousing and Machine | by Mike Shakhomirov | Dec, 2023 - Towards Data Science

2023 may have been the year of the LLMwe highlighted our most popular articles on ChatGPT and related topics last weekbut data science and machine learning are far too vast for us to reduce them to a single phenomenon (as inescapable as it might be).

Every day, TDS authors publish excellent work on a staggering range of topics, from the latest tools of the trade to career insights and project walkthroughs. For our final Variable edition of the year, we decided to highlight some of the most memorable and widely read posts weve shared around three themes: programming for data scientists, career growth, and and creative projects and opinion pieces. They do a fantastic job showing just how vibrant, diverse, and dynamic this fieldand our communityis.

We hope you enjoy our selection, and thank you once again for all your support over the past year.

Here is the original post:

Data Science Career Paths, Skills, and Special Projects: Our Best Reads of 2023 - Towards Data Science

As you can see above with greedy strategy, we append the token with the highest probability to the input sequence and predict the next token.

Using this strategy lets generate a longer text with 128 next tokens using greedy-search decoding.

As you we can see from the text above, although it is the simplest logic, the drawback of this approach is the generated repetitive sequences. As it fails to capture the probabilities of sequences, meaning, the overall probability of a several words coming one after another is overlooked. Greedy search predicts and considers only the probability one step at a time.

Repetitive text is a problem. We would desire our generated output to be concise, how can we achieve it?

Instead of choosing the token that has highest probability at each step, we consider future x-steps and calculate the joint probability(simply multiplication of consecutive probabilities) and choose the next token sequence that is most probable. While x refers to number of beams, it is the depth of the future sequence we look into the future steps. This strategy is known as the beam search.

Lets go back to our example from GPT-2 and explore beam vs greedy search scenarios.

Given the prompt, looking at the two tokens with highest probability and their continuation(4 beams) in a tree diagram

Lets calculate the join probabilities of the green sequences above.

Germany is known for its -> high-quality beer

with the joint probability 3.30%*24.24%*31.26%*6.54% = 0.0016353

whereas the lower path with the sequence;

Germany is known for its -> strong tradition of life

2.28%*2.54%*87.02%*38.26% = 0.0019281.

The bottom sequence overall resulted with the higher joint probability, although the first next token prediction step in the top sequence has higher probability.

While greedy search priorities the absolute maximum probability at each prediction step, it neglects the token probabilities in sequences. Beam search decoding enables us to go in depth of sequences and help us decode text in more extensive fashion. So is beam-search the ultimate solution?

Lets explore further and decode the next 128 tokens with the depth of 5 beams.

Here are 10 of the most beautiful places in Germany.

1. Lake Constance

Lake Constance is one of the largest lakes in Germany.

It is located in the state of North Rhine-Westphalia and is the second largest lake in Germany after Lake Constance in Bavaria.

Lake Constance is located in the state of North Rhine-Westphalia and is the second largest lake in Germany after Lake Constance in Bavaria.

"""

Although comparatively lesser than the greedy-search, beam-search suffers from repetitive output too. However, with beam search decoding, we can solve this problem by penalising the repeated pairs of word sequences. In other words, the probability of token sequences is assigned zero, if the sequence has already been decoded before. This penalisation of a repeated tokens sequence is also know as n-gram penalty.

While n signifies the length of the sequence, gram is a term that refers to unit in computational linguistic often corresponds to the term token in our case.

The reasoning behind is to discourage the generation of sequences that contain consecutive repeating n-grams. The decoding algorithm will penalise generated sequences that contain repeating pairs of words in the output.

Knowing this, lets apply n-gram penalty of n = 2.

The country's capital, Berlin, is the largest city in Europe, with a population of more than 8.5 million people.

The city is located in the former East Germany, which was divided into East and West Germany after World War II.

Today, Germany is a member of both the European Union and NATO, as well as the World Trade Organization and the Organization for Economic Cooperation and Development (OECD).<|endoftext|>

"""

This is the best completion of the input prompt we extracted from the model so far in terms of coherence and compactness. Through n-gram penalisation the output decoded with beam-search became more human-like.

When should we use beam-search and when greedy-search? Where the factualness is paramount, like solving a math problem, key information extraction, summarisation or translation, greedy-search should be preferred. However, when we want to achieve creative output and factuality is not our priority (like it can be in the case of story generation) beam-search is often the better suited approach.

Why exactly does your prompt matter? Because every word you choose to use, the sentence structure, the layout of your instructions will activate different series of parameters in the deep layers of large language model and the probabilities will be formed differently for each different prompt. In the essence of the matter, the text generation is a probability expression conditional on your prompt.

There are also alternative methods to prevent repetitions and influence the factuality/creativity of the generated text, such as truncating the distribution of vocabulary or sampling methods. If you are interested in a higher-level in-depth exploration of the subject, Id highly recommend the article from Patrick von Platen in HuggingFace blog.

Next and the last article of this series will explore fine-tuning and reinforcement learning through human feedback which played an important role on why pre-trained models succeeded to surpass SOTA models in several benchmarks. I hope in this blog post, I was able help you understand the reasoning of prompt engineering better. Many thanks for the read. Until next time.

Read this article:

Inside GPT II. The core mechanics of prompt engineering | by Fatih Demirci | Dec, 2023 | Medium - Towards Data Science

Image by Gerd Altmann from Pixabay

Members of the European Parliament and the Council reached provisional agreement on the Artificial Intelligence Act on December 9th, 2023 after years of debate and discussion. The AI Act is broad in scope and is intended to protect public welfare, digital rights, democracy, and the rule of law from the dangers of AI. The Act in this sense underscores the need to ensure and protect data sovereignty of both individuals and organizations.

On the data sovereignty regulation front, Europes approach is comparable to Californias on the vehicle emissions regulation front. Carmakers design to the California emissions requirement, and by doing so make sure theyre compliant elsewhere. Much like the GDPR [the EUs General Data Protection Regulation, which went into effect in 2018], the AI Act could become a global standard. Companies elsewhere that want to do business in the worlds second-largest economy will have to comply with the law, pointed out Melissa Heikkil in a December 11, 2023 piece in the MIT Technology Review.

In November 2023, the Organisation for Economic Co-operation and Developments (OECDs) Council updated its definition of artificial intelligence. The European Parliament then adopted the OECDs definition, which is as follows (emphasis mine):

An AI system is a machine-based system that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments. Different AI systems vary in their levels of autonomy and adaptiveness after deployment.

Note the text above in boldface. AI systems infer how to generate outputs from inputs. In other words, AI systems are entirely dependent on the quality of their data input.

We can talk all we want to about trustworthy models, but when it comes to statistical models being trustworthy, inputs rule. High data quality is a prerequisite. When the input is garbage, the output will be garbage also.

Most of the time, data scientists grapple with the input before training their models, so the output they end up with often seems reasonable. But the output despite their efforts can be problematic in ways that arent straightforward. How to solve that problem? Make sure the data quality is high to begin with, before it gets to the data scientist. And then make sure the data scientists preserve that quality by preserving context throughout the rest of the process.

The best way to think about ensuring data quality up front is domain by domain. Each business domain needs to produce relevant, contextualized data specific to that domain. Then at a higher level of abstraction, the organization needs to knit that context together to be able to scale data management.

What results is an input model of the business, described as consumable data, that accompanies the rest of the data when fed to machines.

With specific context articulated in the input data, the data becomes explicit enough for machines to associate the data supplied as input with a given context. Explicit relationships stated as facts in domain-specific data are what help to create sufficient context. Theyre what distinguishes tennis matches from kitchen matches.

Organizations need to spell things out for machines by feeding them contextualized facts about their businesses. Volumes and volumes of text, systematically accumulated, can deliver bits and pieces of context. But still, a good portion of that context will be missing from the input data. How to solve that problem? Those responsible for each domains data can make each context explicit by making the relationships between entities explicit.

Once those relationships are explicit, each organization can connect the contexts for each domain together with a simplified model of the business as a whole, whats called an upper ontology.

Most organizations have been siloing data and trapping relationship information separately in applications because thats what existing data and software architecture mandates.

Knowledge graphs provide a place to bring the siloed data and the necessary relationship information for context together. These graphs, which can harness the power of automation in various ways, also provide a means of organization-wide access to unified, relationship-rich whole. Instead of each app holding the relationship information for itself, the graph becomes the resource for that information too. That way, instance data and relationship data can evolve together.

Graphs facilitate the creation, storage and reuse of fully articulated, any-to-any relationships. This graph paradigm itself encourages data connections and reuse by contrast with the data siloing and code sprawl of older data management techniques.

Intelligent data is data that describes itself so that machines dont have to guess what it means. That self-describing data in true knowledge graphs provides machines sufficient context so that machines can provide accurately contextualized output. This addition of context is what makes the difference when it comes to AI accuracy. The larger, logically interconnected context, moreover, can become an organic, reusable resource for the entire business.

View post:

Data management implications of the AI Act - DataScienceCentral.com - Data Science Central

Dec. 22, 2023 The San Diego Supercomputer Center at UC San Diego, along with theGO FAIR Foundation, the National Center for Atmospheric Research, the Ronin Institute and other partners, will conduct data landscaping work funded by theFrederick National Laboratory for Cancer Research, operated by Leidos Biomedical Research, Inc., on behalf of the National Institute of Allergy and Infectious Diseases (NIAID). SDSCs Research Data Services Director Christine Kirkpatrick leads theGO FAIR U.S.Office at SDSC and serves as PI for the new project.

The NIAID Data Landscaping and FAIRification project seeks to benefit biomedical researchers and the broader community to generate and analyze infectious, allergic and immunological data. Using theFAIR Principles as a guide, the project teamoffering a broad background to ensure that metadata, a set of data that describes and gives information about other data, for biomedical research is findable, accessible, interoperable and reusable (FAIR)will provide guidance on approaches to enhance the quality of metadata within NIAID and NIH supported repositories and resources that harbor data and metadata.

Structured trainings and guidance will be offered to support stakeholders, including components from the model pioneered by GO FAIR leveraging established M4M workshops and adopting FAIR Implementation Profiles (FIPs). This work will be underpinned by interviews with stakeholders and an assessment to explore the relationship between FAIR resources and scientific impact. The initial period of the federally funded contract, which runs from Sept. 20, 2023 to Sept. 30, 2024, is valued at $1.3 million.

Highlights of the teams expertise include co-authoring the FAIR Guiding Principles, facilitating metadata for machines (M4M) workshops, developing the FAIR Implementation Profile approach, and contributing to improvements on data policy and metadata practices and standards.

Our team is elated to be working with our NIAID project sponsors at the Office of Data Science and Emerging Technologies (ODSET) through Leidos Biomedical Research, remarked Kirkpatrick, PI of the landscaping project. NIAID is renowned for its significant data resources and impactful scientific research. Having the chance to apply our collective expertise in research data management in support of the NIAID mission areas of infectious disease, allergy and immunology will be both impactful to the FAIR ecosystem, and meaningful work for our team. Further, I believe this work will become more common in the future as organizations begin to see data as a strategic asset, rather than focus on the cost of storing it.

The project follows alongside another key project in the Leidos Biomedical Research portfolio, theNIAID Data Ecosystem Discovery Portal, led by The Scripps Research Institute. The project team will work hand in hand with the Scripps team to ensure repository improvements maximize the Discovery Portals ability to search across the wide array of data assets produced by NIAID-funded research.

The project team includes co-authors of the 2016 FAIR Principles paper (Barend Mons and Erik Schultes), leaders in research data consortia, scholars in informatics, biomedical research and pioneers in FAIR training, interoperability practices and methodology for assessing scientific impact. Team members are Chris Erdmann, Doug Fils, John Graybeal, Nancy Hoebelheinrich, Kathryn Knight, Natalie Meyers, Bert Meerman, Barbara Magagna, Keith Maull and Matthew Mayernik. These experts are complemented by world-class systems integrators and project managers from SDSC: Alyssa Arce, Julie Christopher and Kevin Coakley.

Source: Christine Kirkpatrick and Julie Christopher, SDSC Research Data Services

Read the original:

FAIR Skies Ahead for Biomedical Data Project Looking to Benefit Research Community - Datanami