Cloud Hosting

Self-made image

Over the past year, Large Language Models (LLMs) have demonstrated astonishing capabilities thanks to their natural language understanding. These advanced models have not only redefined the standards in Natural Language Processing but also populated applications and services.

There has been a rapidly growing interest in using LLMs for coding, with some companies striving to turn natural language processing into code understanding and generation. This task has already highlighted several challenges yet to be addressed in using LLMs for coding. Despite these obstacles, this trend has led to the development of AI code generator products.

Have you ever used ChatGPT for coding?

While it can be helpful in some instances, it often struggles to generate efficient and high-quality code. In this article, we will explore three reasons why LLMs are not inherently proficient at coding out of the box: the tokenizer, the complexity of context windows when applied to code and the nature of the training itself .

Identify the key areas that need improvement is crutial to transform LLMs into more effective coding assistants!

The LLM tokenizer is the responsible of converting the user input text, in natural language, to a numerical format that the LLMs can understand.

The tokenizer processes raw text by breaking it down into tokens. Tokens can be whole words, parts of words (subwords), or individual characters, depending on the tokenizers design and the requirements of the task.

Since LLMs operate on numerical data, each token is given an ID which depends on the LLM vocabulary. Then, each ID is further associated with a vector in the LLMs latent high-dimensional space. To do this last mapping, LLMs use learned embeddings, which are fine-tuned during training and capture complex relationships and nuances in the data.

If you are interested in playing around with different LLM tokenizers and see how they

Follow this link:

Why LLMs are not Good for Coding. Challenges of Using LLMs for Coding | by Andrea Valenzuela | Feb, 2024 - Towards Data Science

In some situations, youll need to do some advanced indexing / selection with Pytorch, e.g. answer the question: how can I select elements from Tensor A following the indices specified in Tensor B?

In this post well present the three most common methods for such tasks, namely torch.index_select, torch.gather and torch.take. Well explain all of them in detail and contrast them with one another.

Admittedly, one motivation for this post was me forgetting how and when to use which function, ending up googling, browsing Stack Overflow and the, in my opinion, relatively brief and not too helpful official documentation. Thus, as mentioned, we here do a deep dive into these functions: we motivate when to use which, give examples in 2- and 3D, and show the resulting selection graphically.

I hope this post will bring clarity about said functions and remove the need for further exploration thanks for reading!

And now, without further ado, lets dive into the functions one by one. For all, we first start with a 2D example and visualize the resulting selection, and then move to somewhat more complex example in 3D. Further, we re-implement the executed operation in simple Python s.t. you can look at pseudocode as another source of information what these functions do. In the end, we summarize the functions and their differences in a table.

torch.index_select selects elements along one dimension, while keeping the other ones unchanged. That is: keep all elements from all other dimensions, but pick elements in the target dimensions following the index tensor. Lets demonstrate this with a 2D example, in which we select along dimension 1:

The resulting tensor has shape [len_dim_0, num_picks]: for every element along dimension 0, we have picked the same element from dimension 1. Lets visualize this:

Read more here:

Advanced Selection from Tensors in Pytorch | by Oliver S | Feb, 2024 - Towards Data Science

Predicting the future of ecosystems requires a plethora of accurate data available in real-time.

To enable real-time data collection, three Virginia Tech researchers have received a prestigious five-year National Science Foundation grant to help better predict the future of ecosystems.

The $450,000 Long-Term Research in Environmental Biology (LTREB) grant will support the enhancement and continuation of field monitoring and data sharing at two freshwater drinking water supply reservoirs in Roanoke.

This grant will allow the researchers to create a cutting-edge, ecological monitoring program with real-time data access and publishing, which normally takes weeks to years after data collection.

We are developing one of the first open-source automated forecasting systems in the world by using the reservoirs as a test bed for exploring new data collection, data access, and forecasting methods, said Cayelan Carey, professor and the Roger Moore and Mejdeh Khatam-Moore Faculty Fellow in the Department of Biological Sciences. We will use our new designation as an official long-term research environmental biology monitoring site as a platform for scaling and disseminating our data so that other researchers can similarly start to forecast water quality in lakes and reservoirs around the globe.

The LTREB program is one of the National Science Foundations premier environmental science programs to support long-term monitoring at select exemplar terrestrial, coastal, and freshwater ecosystems across the U.S.

Carey leads the Virginia Reservoirs LTREB team with Professor Madeline Schreiber in the Department of Geosciences and Associate Professor Quinn Thomas, who has a joint appointment in the Departments of Forest Resources and Environmental Conservation and Biological Sciences.

With a group of co-mentored students, technicians, and postdoctoral researchers, Carey, Schreiber, and Thomas have been monitoring biological, chemical, and physical metrics of water quality in the two reservoirs for the past decade in partnership with the Western Virginia Water Authority, which owns and manages the reservoirs for drinking water.

See the rest here:

Researchers receive National Science Foundation grant for long-term data research - Virginia Tech

Exploring the architecture that brought transformers into image generation Image generated with DALLE.

After shaking up NLP and moving into computer vision with the Vision Transformer (ViT) and its successors, transformers are now entering the field of image generation. They are gradually becoming an alternative to the U-Net, the convolutional architecture upon which all the early diffusion models were built. This article looks into the Diffusion Transformer (DiT), introduced by William Peebles and Saining Xie in their paper Scalable Diffusion Models with Transformers.

DiT has influenced the development of other transformer-based diffusion models like PIXART-, Sora (OpenAIs astonishing text-to-video model), and, as I write this article, Stable Diffusion 3. Lets start exploring this emerging class of architectures that are contributing to the evolution of diffusion models.

Given that this is an advanced topic, Ill have to assume a certain familiarity with recurring concepts in AI and, in particular, in image generation. If youre already familiar with this field, this section will help refresh these concepts, providing you with further references for a deeper understanding.

If you want an extensive overview of this world before reading this article, I recommend reading my previous article below, where I cover many diffusion models and related techniques, some of which well revisit here.

At an intuitive level, diffusion models function by first taking images, introducing noise (usually Gaussian), and then training a neural network to reverse this noise-adding

Read the original post:

Diffusion Transformer Explained. Exploring the architecture that brought | by Mario Namtao Shianti Larcher | Feb, 2024 - Towards Data Science

Best practice and advanced techniques for beginners 12 min read

In this story, I would like to talk about data warehouse design and how we organise the process. Data modelling is an essential part of data engineering. It defines the database structure, schemas we use and data materialisation strategies for analytics. Designed in the right way it helps to ensure our data warehouse runs efficiently meeting all business requirements and cost optimisation targets. We will touch on some well-known best practices in data warehouse design using the dbt tool as an example. We will take a better look into some examples of how to organise the build process, test our datasets and use advanced techniques with macros for better workflow integration and deployment.

Lets say we have a data warehouse and lots of SQL to deal with the data we have in it.

In my case it is Snowflake. Great tool and one of the most popular solutions in the market right now, definitely among the top three tools for this purpose.

So how do we structure our data warehouse project? Consider this starter project folder structure below. This is what we have after we run dbt init command.

At the moment we can see only one model called example with table_a and table_b objects. It can be any data warehouse objects that relate to each other in a certain way, i.e. view, table, dynamic table, etc.

When we start building our data warehouse the number of these objects will grow inevitably and it is the best practice to keep it organised.

The simple way of doing this would be to organise the model folder structure being split into base (basic row transformations) and analytics models. In the analytics subfolder, we would typically have data deeply enriched and

Excerpt from:

Building a Data Warehouse. Best practice and advanced techniques | by Mike Shakhomirov | Feb, 2024 - Towards Data Science