Cloud Hosting

Summary: Researchers developed a new machine learning technique to improve red-teaming, a process used to test AI models for safety by identifying prompts that trigger toxic responses. By employing a curiosity-driven exploration method, their approach encourages a red-team model to generate diverse and novel prompts that reveal potential weaknesses in AI systems.

This method has proven more effective than traditional techniques, producing a broader range of toxic responses and enhancing the robustness of AI safety measures. The research, set to be presented at the International Conference on Learning Representations, marks a significant step toward ensuring that AI behaviors align with desired outcomes in real-world applications.

Key Facts:

Source: MIT

A user could ask ChatGPT to write a computer program or summarize an article, and the AI chatbot would likely be able to generate useful code or write a cogent synopsis. However, someone could also ask for instructions to build a bomb, and the chatbot might be able to provide those, too.

To prevent this and other safety issues, companies that build large language models typically safeguard them using a process called red-teaming. Teams of human testers write prompts aimed at triggering unsafe or toxic text from the model being tested. These prompts are used to teach the chatbot to avoid such responses.

But this only works effectively if engineers know which toxic prompts to use. If human testers miss some prompts, which is likely given the number of possibilities, a chatbot regarded as safe might still be capable of generating unsafe answers.

Researchers from Improbable AI Lab at MIT and the MIT-IBM Watson AI Lab used machine learning to improve red-teaming. They developed a technique to train a red-team large language model to automatically generate diverse prompts that trigger a wider range of undesirable responses from the chatbot being tested.

They do this by teaching the red-team model to be curious when it writes prompts, and to focus on novel prompts that evoke toxic responses from the target model.

The technique outperformed human testers and other machine-learning approaches by generating more distinct prompts that elicited increasingly toxic responses. Not only does their method significantly improve the coverage of inputs being tested compared to other automated methods, but it can also draw out toxic responses from a chatbot that had safeguards built into it by human experts.

Right now, every large language model has to undergo a very lengthy period of red-teaming to ensure its safety. That is not going to be sustainable if we want to update these models in rapidly changing environments.

Our method provides a faster and more effective way to do this quality assurance, says Zhang-Wei Hong, an electrical engineering and computer science (EECS) graduate student in the Improbable AI lab and lead author of apaper on this red-teaming approach.

Hongs co-authors include EECS graduate students Idan Shenfield, Tsun-Hsuan Wang, and Yung-Sung Chuang; Aldo Pareja and Akash Srivastava, research scientists at the MIT-IBM Watson AI Lab; James Glass, senior research scientist and head of the Spoken Language Systems Group in the Computer Science and Artificial Intelligence Laboratory (CSAIL); and senior author Pulkit Agrawal, director of Improbable AI Lab and an assistant professor in CSAIL. The research will be presented at the International Conference on Learning Representations.

Automated red-teaming

Large language models, like those that power AI chatbots, are often trained by showing them enormous amounts of text from billions of public websites. So, not only can they learn to generate toxic words or describe illegal activities, the models could also leak personal information they may have picked up.

The tedious and costly nature of human red-teaming, which is often ineffective at generating a wide enough variety of prompts to fully safeguard a model, has encouraged researchers to automate the process using machine learning.

Such techniques often train a red-team model using reinforcement learning. This trial-and-error process rewards the red-team model for generating prompts that trigger toxic responses from the chatbot being tested.

But due to the way reinforcement learning works, the red-team model will often keep generating a few similar prompts that are highly toxic to maximize its reward.

For their reinforcement learning approach, the MIT researchers utilized a technique called curiosity-driven exploration. The red-team model is incentivized to be curious about the consequences of each prompt it generates, so it will try prompts with different words, sentence patterns, or meanings.

If the red-team model has already seen a specific prompt, then reproducing it will not generate any curiosity in the red-team model, so it will be pushed to create new prompts, Hong says.

During its training process, the red-team model generates a prompt and interacts with the chatbot. The chatbot responds, and a safety classifier rates the toxicity of its response, rewarding the red-team model based on that rating.

Rewarding curiosity

The red-team models objective is to maximize its reward by eliciting an even more toxic response with a novel prompt. The researchers enable curiosity in the red-team model by modifying the reward signal in the reinforcement learning set up.

First, in addition to maximizing toxicity, they include an entropy bonus that encourages the red-team model to be more random as it explores different prompts. Second, to make the agent curious they include two novelty rewards.

One rewards the model based on the similarity of words in its prompts, and the other rewards the model based on semantic similarity. (Less similarity yields a higher reward.)

To prevent the red-team model from generating random, nonsensical text, which can trick the classifier into awarding a high toxicity score, the researchers also added a naturalistic language bonus to the training objective.

With these additions in place, the researchers compared the toxicity and diversity of responses their red-team model generated with other automated techniques. Their model outperformed the baselines on both metrics.

They also used their red-team model to test a chatbot that had been fine-tuned with human feedback so it would not give toxic replies. Their curiosity-driven approach was able to quickly produce 196 prompts that elicited toxic responses from this safe chatbot.

We are seeing a surge of models, which is only expected to rise. Imagine thousands of models or even more and companies/labs pushing model updates frequently. These models are going to be an integral part of our lives and its important that they are verified before released for public consumption.

Manual verification of models is simply not scalable, and our work is an attempt to reduce the human effort to ensure a safer and trustworthy AI future, says Agrawal.

In the future, the researchers want to enable the red-team model to generate prompts about a wider variety of topics. They also want to explore the use of a large language model as the toxicity classifier. In this way, a user could train the toxicity classifier using a company policy document, for instance, so a red-team model could test a chatbot for company policy violations.

If you are releasing a new AI model and are concerned about whether it will behave as expected, consider using curiosity-driven red-teaming, says Agrawal.

Funding: This research is funded, in part, by Hyundai Motor Company, Quanta Computer Inc., the MIT-IBM Watson AI Lab, an Amazon Web Services MLRA research grant, the U.S. Army Research Office, the U.S. Defense Advanced Research Projects Agency Machine Common Sense Program, the U.S. Office of Naval Research, the U.S. Air Force Research Laboratory, and the U.S. Air Force Artificial Intelligence Accelerator.

Author: Adam Zewe Source: MIT Contact: Adam Zewe MIT Image: The image is credited to Neuroscience News

Original Research: The findings will be presented at the International Conference on Learning Representations

More here:
Reducing Toxic AI Responses - Neuroscience News

For patients experiencing cognitive decline due to Alzheimers Disease (AD), choosing the most appropriate treatment course at the right time is of great importance. A key element to these decisions is the careful consideration of the available scientific evidence, particularly from randomized clinical trials (RCTs) such as the recent lecanemab trial. Translating RCT results into patient-level decisions, however, can be challenging. This is because trial results tell us about the outcomes of groups rather than individuals. A doctor must judge how similar their patient is to the groups studied in trials. For AD, where patients vary widely in clinical presentations and rates of cognitive decline, this may be a difficult task.

As a step towards more personalized decision-making, prescribing physicians may focus on specific patient characteristics that would affect the disease course and response to treatment, like demographics (e.g., sex, age, education) or genetic factors. In fact, subgroup analyses from some RCTs suggest that at least some drugs could differ in safety or efficacy based on these factors. Nevertheless, the main limitations of these types of results are that the group sizes are often small, increasing the risk of spurious findings. Furthermore, they do not consider the overall impact of many different factors simultaneously. This is where machine learning (ML) may close the gap between data and decision-making.

ML uses patterns found in large datasets to predict health outcomes and treatment response by considering many patient characteristics at once and, further, how they may interact. This underlying model can subsequently be used to form a digital twin for a patient, or the best possible copy of their characteristics and health status. We can use this twin to ask what if questions. For example, If we prescribed this patient this drug at this time, what would be their most likely outcome six months from now? Under the hood, an ML algorithm would utilize previously collected data, such as from RCTs, to locate potential twins and use their outcomes to formulate a response. This could give us a more pinpointed prediction of patient outcomes compared to subgroup analyses. Ideally, this targeted view on patients would help facilitate better care for AD patients.

Roy S. Zawadzki

The stage is set for digital twins to play a bigger role in clinical research and practice in AD: we have the methodology, the data, and, most importantly, a large unmet clinical need for new and more effective treatments. Digital twins can be integrated in a wide variety of contexts that can potentially save clinical trial costs, quicken the time until approval, and better utilize the treatments we already have for the patients that need them the most. For these reasons, biotech companies, academic researchers, and healthcare systems alike should be investigating how digital twins can help assist their particular goals.

To learn more about real-world opportunities and considerations surrounding digital twins, please check out my latest post on my Substack

Roy S. Zawadzki, graduate trainee with Professor Daniel Gillen and supported by the TITAN T32 training grant

Read this article:
From data to decision-making: the role of machine learning and digital twins in Alzheimers Disease - UCI MIND

Zhou, E. & Mai, T. Electrification Futures Study: Operational Analysis of U.S. Power Systems with Increased Electrification and Demand-Side Flexibility (US National Renewable Energy Laboratory, 2021); https://www.nrel.gov/docs/fy21osti/79094.pdf

Xexakis, G. & Trutnevyte, E. Consensus on future EU electricity supply among citizens of France, Germany, and Poland: implications for modeling. Energy Strategy Rev. 38, 100742 (2021).

Article Google Scholar

Steggals, W., Gross, R. & Heptonstall, P. Winds of change: how high wind penetrations will affect investment incentives in the GB electricity sector. Energy Policy 39, 13891396 (2011).

Article Google Scholar

Brinkman, G. et al. The North American Renewable Integration Study: A U.S. Perspective (US National Renewable Energy Laboratory, 2021); https://www.nrel.gov/docs/fy21osti/79224.pdf

Boie, I., Fernandes, C., Fras, P. & Klobasa, M. Efficient strategies for the integration of renewable energy into future energy infrastructures in European analysis based on transnational modeling and case studies for nine European regions. Energy Policy 67, 170185 (2014).

Article Google Scholar

Sun, X., Zhang, B., Tang, X., McLellan, B. C. & Hk, M. Sustainable energy transitions in China: renewable options and impacts on the electricity system. Energies 9, 980 (2016).

Article Google Scholar

Carvallo, J. et al. A Guide for Improved Resource Adequacy Assessments in Evolving Power Systems: Institutional and Technical Dimensions (Ernest Orlando Lawrence Berkeley National Laboratory, 2023); https://eta-publications.lbl.gov/sites/default/files/ra_project_-_final.pdf

Stenclik, D. et al. Redefining Resource Adequacy for Modern Power Systems (Energy Systems Integration Group, 2021); https://www.esig.energy/wp-content/uploads/2022/12/ESIG-Redefining-Resource-Adequacy-2021-b.pdf

Auffhammer, M., Baylis, P. & Hausman, C. H. Climate change is projected to have severe impacts on the frequency and intensity of peak electricity demand across the United States. Proc. Natl Acad. Sci. USA 114, 18861891 (2017).

Article Google Scholar

Huang, J. & Gurney, K. R. Impact of climate change on U.S. building energy demand: sensitivity to spatiotemporal scales, balance point temperature, and population distribution. Clim. Change 137, 171185 (2016).

Article Google Scholar

Craig, M. T. et al. A review of the potential impacts of climate change on bulk power system planning and operations in the United States. Renew. Sustain. Energy Rev. 98, 255267 (2018).

Article Google Scholar

Bloomfield, H. C., Brayshaw, D. J., Shaffrey, L. C., Coker, P. J. & Thornton, H. E. Quantifying the increasing sensitivity of power systems to climate variability. Environ. Res. Lett. 11, 124025 (2016).

Article Google Scholar

Yalew, S. G. et al. Impacts of climate change on energy systems in global and regional scenarios. Nat. Energy 5, 794802 (2020).

Article Google Scholar

Craig, M. T., Jaramillo, P., Hodge, B.-M., Nijssen, B. & Brancucci, C. Compounding climate change impacts during high stress periods for a high wind and solar power system in Texas. Environ. Res. Lett. 15, 024002 (2020).

Article Google Scholar

Dowling, P. The impact of climate change on the European energy system. Energy Policy 60, 406417 (2013).

Article Google Scholar

Craig, M. T. et al. Overcoming the disconnect between energy system and climate modeling. Joule 6, 14051417 (2022).

Article Google Scholar

Tapiador, F. J., Navarro, A., Moreno, R., Snchez, J. L. & Garca-Ortega, E. Regional climate models: 30 years of dynamical downscaling. Atmos. Res. 235, 104785 (2020).

Article Google Scholar

Pierce, D. W., Cayan, D. R. & Thrasher, B. L. Statistical downscaling using localized constructed analogs (LOCA). J. Hydrometeorol. 15, 25582585 (2014).

Article Google Scholar

Wood, A. W., Leung, L. R., Sridhar, V. & Lettenmaier, D. P. Hydrologic implications of dynamical and statistical approaches to downscaling climate model outputs. Clim. Change 62, 189216 (2004).

Article Google Scholar

Kaczmarska, J., Isham, V. & Onof, C. Point process models for fine-resolution rainfall. Hydrol. Sci. J. 59, 19721991 (2014).

Article Google Scholar

Vandal, T., Kodra, E. & Ganguly, A. R. Intercomparison of machine learning methods for statistical downscaling: the case of daily and extreme precipitation. Theor. Appl. Climatol. 137, 557570 (2019).

Article Google Scholar

Stengel, K., Glaws, A., Hettinger, D. & King, R. N. Adversarial super-resolution of climatological wind and solar data. Proc. Natl Acad. Sci. USA 117, 1680516815 (2020).

Article Google Scholar

Tran, D. T. et al. GANs enabled super-resolution reconstruction of wind field. J. Phys. Conf. Ser. 1669, 012029 (2020).

Article Google Scholar

Kim, J., Lee, J. K. & Lee, K. M. Deeply-recursive convolutional network for image super-resolution. in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 16371645 (2016).

Hess, P., Drke, M., Petri, S., Strnad, F. M. & Boers, N. Physically constrained generative adversarial networks for improving precipitation fields from Earth system models. Nat. Mach. Intell. https://doi.org/10.1038/s42256-022-00540-1 (2022).

Goodfellow, I. et al. Generative adversarial nets. In Proc. Advances in Neural Information Processing Systems Vol. 27 (eds Ghahramani, Z. et al.) (Curran Associates, Inc., 2014).

Di Luca, A., de Ela, R. & Laprise, R. Potential for small scale added value of RCMs downscaled climate change signal. Clim. Dyn. 40, 601618 (2013).

Article Google Scholar

Flato, G. et al. in IPCC Climate Change 2013: The Physical Science Basis Ch. 9 (eds Stocker, T. F. et al.) (IPCC, Cambridge Univ. Press, 2013).

Yukimoto, S. et al. MRI MRI-ESM2.0 Model Output Prepared for CMIP6 C4MIP esm-ssp585 Version 20191108 (WDC Climate, 2019); https://doi.org/10.22033/ESGF/CMIP6.6811

EC-Earth Consortium (EC-Earth). EC-Earth-Consortium EC-Earth3 Model Output Prepared for CMIP6 CMIP esm-ssp585, Version 20200310 (Earth System Grid Federation, 2019); https://doi.org/10.22033/ESGF/CMIP6.4700

Kao, S.-C. et al. The Third Assessment of the Effects of Climate Change on Federal Hydropower (OSTI, 2022); https://www.osti.gov/biblio/1887712/

Martinez, A. & Iglesias, G. Climate change impacts on wind energy resources in North America based on the CMIP6 projections. Sci. Total Environ. 806, 150580 (2022).

Article Google Scholar

Sengupta, M. et al. The National Solar Radiation Data Base (NSRDB). Renew. Sustain. Energy Rev. 89, 5160 (2018).

Article Google Scholar

Draxl, C., Clifton, A., Hodge, B.-M. & McCaa, J. The Wind Integration National Dataset (WIND) Toolkit. Appl. Energy 151, 355366 (2015).

Article Google Scholar

James, E. P. et al. The High-Resolution Rapid Refresh (HRRR): an hourly updating convection-allowing forecast model. Part II: forecast performance. Weather Forecast. 37, 13971417 (2022).

Article Google Scholar

Jafari, S., Sommer, T., Chokani, N. & Abhari, R. S. Wind resource assessment using a mesoscale model: the effect of horizontal resolution. in Proc. ASME Turbo Expo 2012: Turbine Technical Conference and Exposition (eds Bainier, F. et al.) 987995 (American Society of Mechanical Engineers Digital Collection, 2013).

Perez, R., David, M. & Hoff, T. E. in Foundations and Trends in Renewable Energy (eds Norton, B. et al.) 144 (Now Publishers Inc., 2016).

Kolmogorov, A. N. Dissipation of energy in the locally isotropic turbulence. Proc. Math. Phys. Sci. 434, 1517 (1991).

MathSciNet Google Scholar

Holttinen, H. et al. Design and Operation of Power Systems with Large Amounts of Wind Power: Final Summary Report, IEA WIND Task 25, Phase Four 20152017 (VTT Technical Research Centre of Finland, 2019); https://doi.org/10.32040/2242-122X.2019.T350

Dobos, A. P. PVWatts Version 5 Manual (OSTI, 2014); https://www.osti.gov/biblio/1158421

Gueymard, C. A. REST2: high-performance solar radiation model for cloudless-sky irradiance, illuminance, and photosynthetically active radiationvalidation with a benchmark dataset. Sol. Energy 82, 272285 (2008).

Article Google Scholar

Maxwell, E. L. A Quasi-Physical Model for Converting Hourly Global Horizontal to Direct Normal Insolation (OSTI, 1987); https://www.osti.gov/biblio/5987868

Olea, R. A. in Geostatistics for Engineers and Earth Scientists (ed. Olea, R. A.) 6790 (Springer, 1999).

Stull, R. Wet-bulb temperature from relative humidity and air temperature. J. Appl. Meteorol. Climatol. 50, 22672269 (2011).

Article Google Scholar

Gelaro, R. et al. The modern-era retrospective analysis for research and applications, version 2 (MERRA-2). J. Clim. 30, 54195454 (2017).

Article Google Scholar

Atmospheric Radiation Measurement (ARM). Data Quality Assessment for ARM Radiation Data (QCRADBRS1LONG). 2015-01-01 to 2021-12-31, Southern Great Plains (SGP) Central Facility, Lamont, OK (C1) (eds Shi, Y. & Riihimaki, L.) (ARM Data Center, 1993); https://doi.org/10.5439/1027745

Brinkman, G. et al. The North American Renewable Integration Study (NARIS): A U.S. Perspective (OSTI, 2021); https://www.osti.gov/biblio/1804701

Peacock, J. A. Two-dimensional goodness-of-fit testing in astronomy. Mon. Not. R. Astron. Soc. 202, 615627 (1983).

Article Google Scholar

Novacheck, J. et al. The Evolving Role of Extreme Weather Events in the U.S. Power System with High Levels of Variable Renewable Energy (OSTI, 2021); https://www.osti.gov/biblio/1837959

IPCC Climate Change 2023: Synthesis Report Contribution of Working Groups I, II and III to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change (eds Lee, H. & Romero, J.) 184 (IPCC, 2023).

Ralston Fonseca, F. et al. Climate-induced tradeoffs in planning and operating costs of a regional electricity system. Environ. Sci. Technol. 55, 1120411215 (2021).

Article Google Scholar

Avery, C. W. et al. in Impacts, Risks, and Adaptation in the United States: Fourth National Climate Assessment Vol. II (eds Reidmiller, D. R. et al.) 14131430 (US Global Change Research Program, 2018).

Draxl, C., Hodge, B. M., Clifton, A. & McCaa, J. Overview and Meteorological Validation of the Wind Integration National Dataset Toolkit (OSTI, 2015); https://www.osti.gov/biblio/1214985

Hassanaly, M., Glaws, A., Stengel, K. & King, R. N. Adversarial sampling of unknown and high-dimensional conditional distributions. J. Comput. Phys. 450, 110853 (2022).

Article MathSciNet Google Scholar

Wootten, A., Terando, A., Reich, B. J., Boyles, R. P. & Semazzi, F. Characterizing sources of uncertainty from global climate models and downscaling techniques. J. Appl. Meteorol. Climatol. 56, 32453262 (2017).

Article Google Scholar

Karnauskas, K. B., Lundquist, J. K. & Zhang, L. Southward shift of the global wind energy resource under high carbon dioxide emissions. Nat. Geosci. 11, 3843 (2018).

Article Google Scholar

Cohen, J. et al. Divergent consensuses on Arctic amplification influence on midlatitude severe winter weather. Nat. Clim. Chang. 10, 2029 (2020).

Article Google Scholar

Voigt, A. et al. Clouds, radiation, and atmospheric circulation in the present-day climate and under climate change. WIREs Clim. Change 12, e694 (2021).

Article Google Scholar

Springenberg, J. T., Dosovitskiy, A., Brox, T. & Riedmiller, M. A. Striving for simplicity: the all convolutional net. in CoRR Vol. abs/1412.6806 (2014).

He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. in Proc. IEEE Conference on Computer Vision and Pattern Recognition 770778 (2016).

He, K., Zhang, X., Ren, S. & Sun, J. Identity mappings in deep residual networks. in Proc. Computer VisionECCV 2016 (eds Leibe, B. et al.) 630645 (Springer International Publishing, 2016).

Shi, W. et al. Is the deconvolution layer the same as a convolutional layer? Preprint at arXiv http://arxiv.org/abs/1609.07009 (2016).

Federal Aviation Administration. in Pilots Handbook of Aeronautical Knowledge Ch. 4 (FAA, US Government, 2023).

Ho, C. K., Stephenson, D. B., Collins, M., Ferro, C. A. T. & Brown, S. J. Calibration strategies: a source of additional uncertainty in climate change projections. Bull. Am. Meteorol. Soc. 93, 2126 (2012).

Article Google Scholar

Read the original post:
High-resolution meteorology with climate change impacts from global climate model data using generative machine ... - Nature.com

COMMENTARY

The days of large, monolithic apps are withering. Today's applications rely on microservices and code reuse, which makes development easier but creates complexity when it comes to tracking and managing the components they use.

This is why thesoftware bill of materials (SBOM)has emerged as an indispensable tool for identifying what's in a software app, including the components, versions, and dependencies that reside within systems. SBOMs also deliver deep insights into dependencies, vulnerabilities, and risks that factor into cybersecurity.

An SBOM allows CISOs and other enterprise leaders to focus on what really matters by providing an up-to-date inventory of software components. This makes it easier to establish and enforce strong governance and spot potential problems before they spiral out of control.

Yet in the age of artificial intelligence (AI), the classic SBOM has some limitations. Emerging machine learning (ML) frameworks introduce remarkable opportunities, but they also push the envelope on risk and introduce a new asset to organizations: the machine learning model. Without strong oversight and controls over these models, an array ofpractical, technical, and legal problemscan arise.

That's where machine learning bills of materials (MLBOMs) enter the picture. The framework tracks names, locations, versions, and licensing for assets that comprise an ML model. It also includes overarching information about the nature of the model, training configurations embedded in metadata, who owns it, various feature sets, hardware requirements, and more.

CISOs are realizing that AI and ML require a different security model and the underlying training data and models that run them are frequently not tracked or governed. An MLBOM can help an organization avoid security risks and failures. It addresses critical factors like model and data provenance, safety ratings, and dynamic changes that extend beyond the scope of SBOM.

Because ML environments are in a constant state of flux and changes can take place with little or no human interaction, issues related to data consistency including where it originated, how it was cleaned, and how it was labeled are a constant concern.

For example, if a business analyst or data scientist determines that a data set is poisoned, the MLBOM simplifies the task of findingall the various touch points and models that were trained with that data.

Transparency, auditability, control, and forensic insight are all hallmarks of an MLBOM. With a comprehensive view of the "ingredients" that go into an ML model, an organization is equipped to manage its ML models safely.

Here are some ways to build a best practice framework around an MLBOM:

Recognize the need for an MLBOM:It's no secret that ML fuels business innovation and even disruption. Yet it also introduces significant risks that can extend to reputation, regulatory compliance, and legal issues. Having visibility into ML models is critically important.

Conduct essential due diligence:An MLBOM should integrate withthe CI/CD pipelineand deliver a high level of clarity. Support for standard frameworks like JSON or OWASP'sCycloneDXcanunify SBOM and MLBOM processes.

Analyze policies, processes, and governance:It's essential to sync an MLBOM with an organization's workflows and business processes. This increases the odds that ML pipelines will work as intended, while minimizing risks related to cybersecurity, data privacy, compliance, and other risk-associated areas.

Use an MLBOM with machine learning gates:Rigorous controls and gateways lead to essential AI and ML guardrails. In this way, the business and the CSO can build on successes and harness ML to unlock greater cost savings, performance gains, and business value.

Machine learning is radically changing the business and IT landscape. By extending proven SBOM methodologies to ML through MLBOMs, it's possible to take a giant step toward boosting machine learning performance and protecting data and assets.

Read more:
Why MLBOMs Are Useful for Securing the AI/ML Supply Chain - Dark Reading

Blog Article

Published April 8, 2024

By Nick Fetty

Yann LeCun, a Turing Award winning computer scientist, had a wide-ranging discussion about artificial intelligence (AI) with Nicholas Dirks, President and CEO of The New York Academy of Sciences, as part of the first installment of the Tata Series on AI & Society on March 14, 2024.

LeCun is the Vice President and Chief AI Scientist at Meta, as well as the Silver Professor for the Courant Institute of Mathematical Sciences at New York University. A leading researcher in machine learning, computer vision, mobile robotics, and computational neuroscience, LeCun has long been associated with the Academy, serving as a featured speaker during past machine learning conferences and also as a juror for the Blavatnik Awards for Young Scientists.

As a postdoc at the University of Toronto, LeCun worked alongside Geoffrey Hinton, whos been dubbed the godfather of AI, conducting early research in neural networks. Some of this early work would later be applied to the field of generative AI. At this time, many of the fields foremost experts cautioned against pursuing such endeavors. He shared with the audience what drove him to pursue this work, despite the reservations some had.

Everything that lives can adapt but everything that has a brain can learn, said LeCun. The idea was that learning was going to be critical to make machines more intelligent, which I think was completely obvious, but I noticed that nobody was really working on this at the time.

LeCun joked that because of the fields relative infancy, he struggled at first to find a doctoral advisor, but he eventually pursued a PhD in computer science at the Universit Pierre et Marie Curie where he studied under Maurice Milgram. He recalled some of the limitations, such as the lack of large-scale training data and limited processing power in computers, during those early years in the late 1980s and 1990s. By the early 2000s, he and his colleagues began developing a research community to revive and advance work in neural networks and machine learning.

Work in the field really started taking off in the late 2000s, LeCun said. Advances in speech and image recognition software were just a couple of the instances LeCun cited that used neural networks in deep learning applications. LeCun said he had no doubt about the potential of neural networks once the data sets and computing power was sufficient.

Large language models (LLMs), such as ChatGPT or autocomplete, use machine learning to predict and generate plausible language. While some have expressed concerns about machines surpassing human intelligence, LeCun admits that he takes an unpopular opinion in thinking that he doesnt think LLMs are as intelligent as they may seem.

LLMs are developed using a finite number of words, or more specifically tokens which are roughly three-quarters of a word on average, according to LeCun. He said that many LLMs are developed using as many as 10 trillion tokens.

While much consideration goes into deciding what tunable parameters will be used to develop these systems, LeCun points out that theyre not trained for any particular task, theyre basically trained to fill in the blanks. He said that more than just language needs to be considered to develop an intelligent system.

Thats pretty much why those LLMs are subject to hallucinations, which really you should call confabulations. They cant really reason. They cant really plan. They basically just produce one word after the other, without really thinking in advance about what theyre going to say, LeCun said, adding that we have a lot of work to do to get machines to the level of human intelligence, were nowhere near that.

LeCun argued that to have a smarter AI, these technologies should be informed by sensory input (observations and interactions) instead of language inputs. He pointed to orangutans, which are highly intelligent creatures that survive without using language.

Part of LeCuns argument for why sensory inputs would lead to better AI systems is because the brain processes these inputs much faster. While reading text or digesting language, the human brain processes information at about 12 bytes per second, compared to sensory inputs from observations and interactions, which the brain processes at about 20 megabytes per second.

To build truly intelligent systems, theyd need to understand the physical world, be able to reason, plan, remember and retrieve. The architecture of future systems that will be capable of doing this will be very different from current large language models, he said.

As part of his work with Meta, LeCun uses and develops AI tools to detect content that violates the terms of services on social media platforms like Facebook and Instagram, though he is not directly involved with the moderation of content itself. Roughly 88 percent of content removed is initially flagged by AI, which helps his team in taking down roughly 10 million items every three months. Despite these efforts, misinformation, disinformation, deep fakes, and other manipulated content continue to be problematic, though the means for detecting this content automatically has vastly improved.

LeCun referenced statistics stating that in late 2017, roughly 20 to 25 percent of hate speech content was flagged by AI tools. This number climbed to 96 percent just five years later. LeCun said this difference can be attributed to two things: first the emergence of self-supervised, language-based AI systems (which predated the existence of ChatGPT); and second, is the transformer architecture present in LLMs and other systems. He added that these systems can not only detect hate speech, but also violent speech, terrorist propaganda, bullying, fake news and deep fakes.

The best countermeasure against these [concerns] is AI. AI is not really the problem here, its actually the solution, said LeCun.

He said this will require a combination of better technological systems, The AI of the good guys have to stay ahead of the AI of the bad guys, as well as non-technological, societal input to easily detect content produced or adapted by AI. He added that an ideal standard would involve a watermark-like tool that verifies legitimate content, as opposed to a technology tasked with flagging inauthentic material.

LeCun pointed to a study by researchers at New York University which found that audiences over the age of 65 are most likely to be tricked by false or manipulated content. Younger audiences, particularly those who grew up with the internet, are less likely to be fooled, according to the research.

One element that separates Meta from its contemporaries is the formers ability to control the AI algorithms that oversee much of its platforms content. Part of this is attributed to LeCuns insistence on open sourcing their AI code, which is a sentiment shared by the company and part of the reason he ended up at Meta.

I told [Meta executives] that if we create a research lab well have to publish everything we do, and open source our code, because we dont have a monopoly on good ideas, said LeCun. The best way I know, which I learned from working at Bell Labs and in academia, of making progress as quickly as possible is to get as many people as possible contributing to a particular problem.

LeCun added that part of the reason AI has made the advances it has in recent years is because many in the industry have embraced the importance of open publication, open sourcing and collaboration.

Its an ecosystem and we build on each others ideas, LeCun said.

Another advantage is that open sourcing lessens the likelihood of a single company developing a monopoly over a particular technology. LeCun said a single company simply does not have the ability to finetune an AI system that will adequately serve the entire population of the world.

Many of the early systems have been developed using English, where data is abundant, but, for example, different inputs will need to be considered in a country such as India, where 22 different official languages are spoken. These inputs can be utilized in a way that a contributor doesnt need to be literate simply having the ability to speak a language would be enough to create a baseline for AI systems that serve diverse audiences. He said that freedom and diversity in AI is important in the same way that freedom and diversity is vital to having an independent press.

The risk of slowing AI is much greater than the risk of disseminating it, LeCun said.

Following a brief question and answer session, LeCun was presented with an Honorary Life Membership by the Academys President and CEO, Nick Dirks.

This means that youll be coming back often to speak with us and we can all get our questions answered, Dirks said with a smile to wrap up the event. Thank you so much.

Read the original here:
Yann LeCun Emphasizes the Promise AI - NYAS - The New York Academy of Sciences