Cloud Hosting

If 2022 was the year when AI broke through to become a society-changing technology, 2023 has been the year of AI breaking through in the enterprise. Put it this way: generative AI and large language models (LLMs) have become a familiar part of the lexicon for IT departments the world over. Your CIO is now more likely to namecheck ChatGPT than Kubernetes.

Already, its been a year of giant strides forward for AI in the enterprise, so here are five things weve learned so far.

In March, OpenAI announced an enterprise version of ChatGPT. But in this case, OpenAI was a fast follower not the first-to-market. Cohere, a Toronto-based company with close ties to Google, was already selling generative AI to businesses, as I discovered when I spoke to them that month.

Were working with developers in organizations, the AI/ML teams, to bring these capabilities into their organizations, Coheres CEO Martin Kon told us. He added that its approach is fundamentally different from OpenAIs. OpenAI wants you to bring your data to their models, exclusive to Azure. Cohere wants to bring our models to your data, in whatever environment you feel comfortable in.

So far, companies are mostly using generative AI to create semantic search engines for their own private data whether for internal use or for external customers. A related use case is knowledge management (KM). Imagine an employee being able to have a conversation with an AI that was trained on large language models based on company data.

One of the many new AI companies trying to achieve KM in chatbot form is Vectara. Its CEO Amr Awadallah told me that in five years, every single application whether that be on the consumer side or in the business/enterprise side will be re-architected in a way that is a lot more human in nature, in terms of how we express what we are trying to achieve and what were trying to do.

Last month, Google Cloud and Googles Workspace division announced a raft of new AI functionality. These included Generative AI App Builder, which allows organizations to build their own AI-powered chat interfaces and digital assistants, and new generative AI features in Google Workspace.

Google has coined a typically awkward term for applications powered by generative AI: gen apps. It claims that gen apps will become a third major internet application category, after web apps and mobile apps.

Im willing to wager that gen apps as a term will never take off but, regardless, Googles AI-powered tools will be well used within enterprise companies.

Likewise, Microsoft has been releasing new AI tools, such as Semantic Kernel (SK), described as an open-source project helping developers integrate cutting-edge AI models quickly and easily into their apps. SK is in many ways a classic Microsoft low-code tool, only it focuses on helping users do prompts for AI chatbots.

If you scan Stanfords HELM website, which measures LLMs in a variety of ways, youll see that these models vary greatly in size. There are, simply put, tradeoffs between model size and the speed it can work at.

OpenAI has several models, ranging in size from 1.3 billion parameters (its Babbage model) to 175B parameters (its DaVinci model).

Cohere goes as far as differentiating its model sizes as if they were Starbucks cups: small, medium, large, and extra large.

List of Coheres models in Stanford HELM directory

The primary models of OpenAI

However, Stanford also measures accuracy and in these statistics, size doesnt seem to matter much:

Stanford HELM tests for accuracy of ML models

In this new era of generative AI, frameworks like Ray are now just as important as Kubernetes in creating modern applications at scale. An open source platform called Ray offers a distributed machine-learning framework. Its being used by both OpenAI and Cohere to help train their models. Its also used in other highly-scaled products, such as Uber.

The company behind Ray is Anyscale, whose CEO Robert Nishihara told me earlier this year is very developer-centric. All of Rays functionality is designed to be easy to use for developers, he said. This differs from the Kubernetes user experience for devs, he noted, which is notoriously difficult. Ray was designed for Python developers, which is the main programming language being used in AI systems.

Anyscale co-founder Ion Stoica told me that Ray is like an extension of Python and like Python there are a set of Ray libraries that are targeted towards different use cases. The awkwardly named RLlib is for reinforcement learning, but there are similar libraries for training, serving, data pre-processing, and more.

Just as cloud computing ushered in a raft of big data solutions, generative AI is a catalyst for a new wave of data intelligence companies.

I recently spoke to Aaron Kalb, a co-founder of Alation, which styles itself as a data intelligence platform and promotes a concept it calls the data catalog. This combines machine learning with human curation to create a custom store of data for enterprise companies.

Kalb noted that both AI and the common enterprise acronym of BI (business intelligence) are garbage in, garbage out. He said that data intelligence is a layer that precedes AI and BI, that makes sure you can find, understand and trust the right data to put into your AI and BI.

In this context, he said, taking something like ChatGPT from the public internet and bringing it into the enterprise is very risky. He thinks that data needs to be, well, more intelligent before it is used by AI systems within an enterprise. Also, he doesnt think that the internet scale of ChatGPT and similar systems is needed in the enterprise. Every organization has its own terminology, he explained that could be industry terms, or things that are very specific to that company.

Its hard to believe that its been just a year since generative AI burst onto the scene. It all started with OpenAIs DALL-E 2, which was announced last April and launched as a private beta in July. DALL-E 2 is an image generation service powered by deep learning models, and it was a significant step forward for the industry. Also, last July, a company called Midjourney released its eponymous text-to-image generator.

The AI hype really ramped up in August, though, with the release of Stable Diffusion another deep-learning text-to-image generator. Unlike DALL-E 2 and Midjourney, Stable Diffusion had a permissive licensing structure. This was around the time people began paying more attention to the models behind all these services the LLMs. DALL-E 2 used a version of GPT-3, OpenAIs main LLM at the time.

But, in hindsight, we can see that the image generators were just the appetizer. At the end of November, OpenAI launched ChatGPT, a chatbot built on top of GPT-3.5. This was the catalyst for generative AI to enter the enterprise, and it has quickly taken over IT departments since then.

View post:
5 Key Learnings about AI and ChatGPT in the Enterprise - The New Stack