In 2023, generative AI tools gained widespread adoption; however, they still face challenges, notably in the form of hallucinations—plausible-sounding but incorrect responses to user queries. This issue has hindered enterprise adoption, as hallucinations in critical business or customer-facing scenarios could have serious consequences. Retrieval-augmented generation (RAG) has emerged as a technique to mitigate hallucinations, potentially revolutionizing enterprise AI adoption.

RAG combines text generation with information retrieval to improve the accuracy and relevance of AI-generated content. By allowing large language models (LLMs) to access external information, RAG helps them produce more accurate and contextually aware responses. This approach also reduces the need to store all knowledge directly in the LLM, leading to smaller model sizes, faster processing speeds, and lower costs.

“You can use RAG to gather a lot of unstructured information, documents, etc., [and] feed it into a model without having to fine-tune or custom-train a model,” explained Barrington.

These advantages are particularly appealing for enterprise applications where current, factual knowledge is essential. For instance, businesses can leverage RAG with foundation models to create more effective and informative chatbots and virtual assistants.

Customized enterprise generative AI models

While large, general-purpose generative AI tools like Midjourney and ChatGPT have captured consumer attention, the business landscape is seeing a shift towards smaller, specialized models tailored to niche requirements. While developing a new model from scratch is resource-intensive, many organizations are modifying existing AI models to meet their specific needs, such as tweaking architecture or fine-tuning on domain-specific datasets. This approach can be more cost-effective than building a new model or relying on API calls to public large language models (LLMs).

Customized generative AI models offer the advantage of catering to niche markets and user needs. These tailored tools can be developed for a wide range of scenarios, from customer support to supply chain management to document review. This customization is particularly valuable in sectors with specialized terminology and practices, such as healthcare, finance, and legal industries.

For many business use cases, massive LLMs are often unnecessary. While ChatGPT might be suitable for a consumer-facing chatbot handling various queries, it may not be the best choice for smaller enterprise applications. As AI developers’ capabilities converge, enterprises are expected to explore a more diverse range of models in the coming years.

Building a customized model instead of using a public tool also enhances privacy and security, as organizations have greater control over their data. This aspect is particularly crucial for tasks involving sensitive personal data, where sending data to a third party may not be desirable.

With stricter AI regulation on the horizon, organizations may increasingly focus on proprietary models to ensure compliance and data security. This shift could lead to a greater emphasis on domain-specific, private models rather than relying solely on large, publicly available LLMs trained on diverse datasets from the internet.

Need for AI and machine learning talent

Developing, training, and deploying machine learning models is a complex task, especially in a large organizational IT environment. As the demand for AI and machine learning talent continues to grow, professionals who can bridge the gap between theory and practice are increasingly sought after. This need is particularly evident in the field of MLOps, which focuses on deploying, monitoring, and maintaining AI systems in real-world settings.

According to a recent O’Reilly report, the top skills needed for generative AI projects include AI programming, data analysis and statistics, and operations for AI and machine learning. However, these skills are currently in short supply, posing a challenge for organizations looking to leverage AI technologies.

In 2024, organizations are expected to prioritize hiring talent with these critical skills, not just in big tech companies but across industries. As AI becomes more integrated into business operations, building internal AI and machine learning capabilities will be crucial for digital transformation.

Diversity is also crucial in AI initiatives, from technical teams building models to the board overseeing these projects. Ensuring a diverse team can help challenge biases in training data and improve the overall outcomes of AI projects. “One of the big issues with AI and the public models is the amount of bias that exists in the training data,” explained Crossan. “And unless you have that diverse team within your organization that is challenging the results and challenging what you see, you are going to potentially end up in a worse place than you were before AI.”

Shadow AI

The increasing accessibility of AI is leading to a rise in shadow AI within organizations, where employees use AI tools without explicit approval or oversight from the IT department. This trend is particularly common for easy-to-use AI chatbots, which employees can quickly experiment with in their web browsers, bypassing official IT review processes.

Shadow AI often emerges when employees seek quick solutions or want to explore new technology faster than official channels allow. While this demonstrates a proactive and innovative spirit, it also poses risks, as end users may not be aware of important considerations such as security, data privacy, and compliance. For example, using public-facing LLMs without realizing the risk of exposing sensitive information, such as trade secrets, to third parties.

“There’s a bit of a fear factor and risk angle that’s appropriate for most enterprises, regardless of sector, to think through,” noted Barrington. Once data is exposed to public models, it cannot be retracted, highlighting the importance of ensuring proper oversight and approval processes for AI usage within organizations.

In 2024, organizations will need to address the challenge of shadow AI through governance frameworks that balance innovation with privacy and security concerns. This may involve establishing clear policies for acceptable AI use, providing approved platforms, and fostering collaboration between IT and business leaders to understand departmental AI needs.

According to Barrington, “The reality is, everybody’s using it.” Recent EY research found that 90% of respondents used AI at work. Therefore, it is essential for organizations to align employees with ethical and responsible AI use practices.

 A generative AI reality check

As organizations move beyond the initial excitement of generative AI and into actual adoption and integration, they are likely to encounter a phase known as the “trough of disillusionment,” as described in the Gartner Hype Cycle.

“We’re definitely seeing a rapid shift from what we’ve been calling this experimentation phase into [asking], ‘How do I run this at scale across my enterprise?'” noted Barrington.

During this phase, organizations may face challenges such as limitations in output quality, security and ethics concerns, and difficulties integrating generative AI with existing systems and workflows. The complexity of implementing and scaling AI in a business environment can be more significant than anticipated, including tasks like ensuring data quality, training models, and maintaining AI systems in production.

“It’s actually not very easy to build a generative AI application and put it into production in a real product setting,” added Luke.

Despite these challenges, this phase can lead to a more realistic and nuanced understanding of AI’s capabilities and limitations. To move past this phase successfully, organizations should set realistic expectations for AI projects, clearly tie them to business goals and practical use cases, and have a clear plan for measuring outcomes.

“If you have very loose use cases that are not clearly defined, that’s probably what’s going to hold you up the most,” explained Crossan.

Increased attention to AI ethics and security risks

The rise of deepfakes and sophisticated AI-generated content has raised concerns about misinformation, manipulation, identity theft, and fraud. AI can also enhance the effectiveness of ransomware and phishing attacks, making them more convincing and harder to detect.

Efforts are underway to develop technologies for detecting AI-generated content, but this remains a challenging task. Current AI watermarking techniques are relatively easy to bypass, and existing AI detection software can be prone to false positives.

The increasing prevalence of AI systems underscores the importance of ensuring they are transparent and fair. This includes carefully vetting training data and algorithms for bias. Crossan emphasized that these ethics and compliance considerations should be integrated throughout the AI development process.

“When implementing AI, you have to consider the controls you’ll need,” she said. “This helps you plan for regulation concurrently with your AI experimentation, rather than realizing later that you need controls.”

Luke also noted that smaller, more narrowly tailored models can be safer and more ethical. “These smaller, tuned, domain-specific models are just far less capable than the really big ones — and we want that,” he said. “They’re less likely to produce unwanted outputs because they’re not as capable.”

Evolving AI regulation