With the advent of large language models (LLMs), such as OpenAI's GPT series or Meta Llamas models, we’ve significantly enhanced our capabilities. We’ve transitioned to an even more advanced AI system that we use both internally to build and externally that integrates into the products we deliver to optimize finance operations. This approach has allowed us to refine AI-centric processes that streamline workflows, accelerate product development, and bring innovative solutions to our customers.
However, while third-party large language models (LLMs) offer significant benefits, we've also recognized the growing potential of open-source LLMs and smaller, specialized language models, which provide several advantages for niche use cases and domain-specific tasks.
Join us as we explore our journey in integrating cutting-edge generative AI, the advantages and challenges of using third-party LLMs, the opportunities in the open-source AI space, and why small language models will become the hallmark set of valuable solutions for specific applications.
Third-party LLMs, particularly those from OpenAI such as GPT-3.5 and GPT-4, have provided an immediate and impactful solution for enhancing our product capabilities. These models come pre-trained on vast amounts of data, enabling high-level language understanding and complex reasoning without the need for extensive in-house training. This “out-of-the-box” utility significantly reduces our time-to-market, enabling us to bring AI-powered features to customers faster and with fewer development resources.
One major advantage of using third-party LLMs is cost efficiency. Developing a foundational model from scratch requires substantial investment in computational resources and specialized expertise. By leveraging models that are already trained and maintained by external providers, we avoid this upfront cost while benefiting from their high performance.
However, relying on external LLMs isn’t without challenges. Data privacy and service robustness concerns have been key considerations in our approach. When dealing with sensitive financial data, safeguarding confidentiality is paramount. To address these concerns, we apply anonymization and denonymization techniques to protect sensitive information before interacting with external APIs. While this ensures compliance with data privacy regulations, it also adds complexity to our workflows.
The open-source LLM space has seen rapid growth, with initiatives led by companies like Meta (LLaMA), Microsoft Phi, Mistral, Databricks, and Snowflake. These foundational models are continuously improving, with some even outperforming established models such as GPT-3.5, and inching closer to the capabilities of GPT-4. This has opened up new opportunities for a company such as Auditoria to harness high-performing models without being locked into third-party ecosystems.
One key advantage of open-source models is their flexibility. By deploying these models on-premise, in our own secured environment, we retain full control over how the AI is used and ensure the highest level of data security. This also enables us to fine-tune models with proprietary data, optimizing performance for our specific use cases, such as automating financial processes, highly-accurate data extraction, next-level intent detection, and extracting actionable insights from corporate communications.
Furthermore, open-source LLMs allow for incremental training and continuous improvement. Unlike static third-party models, open-source LLMs are able to be retrained and customized over time, ensuring that they stay aligned with evolving business requirements. This flexibility also translates to cost savings, as companies optimize resource usage by selecting the most appropriate models for specific tasks rather than relying on a one-size-fits-all approach.
While the trend towards larger language models continues to dominate the AI landscape, small language models (SLMs) remain a powerful tool for many applications. SLMs offer several advantages that make them particularly appealing for domain-specific tasks, similar to those we handle at Auditoria.
SLMs are significantly cheaper to train and operate than their larger counterparts. This can be a game-changer for businesses that are budget-conscious or operate on tighter margins.
SLMs require far less computational power, making them easier to deploy on a variety of platforms, including edge devices and servers with limited resources.
Reduced Environmental Impact
The massive training and operational requirements of LLMs contribute to significant energy consumption and carbon emissions. SLMs, by contrast, have a much smaller carbon footprint, making them a more environmentally sustainable choice.
SLMs excel when fine-tuned on smaller, curated datasets tailored to specific industries or tasks. In our case, we've found that for specialized domains like corporate finance, SLMs can sometimes outperform general-purpose LLMs because they are trained to handle more focused types of language and data. By tailoring SLMs to our specific workflows, we've achieved high levels of accuracy in tasks like intent recognition and information extraction from financial documents.
SLMs are easier to customize and tweak, allowing for greater control over their behavior. For instance, at Auditoria, we've customized SLMs to understand the nuances of email communications related to accounts receivable and payable, improving the accuracy of our automation tools.
Despite their benefits, SLMs do have limitations. Their smaller size means they may lack the deep understanding or reasoning capabilities of larger models, which can be a drawback in tasks requiring complex language comprehension. Fine-tuning SLMs also requires specialized expertise, particularly when applying techniques like model distillation or compression to ensure optimal performance.
At Auditoria, we have adopted a hybrid approach, deploying both large and small language models depending on the specific use case.
Rather than relying on a single model for all tasks, we select the most appropriate model for each scenario. This allows us to balance cost, performance, and scalability while ensuring data privacy and maintaining high levels of automation accuracy.
The integration of generative AI, whether through third-party LLMs, open-source models, or small language models, has revolutionized Auditoria's approach to automation.
By leveraging a multi-model strategy, we’ve been able to optimize performance, reduce costs, and enhance customization while ensuring robust data privacy and enterprise-grade service quality.