Composable-LLM-Use-Case-Guide

Download as pdf or txt
Download as pdf or txt
You are on page 1of 27

How to Evaluate, Select, and

Implement LLM Use Cases


THE ULTIMATE HOW TO GUIDE
Table of Contents
Introduction 3

LLM Use Case Categories and Examples 4

How to Evaluate & Select LLM Use Cases 7

How to Implement LLM Use Cases 10

Analyst Recommendations 13

The 6 Steps to Evaluate, Select & Implement LLM Use Cases 14

Appendix with Additional Use Case Examples 15

© Composable 2
Introduction
Enterprise business leaders are increasingly Value is defined by the efficiency gains of the
exploring the potential of Large Language task being completed with the help of LLMs
Models (LLMs) to transform their operations, vs without. Effort is the amount of time and
enhance efficiency, and drive innovation. resources needed to complete the task with the
help of LLMs vs without.
However, selecting the first use case can be
daunting given the myriad of possibilities and Additionally, we explore the considerations of
considerations involved, including if the use people and skills, risk, and revenue vs costs on
case is even suitable for LLMs or if the required use case prioritization.
data is easy enough to collect.
And finally, we review implementation best
This guide provides eight LLM use case practices and why leading organizations are
examples to help you discover the many opting for LLM software platforms to accelerate
possibilities of LLM-powered tasks and the efficiency gains, enhance outcome accuracy,
impact they can have on your business with the and support data-driven decision-making.
right technology and strategy. The examples
provided are by no means an exhaustive list.
The use cases are truly endless.

This paper also offers a structured approach to


evaluate and select LLM use cases by focusing
on two key dimensions: value and effort.

© Composable 3
GETTING STARTED

LLM Use Case Categories and Examples


To get started with LLMs, it’s important to understand the categories of LLM use cases and how
LLMs can be used to augment enterprise applications and automate business processes.

Knowledge Querying
Using LLMs to extract, retrieve, and query information from large sets of mixed structure and unstructured content.

Most enterprise information is unstructured — Composable chunks content into semantic


like contracts, memos, and emails— making it parts, enabling tiered analysis and similarity
difficult to access and utilize effectively. This search across complex use cases. This
limits decision-making and extends processing approach simplifies the conversion of
times. unstructured data into actionable insights,
solving business problems that were previously
Traditionally, extracting value from unstructured unmanageable.
content required manual schema creation
and metadata extraction, followed by further
interpretation. This process is time-consuming Benefits of a Platform Approach
and inefficient.
• Significant time and cost savings
• Extraction of valuable insights from
LLMs allow organizations to interact with
existing information
information based on its semantic meaning,
• Enhanced decision-making and faster
bypassing the need for manual metadata.
processes
• Improved risk management and
For example, an LLM can identify and
automated knowledge creation
categorize problematic clauses in contracts
• Unlocking hidden value from
based on contract size. This capability extends
organizational data
to various business areas, such as call centers,
invoices, and emails, unlocking valuable insights
from unstructured content and making them
Example Use Cases:
accessible through existing systems or chat
• Occurrences Identification of a Problematic
interfaces.
Clause
• Supplier Risk Identification
With an LLM software platform like
Composable, unstructured information • Financial Audit Preparation
is semantically typed, structured, and • Product Recall Management
enriched with business-specific knowledge,
automatically.
© Composable 4
Content Analysis & Reasoning
Using LLMs to process and understand content to derive insights or make decisions.

Information analysis in organizations is often An end-to-end LLM software platform allows


time-consuming, inconsistent, and costly due enterprises to quickly create robust analysis
to the reliance on skilled analysts. While AI tasks on both structured and unstructured
can’t fully replace human analysts, LLMs can information and refine prompts that handle both
accelerate analysis, improve consistency, and structured and unstructured data.
scale operations, allowing analysts to focus on
critical tasks that require their expertise. Composable enables the rapid development
and refinement of your analysis framework,
Traditional enterprise analysis depends automates data preparation and supports
on structured data, requiring extensive large-scale execution with error management,
preprocessing to fit predefined formats. This extended processing, and retries.
inflexibility creates information silos, leading to
inconsistent data and redundant efforts when It also enables multi-step analysis by using
new analyses are needed. interim results from initial tasks as inputs for
subsequent processes, ensuring thorough
LLMs enable the dynamic analysis content evaluation.
of raw information without extensive
preprocessing. By using structured rules and
guidelines, LLMs ensure consistent analysis Benefits of a Platform Approach
across large datasets and allow for rapid
• Efficient use of skilled analysts
iteration and adaptation to new inputs.
• Scalable, standardized analysis
• Flexibility to adapt to new business needs
Leveraging the prior example, after identifying
• Cost-effective analysis of previously
a problematic clause in a contract, an LLM
resource-intensive areas
can analyze its risk level using the company’s
legal policy, providing a detailed compliance
assessment. High-risk contracts are flagged for
legal review, while low-risk ones are processed
Example Use Cases
accordingly, streamlining the overall workflow. • Proposal Evaluation & Scoring
• Large Document Review
• Due Diligence Analysis
• Compliance Analysis
• Legal Discovery and Case Analysis

© Composable 5
Generating or Repurposing Content
Using LLMs to create original content, repurpose existing content, or turn unstructured content into a more structured,
personalized format.

With 90% of the world’s content created in the Generating short-form content with LLMs is
last two years, the pace of content production often straightforward, but creating complex
is accelerating. Enterprises rely on content to long-form content that integrates multiple
communicate, and much of this content builds inputs is more challenging.
on existing materials like meeting notes, emails,
and documents. Effective content creation Composable addresses these challenges by
from diverse inputs is essential for empowering offering tools that streamline the most complex
employees and engaging customers. short and long-form content creation.

Content creation requires diverse inputs (e.g. Enterprises can easily integrate their inputs,
ideas, documents, emails) and a clear output which are optimized for LLMs, specify the
structure (e.g., white papers, briefs). output format, and leverage Composable’s
workflow engine and content generation
Enterprises often struggle to process these algorithms to produce high-quality content
inputs efficiently and produce high-quality while overcoming LLM token limits.
content across various formats. Without a
well-defined output format, content creation
can suffer in speed, quality, and consistency, Benefits of a Platform Approach
making the process time-consuming and • Quickly generate scalable content for
challenging. publishing or refinement
• Integrate multiple sources for content
LLMs can generate initial drafts quickly, creation
allowing humans to refine and edit as • Standardize and control input/output
needed. Often, the draft from an LLM is formats
sufficient, streamlining the content creation • Overcome LLM response token limits
process.

For example, after identifying and analyzing


Example Use Cases
problematic clauses in contracts, an LLM can
• Documentation Generation & Maintenance
generate a comprehensive report by integrating
• Code Generation for Tooling
source content and insights, ensuring a
• Content Summarization for Marketing
cohesive and informed output.
• Personalized Customer Communication
• Training and Educational Materials Creation

© Composable 6
HOW TO EVALUATE LLM USE CASES

Value vs Effort Framework


To evaluate and select LLM use cases, you should consider two primary dimensions: value and effort.
Let’s start with value.

Use Case Value


Value refers to the expected benefits of the use case to your organization. High-value use cases
are those that can significantly enhance efficiency, reduce costs, increase revenue, or provide a
competitive advantage. Consider the following when assessing value:

Business Impact Strategic Alignment Competitive Advantage


How will the use case affect key Does the use case align with Will the use case provide
business metrics such as revenue, your organization’s strategic a unique advantage over
costs, and customer satisfaction? goals and priorities? competitors?

Additional Dimensions to Consider


Revenue vs Costs Risks
• Will the use case reduce operational costs? • What are the potential risks associated with
• Will the use case generate new revenue the use case (e.g., technical, operational,
streams? financial, reputational)?
• What new costs might be introduced by • What is the likelihood of these risks
implementing the use case? materializing?
• What mitigation strategies can be put in
place to manage these risks?

VALUE NO VALUE HIGH VALUE


�10 �9 �8 �7 �6 �5 �4 �3 �2 �1 0 1 2 3 4 5 6 7 8 9 10
EXPECTED
BENEFITS 0

LOW RISK MEDIUM RISK HIGH RISK


�1 �0�5 �1

RISK 0

LOW COST / HIGH REVENUE MEDIUM COST / MEDIUM REVENUE HIGH COST / LOW REVENUE
�1 �0�5 �1
COST VS
REVENUE
0

TOTAL VALUE SCORE

LOW
EFFORT EFFORT HIGH EFFORT
�10 �9 �8 �7 �6 �5 �4 �3 �2 �1 0 1 2 3 4 5 6 7 8 9 10
COMPLEXITY,
RESOURCES & 0
TIME

© Composable 7
NO ADDITIONAL PEOPLE OR TRAINING NEEDED SOME ADDITIONAL PEOPLE OR TRAINING NEEDED MANY ADDITIONAL PEOPLE OR TRAINING NEEDED
Use Case Effort
Effort refers to the ease with which the use case can be implemented. Easy use cases are those
that can be developed and deployed with minimal effort and resources. Keep in mind that LLMs
can produce incredible results, but only when they are given the right context, prompts, and output
schema. Consider the following when assessing effort:

Complexity Resource Availability Time to Market


How well are LLMs positioned Do you have the necessary How quickly can the use case be
to perform the task given their resources, including people, developed and deployed?
capabilities and features? What skills, and budget, to implement
software is needed? the use case?
VALUE NO VALUE HIGH VALUE
�10 �9 �8 �7 �6 �5 �4 �3 �2 �1 0 1 2 3 4 5 6 7 8 9 10
EXPECTED
BENEFITS 0

Additional Dimensions to Consider LOW RISK


�1
MEDIUM RISK
�0�5
HIGH RISK
�1

RISK 0
People Knowledge Training Costs
How many people are required Do you need to hire new What are the costs associated
LOW COST / HIGH REVENUE MEDIUM COST / MEDIUM REVENUE HIGH COST / LOW REVENUE
�1 �0�5 �1
COST VS
to develop and maintain the use
REVENUE
employees with specific skills? with hiring, training, and retaining
0

case? the necessary talent? TOTAL VALUE SCORE

LOW
EFFORT EFFORT HIGH EFFORT
�10 �9 �8 �7 �6 �5 �4 �3 �2 �1 0 1 2 3 4 5 6 7 8 9 10
COMPLEXITY,
RESOURCES & 0
TIME

NO ADDITIONAL PEOPLE OR TRAINING NEEDED SOME ADDITIONAL PEOPLE OR TRAINING NEEDED MANY ADDITIONAL PEOPLE OR TRAINING NEEDED
�1 �0�5 1

PEOPLE & SKILLS 0

TOTAL EFFORT SCORE

If you don’t have the right software in place today, potentially all of your use cases
could require a lot of effort to get them into production. An LLM software platform, like
Composable, significantly reduces the amount of effort needed to design, test, deploy,
operate, and monitor multiple LLM use cases.

© Composable 8
A Real World Example
A leading philanthropic organization receives thousands of grant proposal applications multiple
times per year. They rely on a team of volunteers to manually review and score each proposal. This
evaluation process takes months, is prone to human errors, and produces inconsistent results.

The organization assessed the expected benefits The organization assessed the amount of effort
of automating these tasks and agreed they were required to complete these tasks manually vs if they
high (+8) given the importance of grant evaluation were automated by LLMs. By leveraging an LLM
and scoring to the organization’s grant funding software platform with integrated tooling to support
process. This process however was a critical part the use case, the effort would be dramatically
of their business making it high risk (-1) but it had reduced (-9) and there would be minimal training
the potential to significantly improve operational required (+0.5).
efficiencies and reduce costs (+1).

VALUE NO VALUE HIGH VALUE


�10 �9 �8 �7 �6 �5 �4 �3 �2 �1 0 1 2 3 4 5 6 7 8 9 10
EXPECTED
BENEFITS 8

LOW RISK MEDIUM RISK HIGH RISK


�1 �0�5 �1

RISK �1

LOW COST / HIGH REVENUE MEDIUM COST / MEDIUM REVENUE HIGH COST / LOW REVENUE
�1 �0�5 �1
COST VS
REVENUE
�1

TOTAL VALUE SCORE

LOW
EFFORT EFFORT HIGH EFFORT
�10 �9 �8 �7 �6 �5 �4 �3 �2 �1 0 1 2 3 4 5 6 7 8 9 10
COMPLEXITY,
RESOURCES & �9
TIME

NO ADDITIONAL PEOPLE OR TRAINING NEEDED SOME ADDITIONAL PEOPLE OR TRAINING NEEDED MANY ADDITIONAL PEOPLE OR TRAINING NEEDED
�1 �0�5 1

PEOPLE & SKILLS �0�5

TOTAL EFFORT SCORE


10
NO BRAINER BIG BET
HIGH

�8�5
8

2
Grant Proposal Evaluation Use Case
VALUE

-2

-4

-6

-8

Figure 1
LOW

LOW-HANGING FRUIT FORGET IT


-10 -8 -6 -4 -2 0 2 4 6 8 10
The grant proposal evalution use case on a Value vs Effort graph shows
LOW
EFFORT
HIGH
that this use case is a No Brainer with an LLM software platform.

© Composable 9
GET YOUR LLM PROJECTS INTO PRODUCTION

How to Implement LLM Use Cases


Now that you’ve evaluated and selected your LLM use cases, it’s time to deliver efficiency gains,
lower costs, and improve business operations. An end-to-end LLM software platform can effectively
deliver on use case objectives and maximize ROI.

Use an LLM Software Platform


While LLM application development frameworks offer tools for building applications, LLM software
platforms are better suited for large organizations.

END-TO-END ENTERPRISE-READY

An LLM software platform offers A platform has the architecture to


comprehensive capabilities and support support multiple teams, use cases,
for the entire enterprise LLM lifecycle. LLMs, inference providers, and more -
From ideation and experimentation all within one UI. Governance, security,
to prompt design, task configuration, orchestration, virtualization, fine-tuning,
testing, monitoring, and optimization. A workflows, semantic RAG, multiple
platform helps prevent vendor and model integration and deployment options,
lock-in by avoiding hard-coding prompts collaboration, analytics, and monitoring
and LLMs and provides the flexibility are standard in LLM software platforms
to easily switch between inference like Composable.
providers and models.

LLM software platforms enable organizations to streamline the GenAI development process, allowing
them to move more quickly, build on successes, correct failures, reuse results, and broaden how and
where GenAI can be implemented.

© Composable 10
Benefits of an LLM Software Platform
Analyst research firm, IDC, recommends that organizations quickly identify, procure, and deploy
technologies that improve the efficacy, speed, repeatability, control, cost, and use of GenAI1. LLM
software platforms remain an essential puzzle piece to help organizations think beyond just the
model, including helping deliver:

Expanded GenAI use case and As organizations move through the continuum
of deploying GenAI across more and more use
user reach cases, they will need to integrate and utilize
critical, often sensitive, data sources.
LLM software platforms enable organizations
to streamline the GenAI development process,
allowing them to move more quickly, build Comprehensive data privacy
on successes, correct failures, reuse results, and security compliance,
and broaden how and where GenAI can be
implemented.
auditing, and data controls
LLM software platforms have designed


sufficient guard rails, data controls, and auditing
capabilities to ensure an organization can limit
and manage how and where their data is
LLM software platforms allow being used, maintain adequate access control,
organizations to target multiple and perform version control.
use cases in parallel, reducing
the need to serialize each use
case individually while providing a Embedded life-cycle monitoring
seamless UI and UX that reduces and management capabilities
the need for users to understand
the nuances of today’s disjointed AI Organizations spend so much time building,
testing, validating, and deploying their AI-
ecosystem’s tooling.
powered solutions that anything else becomes
an afterthought. LLM software platforms give
Matt Arcaro
DIRECTOR, COMPUTER VISION & AI organizations vital visibility into their solutions,
IDC offering real-time performance insights,
recommended optimizations, and the ability
to implement updates remotely.

1. “Moving Beyond the Model to Maximize the Effectiveness & Reach of GenAI Deployments,” written by: Matt Arcaro, Director of
Computer Vision and AI at IDC. Published July 2024.

© Composable 11
Tooling to build complex Improved solution accuracy
workflows that include the and coherence across varying
extension and integration of models and providers
third-party data sources
Organizations pursuing GenAI often follow the
GenAI applications are only as effective as their pathway of going deep into (i.e., fine-tuning)
ability to access and integrate sufficient relevant a single model or utilizing a multimodel and/or
contextual information and data. provider approach.

LLM software platforms can unify and


optimize the relationship between
structured and unstructured data sources
and LLMs across various documents, chatbots,
and emerging assistant/agent use cases. LLM software platforms help
organizations streamline the
These platforms include capabilities that creation of training data sets,
help with indexing/vectorization, retrieval- benchmarking model performance,
augmented generation (RAG), and usage within normalizing prompt formats
a workflow pipeline. across models, creating complex
orchestration pipelines,
seamlessly transitioning to
Seamless future proofing to new foundation models, and
accelerate the incorporation of monitoring solution performance
new models and capabilities to helping with cost and access
controls.
GenAI model providers (OpenAI, Cohere, IBM,
etc.) continuously release new models with Matt Arcaro
DIRECTOR, COMPUTER VISION & AI
improved accuracy, efficiency, coherence, IDC
and applicability to additional use cases.
Organizations should be able to leverage these
new models without considerable refactoring,
retraining, or constant reconfiguration. LLM software platforms are outcome oriented
to help organizations build GenAI applications
LLM software platforms are designed to ensure that maximize their business requirements and
that customers have the flexibility to adapt to objectives.
new and emerging capabilities and models in
a lightweight, often zero-touch way.

© Composable 12
ANALYST RECOMMENDATIONS

How to Evaluate LLM Software Platforms


Organizations researching, experimenting with, or pursuing LLM software platforms need to
recognize that the nascency of the GenAI technology ecosystem and pace of innovation create a
platform ecosystem that spans a wide range of capabilities, business models, and product offerings.
In this way, not all platforms are created equal.

IDC has identified several strategic and technological attributes that organizations should consider
when pursuing an LLM software platform1. These attributes include:

• Broad ecosystem technology support, • Multi-persona (i.e., business analyst, prompt


including integrating multiple open source engineer, developer, product owner) tooling
and proprietary LLMs and providers to enable collaboration across user bases

• Highly tunable model orchestration • End-to-end governance, including support


strategies, including support for synthetic for auditability, monitoring, cost, and change
LLMs management

• Sandboxing capabilities that enable • Progressive technology development


the ability to experiment, validate, and strategy that includes a well-thought-out
productize LLM-enabled solutions and road map of features
applications quickly

1. “Moving Beyond the Model to Maximize the Effectiveness & Reach of GenAI Deployments,” written by: Matt Arcaro, Director of
Computer Vision and AI at IDC. Published July 2024.

© Composable 13
FROM IDEATION TO PRODUCTION

The 6 Steps to Evaluate, Select and


Implement LLM Use Cases
1. Identify Potential Use Cases 4. Prioritize the Best Use Cases
Brainstorm, list, and categorize all potential Plot use cases on a value vs effort graph.
LLM use cases relevant to your organization. Prioritize the “high-value, low-effort” use
Collaborate with different departments and cases (no brainers). Consider “low-effort,
teams to compile a list of requests. low-value” and “high-effort, high-value” use
cases for future implementations. Put a pin
2. Assess Value in the “high-effort, low-value” use cases for
Evaluate the potential efficiency gains if now, but revisit them after you’ve deployed
each task in the use case reaches 100% higher value use cases.
completion by LLMs. What about 80%?
60%? 40%? Consider strategic alignment 5. Select & Validate Use Cases
and competitive advantages. Assess the Select the top-priority use cases and
risks associated with each use case and validate the selection with key stakeholders
analyze the revenue vs cost implications. to refine as necessary.

3. Assess Effort 6. Leverage an LLM Software Platform for


Model the approach to deliver on the Implementation
objective. Score the suitability for an LLM: Is Maximize the effectiveness of your GenAI
it content compression or expansion? How development efforts to enable performance
complex is the reasoning? Evaluate resource improvements, solution flexibility, cost
availability and time to market. Evaluate if management, third-party data usage and
you have the infrastructure in place to follow integration, and life-cycle management
the suggested approach and if you have the capabilities.
people and skills readily available.

Organizations have realized that the potential value of GenAI is too big to ignore yet the challenges of
deploying LLM use cases have left many teams stuck in experimentation. Luckily, you have this guide.
By using the evaluation framework and scoring template provided, you can easily evaluate and select
LLM use cases. And with an LLM software platform like Composable, you can deploy LLM use cases
in days instead of weeks or months.

To learn more about Composable, visit becomposable.com and schedule a free, one-hour workshop
with one of our LLM experts where you’ll show you how easy it is to build and deploy your first LLM
use case.
© Composable 14
Appendix
The following section contains a blank use case scoring template you can print out and seven
additional use case examples that are well suited for LLMs.

Use Case Scoring Template 16

Documentation Generation & Maintenance 17

Occurrences Identification of a Problematic Clause 22

Large Document Review 23

Contract Liabilities Monitoring 24

Code Generation for Tooling 25

Earnings Call Transcript Analysis 26

Transmittal Document Analysis 27

© Composable 15
HOW TO EVALUATE LLM USE CASES

LLM Use Case Scoring Template


Use Case: ___________________________________________________________________
VALUE NO VALUE HIGH VALUE
�10 �9 �8 �7 �6 �5 �4 �3 �2 �1 0 1 2 3 4 5 6 7 8 9 10
EXPECTED
BENEFITS 0

LOW RISK MEDIUM RISK HIGH RISK


�1 �0�5 �1

RISK 0

LOW COST / HIGH REVENUE MEDIUM COST / MEDIUM REVENUE HIGH COST / LOW REVENUE
�1 �0�5 �1
COST VS
REVENUE
0

TOTAL VALUE SCORE

LOW
EFFORT EFFORT HIGH EFFORT
�10 �9 �8 �7 �6 �5 �4 �3 �2 �1 0 1 2 3 4 5 6 7 8 9 10
COMPLEXITY,
RESOURCES & 0
TIME

NO ADDITIONAL PEOPLE OR TRAINING NEEDED SOME ADDITIONAL PEOPLE OR TRAINING NEEDED MANY ADDITIONAL PEOPLE OR TRAINING NEEDED
�1 �0�5 1

PEOPLE & SKILLS 0

TOTAL EFFORT SCORE

10 0
NO BRAINER BIG BET
HIGH

2
VALUE

-2

-4

-6

-8
LOW

LOW-HANGING FRUIT FORGET IT


-10 -8 -6 -4 -2 0 2 4 6 8 10
LOW HIGH
EFFORT

© Composable 16
A REAL LIFE EXAMPLE

Documentation Generation & Maintenance


A leading software company had acquired several SaaS companies and found itself grappling
with disjointed and inconsistent technical documentation. Each acquired company had its own
documentation practices, leading to a fragmented ecosystem of product manuals, user guides, and
support materials. As a result, users struggled to find the correct information, leading to a spike in
support tickets. Meanwhile, the internal documentation team was overwhelmed, struggling to keep
pace with frequent product updates, which led to frustration and burnout.

Challenges & Pain


• Each acquired company had its own style, • The documentation team was burdened by
format, and approach to documentation, the sheer volume of content that needed to
resulting in a lack of uniformity across the be updated and maintained.
company’s documentation assets.
• The frequent release of product updates
• Users had difficulty navigating the further exacerbated the issue, as the team
documentation, often encountering struggled to keep up with the demand,
outdated or conflicting information. leading to frustration and burnout.

• The disjointed documentation led to a higher


volume of support tickets as users were
unable to find the answers they needed
on their own. This increased the workload
on the support team, leading to delays in
resolving customer issues.

The organization evaluated the use case using the Effort vs Value scoring criteria and determined that
while the value was fairly high, the amount of effort required to complete these tasks without an LLM
software platform was significant.

Use Case Scoring


Value: +7 Initial Effort: +7.5
The expected benefits were high (+8) but These were fairly complex tasks (+7) that
the risk was medium (-0.5) and the cost required some additional skills (+0.5) to build
implications were medium (-0.5). and maintain the processes.

© Composable 17
A PLATFORM APPROACH

Documentation Generation & Maintenance


The company needed a structured approach to manage and maintain technical documentation. An
LLM software platform was selected to unify and streamline the company’s documentation across
all acquired SaaS products to ensure that all documentation was consistent, up-to-date, and easily
accessible to users, while also reducing the workload on the documentation team.

Content Structuring and Storage Validation of Output


The platform structures and stores source The platform iteratively refines the proposed
data in a modular and hierarchical manner. documentation by continuously reviewing
Each module or component of the system can changes, integrating feedback, and ensuring
have its corresponding documentation source, alignment with the project’s documentation
linked directly to the relevant codebase or standards and style guides. The platform also
process description. An LLM then analyzes supports collaborative editing, allowing team
and categorizes content based on its semantic members to review, comment, and approve
meaning, allowing the platform to tag and changes before finalizing the documentation
organize documentation components for easy update.
retrieval and update.
The platform performs automatic checks
Linking with Changesets on the generated documentation, verifying
The platform integrates directly with Version consistency, completeness, and adherence to
Control Systems (VCSs) to monitor changesets. predefined style and format rules. For critical
When a changeset is detected (e.g., code documentation, the platform facilitates manual
changes, process updates), the platform reviews by subject matter experts. The platform
triggers an analysis to determine which parts of also incorporates feedback from manual
the documentation may be impacted. reviews into its learning model, enhancing
future automated updates.
Iterative Content Generation
Using one or more LLMs, the platform assesses
the extent and nature of changes, identifying
relevant sections of the documentation that
need updating or revision. Then, an LLM
generates a draft of the required documentation
updates based on the changeset analysis. This
draft might include new content, revisions, or
annotations suggesting areas for review.

© Composable 18
SOLUTION

Documentation Generation & Maintenance


The company selected an LLM software platform because it offered a scalable, efficient solution to
their documentation challenges. The key benefits that drove their decision included:

Scalability Efficiency
The platform can easily handle the growing By automating large portions of the
volume of documentation as the company documentation process, the platform
continues to expand its product offerings. significantly reduced the workload on the
documentation team, alleviating burnout and
Consistency allowing them to focus on more strategic tasks.
The LLM platform ensured that all
documentation was uniform, eliminating the Improved User Experience
inconsistencies that plagued the company’s With accurate, up-to-date documentation
previous efforts. readily available, users could find the
information they needed without resorting to
Improved Collaboration support tickets, leading to higher customer
By integrating with VCSs and offering satisfaction.
collaboration tools, the platform facilitates
better coordination among teams, ensuring
that documentation updates are reviewed and
approved efficiently.

Revised Effort Score with an LLM Software Platform


Effort: -4
By leveraging the LLM software platform with integrated tooling to support the use case, the
complexity, resource requirements, and time was dramatically reduced (-3) and no additional people
or skills were needed (-1).

© Composable 19
USE CASE SCORE

Documentation Generation & Maintenance


VALUE NO VALUE HIGH VALUE
�10 �9 �8 �7 �6 �5 �4 �3 �2 �1 0 1 2 3 4 5 6 7 8 9 10
EXPECTED
BENEFITS 8

LOW RISK MEDIUM RISK HIGH RISK


�1 �0�5 �1

RISK �0�5

LOW COST / HIGH REVENUE MEDIUM COST / MEDIUM REVENUE HIGH COST / LOW REVENUE
�1 �0�5 �1
COST VS
REVENUE
�0�5

TOTAL VALUE SCORE

LOW
EFFORT EFFORT HIGH EFFORT
�10 �9 �8 �7 �6 �5 �4 �3 �2 �1 0 1 2 3 4 5 6 7 8 9 10
COMPLEXITY,
RESOURCES & �3
TIME

NO ADDITIONAL PEOPLE OR TRAINING NEEDED SOME ADDITIONAL PEOPLE OR TRAINING NEEDED MANY ADDITIONAL PEOPLE OR TRAINING NEEDED
�1 �0�5 1

PEOPLE & SKILLS �1

TOTAL EFFORT SCORE


10
NO BRAINER BIG BET
HIGH

�4

2
VALUE

0
With an LLM software platform
Without an LLM software platform
-2

-4

-6

-8
LOW

LOW-HANGING FRUIT FORGET IT


-10 -8 -6 -4 -2 0 2 4 6 8 10
LOW HIGH
EFFORT
Figure 2
Value vs Effort graph for the documentation and generation use case showing the use case score with and without an
LLM software platform.
© Composable 20
RESULTS

Documentation Generation & Maintenance

• Reduction in Support Tickets • Consistent and Accurate Documentation


The company saw a significant decrease The platform ensured that all documentation
in the number of support tickets, as users was consistent across products, reducing
were able to find accurate information in the user confusion and enhancing the overall
updated documentation. This led to faster user experience.
resolution times for the remaining tickets
and improved overall customer satisfaction. • Increased Documentation Team
Productivity
• Scalability & Future-Proofing The documentation team’s workload was
The company’s documentation processes drastically reduced, allowing them to focus
were now scalable, allowing them to on high-value tasks such as strategic
easily incorporate new acquisitions and content planning and quality assurance.
product updates without overwhelming the Team morale improved, and burnout was
documentation team. reduced.

Conclusion
In summary, the LLM software platform transformed the company’s documentation process,
delivering substantial business value and reducing the effort required to maintain high-quality,
consistent, and up-to-date technical documentation.

© Composable 21
KNOWLEDGE QUERYING

Occurrences Identification of a Problematic Clause


Context
Lots of important documents, such as contracts Platform Approach
or laws, are divided into pieces, whether they Ingest contracts after having turned
are called clauses or chapters or articles. on chunking. Similarity is automatically
calculated for all clauses (of chunked
contracts).
Challenges & Pain
An organization may have identified a Select the problematic clause in a given
problematic clause: for example one that puts contract, then look at the “Similar” tab
the company at risk in customer contracts. to see similar clauses, and their parent
contract. Filter by region/segment/product,
A decision is made to identify all related legacy and get the acceptable ranges for each
contracts in order to properly anticipate based on actual history.
associated risk.

However, finding all occurrences of such a


Key Benefits
clause in legacy contracts may be difficult, since • Significant time and energy savings
the clause may have been slightly modified or • Improve risk management
even translated in several languages. • Potential automation of risk management
(i.e. automated detection of new
occurrence of a problematic clause)

© Composable 22
CONTENT ANALYSIS & REASONING

Large Document Review


Context
Organizations need to review documents and Platform Approach
check compliance with policies, standards Submit any kind of document to be
and regulatory frameworks, whether those reviewed (e.g. contract, project plan,
documents are internal or come from a supplier. product specification) as well as a specific
set of rules (e.g. policy, standard, guidelines)
Challenges & Pain
Many teams are overloaded with reviewing Ask for a review returning both structured
activities and most of them wouldn’t consider data, such as: “is compliant?”, “non-
re-validating the compliance of hundreds compliance issues”; and unstructured data,
or thousands of legacy contracts with the such as a “remediation plan” for each issue
company’s latest terms and conditions due to
the tedious nature of the work.
Key Benefits
• Easily process large volumes of
documents that would not be processed
at all otherwise
• Perform objective reviews
• Improve employee productivity by
pointing collaborators directly to the
parts of the document that need
attention
• Detect and reduce risks
• Optimize routing based on early analysis
(for example: dispatch to the right team
based on the severity/type of comment)

© Composable 23
CONTENT ANALYSIS & REASONING

Contract Liabilities Monitoring


CONTEXT
Organizations are bound by law and contracts PLATFORM APPROACH
they sign both internally and externally, with Contracts are chunked into clauses. Both
suppliers, partners and customers. clauses and the main contracts have
embeddings created and contextual
CHALLENGE & PAIN metadata generated.
Liabilities are not always formalized or captured
in a way that makes it easy for operations to The platform regularly monitors a set of
take them into account. Forgetting them may risks based on a guideline created by the
result in significant penalties. business. Using semantic RAG, the platform
automatically queries the specific contracts
using structured identifiers generated by the
LLM.

The system is able to identify relevant


clauses (e.g. Indemnification) using vector
search for the subset of contracts using the
correct policy rules, and verifies that the
indemnification range is within what has
been accepted by the guideline and decides
to flag the contract for review if issues are
detected.

KEY BENEFITS
• Better management of liabilities, with
less breaches and crises
• Significant reduction in risk
• Improved brand image

© Composable 24
GENERATING CONTENT

Code Generation for Tooling


CONTEXT
The IT department always needs to perform a PLATFORM APPROACH
lot of administrative actions on both digital and Automate PowerShell script generation to
physical assets, such as software, personal express the requirement in a functional way.
computers, servers, storage systems, network Get corresponding scripts generated and
elements, and more. select the most suitable LLM for the task.

CHALLENGE & PAIN KEY BENEFITS


Automating IT tasks is critical and increases the • Decreased IT staff workload
need for writing automation scripts, for instance • Increased IT productivity
in PowerShell. These may be complex, long, too • Better service to employees &
numerous or any combination of these factors. stakeholders with more personalized
Scripts must also be updated regularly, resulting service
in increased IT workloads.

© Composable 25
CONTENT ANALYSIS & REASONING + CONTENT GENERATION

Earnings Call Transcript Analysis


CONTEXT
Publicly traded companies must communicate PLATFORM APPROACH
their financial results on a regular basis, typically The platform automatically categorizes the
quarterly. This is an Earnings Call, which results document type (e.g. transcript, financial
in an Earnings Call Transcript. data sheet), extracts metadata (e.g.
company, fiscal year, quarter, sentiment),
CHALLENGE & PAIN and then generates a quarterly transcript
Financial analysts spend hundreds of hours summary with an extraction of the main
reading and summarizing these transcripts for topics and key financial metrics. An annual
sometimes thousands of companies. This is synthesis is then generated based on the
a massive investment of time which prevents quarterly summaries, and finally a year-
the analysts from focusing on more valuable over-year (YoY) analysis is generated, with
activities. identification of dynamics (trends YTD/
QoQ/YoY in revenues, operational margin,
customer churn, etc.).

KEY BENEFITS
• Instant processing, homogeneity of
produced results and more sophisticated
analysis
• Content analysis and subsequent
content generation are efficiently
articulated to constitute a full process
• Time savings can be spent on value
added tasks

© Composable 26
CONTENT ANALYSIS & REASONING + CONTENT GENERATION

Transmittal Documents Analysis


CONTEXT
Engineering companies in various industries PLATFORM APPROACH
typically manage projects that involve several The platform automatically categorizes
teams, including suppliers and partners. and classifies the documents, checks for
This implies receiving a well defined set of completeness, generates summaries, and
documents called “transmittals” for each highlights specific areas of risk.
project phase.
KEY BENEFITS
CHALLENGE & PAIN • Decrease the volume of tedious tasks
Transmittals may be huge and contain tens • Faster document processing and
if not hundreds of documents. Document reduced project delays
Controllers spend countless hours checking • Time savings can be spent on more
completeness and accuracy of submitted valuable tasks
documents, as well as properly classifying
them and managing subsequent comments &
revisions.

© Composable 27

You might also like