Building AI Applications with Microsoft Semantic Kernel: Easily integrate generative AI capabilities and copilot experiences into your applications

Building AI Applications with Microsoft Semantic Kernel

Copyright © 2024 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Niranjan Naikwadi

Publishing Product Manager: Tejashwini R

Book Project Manager: Neil D’mello

Senior Editor: Mark D’Souza

Technical Editor: K Bimala Singha

Copy Editor: Safis Editing

Proofreader: Mark D’Souza

Indexer: Pratik Shirodkar

Production Designer: Shankar Kalbhor

DevRel Marketing Coordinator: Vinishka Kalra

First published: June 2024

Production reference: 1060624

Published by

Packt Publishing Ltd.

Grosvenor House

11 St Paul’s Square

Birmingham

B3 1RB, UK

ISBN 978-1-83546-370-3

www.packtpub.com

Contributors

About the author

Lucas A. Meyer is a financial economist and computer scientist with an MBA and an M.Sc. in finance from the University of Washington in Seattle. Lucas works as a principal research scientist at the Microsoft AI for Good Lab, where he works on the economics of AI and using large language models to combat disinformation and improve economic outcomes. Prior to that, Lucas worked for several years in finance, pioneering several uses of natural language processing for financial analysis and automation. Lucas is passionate about AI education and is a Top Voice on LinkedIn, where he posts about new developments in the field.

To my wife, Ilanah, and my children, Thomas, Eric, and Caroline: you make it easy and worthwhile for me to face any challenge. To my mom, Ana Maria, thank you for teaching me that education is the best investment. To my colleagues at the AI for Good Lab: I learn so much from you. To the online communities of LinkedIn and Threads, thanks for all you taught me and all the support.

About the reviewer

Lucas Puskaric has driven billions of dollars in revenue as a staff software engineer and tech lead, specializing in crafting creative customer-focused solutions across the entire stack. Over his career, he has worked at every type of company from non-profit to Fortune 25. After helping vaccinate tens of millions of people, he founded the software company Interweave. When LLMs arrived on the scene, he was an early adopter who released several AI apps. He’s known for his presence on Threads, where he often shares technical knowledge and projects he’s working on. Connect with him at @lucaspuskaric on all socials.

Shoutout to the love of my life, Alex, and our four cats: Reggie, Sonny, Nori, and Figgy. Also, thanks to Threads for introducing me to Lucas Meyer and Packt for giving me the opportunity to review this masterpiece.

Table of Contents

Preface

Part 1: Introduction to Generative AI and Microsoft Semantic Kernel

1

Introducing Microsoft Semantic Kernel

Technical requirements

Obtaining an OpenAI API key

Obtaining an Azure OpenAI API key

Generative AI and how to use it

Text generation models

Understanding the difference between applications and models

Generating text using consumer applications

Generating images

Microsoft Semantic Kernel

Installing the Microsoft Semantic Kernel package

Using Semantic Kernel to connect to AI services

Connecting to OpenAI Services using Python

Connecting to OpenAI services using C#

Running a simple prompt

Running a simple prompt in Python

Running a simple prompt in C#

Using generative AI to solve simple problems

Creating semantic functions

Creating native functions

Plugins

The config.json file for the knock-knock joke function

The skprompt.txt file for the knock-knock joke function

The config.json file for the semantic function that explains jokes

The skprompt.txt file for the explain joke function

Loading the plugin from a directory into the kernel

Using a planner to run a multistep task

Calling the Function Calling Stepwise planner with Python

Summary

References

2

Creating Better Prompts

Technical requirements

A simple plugin template

The skprompt.txt file

The config.json file

Calling the plugin from Python

Calling the plugin from C#

Results

Improving the prompt to get better results

Revising the skprompt.txt file

The result

Prompts with multiple variables

Requesting a complex itinerary with Python

Requesting a complex itinerary with C#

The result of the complex itinerary

Issues when answering math problems

Multistage prompts

CoT – Let’s think step by step

Implementing CoT with Python

Implementing CoT with C#

Results for CoT

An ensemble of answers

Summary

References

Part 2: Creating AI Applications with Semantic Kernel

3

Extending Semantic Kernel

Technical requirements

Getting to know the core plugins

An example – Using the TimePlugin

Introducing the application – Validating grants

Directory structure of our application

Developing native plugins

The directory structure of our plugins

Checking the structure of our Excel spreadsheet

Additional checks

Evaluating the Word document

Developing semantic plugins

Evaluating the grant proposal with a semantic plugin

Summary

4

Performing Complex Actions by Chaining Functions

Technical requirements

Creating a native plugin that generates images

Writing a DALL-E 3 wrapper in Python

Writing a DALL-E 3 wrapper in C#

Using multiple steps to solve a problem

Generating an image from a clue

Chaining semantic and native functions with C#

Chaining semantic and native functions with Python

Dealing with larger, more complex chains

Preparing our directory structure

Understanding the flow of our process

Creating the native function to process a folder

Modifying the Excel native plugin

Modifying the Word native plugin

Modifying the semantic functions

Creating and calling the pipeline

Summary

References

5

Programming with Planners

Technical requirements

What is a planner?

When to use a planner

Instantiating a planner

Creating and running a plan

An example of how a planner can help

How do planners work?

Controlling home automation with the planner

Creating the native functions

Adding a semantic function to suggest movies

Invoking the planner

Summary

6

Adding Memories to Your AI Application

Technical requirements

Defining memory and embeddings

How does semantic memory work?

Embeddings in action

Using memory within chats and LLMs

Using memory with Microsoft Semantic Kernel

Using memory in chats

Reducing history size with summarization

Summary

Part 3: Real-World Use Cases

7

Real-World Use Case – Retrieval-Augmented Generation

Technical requirements

Why would you need to customize GPT models?

Retrieval-augmented generation

Creating an index

Uploading documents to the index

Using the index to find academic articles

Using RAG to create a summary of several articles on a topic

Summary

References

8

Real-World Use Case – Making Your Application Available on ChatGPT

Technical requirements

Custom GPT agents

Creating a custom GPT

Creating a custom GPT that supports actions

Creating a web API wrapper for the native function

Deploying your application to an Azure Web App

Connecting the custom GPT with your custom GPT action

Summary

Index

Other Books You May Enjoy

Preface

Artificial intelligence is experiencing unprecedented growth, with new models emerging daily. With over 20 years in the technology sector, I can attest that the pace of innovation has never been this fast. This brings not only opportunities but also considerable change. Navigating these changes can be challenging and costly, as you may invest a lot of time learning a new technology that might become obsolete.

Enter Microsoft Semantic Kernel – a framework that reduces these risks by enabling access to various AI services through popular programming languages. This framework spares you the details of grappling with constantly evolving APIs. By learning Microsoft Semantic Kernel, you can write code at the framework level, and the framework will call the underlying models for you. This allows you to focus on core concepts instead of the details of each model.

One of the key benefits of Semantic Kernel is its ability to use different AI services. For example, code initially targeting the OpenAI GPT platform can be switched to Google Gemini, often without any modifications. This flexibility makes it easier to integrate AI into applications and to make minimal modifications to them when change inevitably happens.

Moreover, Semantic Kernel makes AI accessible to enterprise programming languages. While Python has long dominated the AI landscape, many enterprise applications rely on C# or Java. Recognizing this, Semantic Kernel not only supports Python but also elevates C# to a first-class AI language. Java support is currently in its beta stages and is expected to launch fully in 2024.

Whether you’re a solo developer or part of a larger enterprise, the demand to add AI functionality to applications is inevitable. This book was created to equip you with the necessary skills to implement AI quickly and effectively, ensuring you are well-prepared to meet this growing demand.

Who this book is for

The three main personas who are the target audience of this book are as follows:

Developers who want to add AI to their applications without adding the complexity of connecting to several different services and maintaining the connections after each release

Technical program managers who can write some code and want to quickly prototype AI functionality without having to learn details

Applied researchers and data scientists who want to focus their time on solving business problems quickly, instead of working on the ever-changing ways of connecting and using AI services

What this book covers

Chapter 1

, Introducing Microsoft Semantic Kernel, introduces several AI concepts and gives a tour of what Semantic Kernel can help you achieve, showing you how to connect to an AI service and use it to achieve a goal.

Chapter 2

, Creating Better Prompts, teaches you several techniques on how to interact better with AI, improving the chances that you will get a good result on your first try, using a concept called prompt engineering.

Chapter 3

, Extending Semantic Kernel, teaches you how to add functionality to Semantic Kernel, by adding native functions and semantic functions that can later be reused by you, as a developer, or your user to achieve their goals.

Chapter 4

, Performing Complex Actions by Chaining Functions, shows you how to use several functions of a kernel in sequence, making programming complex actions a lot easier.

Chapter 5

, Programming with Planners, explores how Semantic Kernel can receive a request in natural language and automatically decide which functions to call to achieve an objective, allowing users of your application to perform functions that you did not have to write code for.

Chapter 6

, Adding Memories to Your AI Application, examines how to add external knowledge to the AI models used by Semantic Kernel, making it easier for AI models to remember recent conversations and personalizations.

Chapter 7

, Real-World Use Case – Retrieval-Augmented Generation, shows how to add a large amount of data to AI models, allowing them to efficiently use information that they have not been trained on, including very recent and private data.

Chapter 8

, Real-World Use Case – Making Your Application Available on ChatGPT, shows how to publish the application you wrote with Microsoft Semantic Kernel on OpenAI’s GPT store, making it instantly available to millions of users.

To get the most out of this book

Readers will have to be familiar with programming in either Python or C#, and familiar with creating and connecting to AI services on the cloud.

For Chapter 7

, you will need to create an index in Azure AI search. There is a free tier. For Chapter 8

, to create a GPT to share with others, you will need a ChatGPT subscription.

If you are using the digital version of this book, we advise you to type the code yourself or access the code from the book’s GitHub repository (a link is available in the next section). Doing so will help you avoid any potential errors related to the copying and pasting of code.

Download the example code files

You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/Building-AI-Applications-with-Microsoft-Semantic-Kernel

. If there’s an update to the code, it will be updated in the GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/

. Check them out!

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: Since our prompt will be a new function and have multiple parameters, we will also need to create a new config.json file.

A block of code is set as follows:

    response = await kernel.invoke(pe_plugin[chain_of_thought], KernelArguments(problem = problem, input = solve_steps))

    print(f\n\nFinal answer: {str(response)}\n\n)

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

[default]

exten => s,1,Dial(Zap/1|30)

exten => s,2,Voicemail(u100)

exten => s,102,Voicemail(b100)

exten => i,1,Voicemail(s0)

Any command-line input or output is written as follows:

dotnet add package Microsoft.SemanticKernel.

s.Handlebars --version 1.0.1-preview

Bold: Indicates a new term, an important word, or words that you see onscreen. For instance, words in menus or dialog boxes appear in bold. Here is an example: S Once these configurations are done, click Review + create and your web application will be deployed in a few minutes.

Tips or important notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, email us at [email protected]

and mention the book title in the subject of your message.

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata

and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected]

with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com

.

Share Your Thoughts

Once you’ve read Building AI Applications with Microsoft Semantic Kernel, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page

for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books everywhere?

Is your eBook purchase not compatible with the device of your choice?

Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.

Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical books directly into your application.

The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content in your inbox daily

Follow these simple steps to get the benefits:

Scan the QR code or visit the link below

https://packt.link/free-ebook/9781835463703

Submit your proof of purchase

That’s it! We’ll send your free PDF and other benefits to your email directly

Part 1:Introduction to Generative AI and Microsoft Semantic Kernel

In this part, you will get an overview of generative AI and how to use it with Microsoft Semantic Kernel. In addition to this, you will also learn best practices for prompting that can be useful not only when you’re using Semantic Kernel, but also when you’re interacting with AI on your own.

This part includes the following chapters:

Chapter 1

, Introducing Microsoft Semantic Kernel

Chapter 2

, Creating Better Prompts

1

Introducing Microsoft Semantic Kernel

The generative artificial intelligence (GenAI) space is evolving quickly, with dozens of new products and services being launched weekly; it is becoming hard for developers to keep up with the ever-changing features and application programming interfaces (APIs) for each of the services. In this book, you will learn about Microsoft Semantic Kernel, an API that will make it a lot easier for you to use GenAI as a developer, making your code shorter, simpler, and more maintainable. Microsoft Semantic Kernel will allow you, as a developer, to use a single interface to connect with several different GenAI providers. Microsoft used Semantic Kernel to develop its copilots, such as Microsoft 365 Copilot.

Billions of people already use GenAI as consumers, and you are probably one of them. We will start this chapter by showing some examples of what you can do with GenAI as a consumer. Then, you will learn how you can start using GenAI as a developer to add AI services to your own applications.

In this chapter, you will learn the differences between using GenAI as a user and as a developer and how to create and run a simple end-to-end request with Microsoft Semantic Kernel. This will help you see how powerful and simple Semantic Kernel is and will serve as a framework for all further chapters. It will enable you to begin integrating AI into your own apps right away.

In this chapter, we’ll be covering the following topics:

Understanding the basic use of a generative AI application like ChatGPT

Installing Microsoft Semantic Kernel

Configuring Semantic Kernel to interact with AI services

Running a simple task using Semantic Kernel

Technical requirements

To complete this chapter, you will