Magically integrating LLMs into your .NET applications with Semantic Kernel

Contributing expert: Povilas Girininkas,

.NET Department Manager at Coherent Solutions

Semantic Kernel is one of the most enjoyable ways to integrate Large Language Models (LLMs) into a .NET application. Starting from scratch, without any prior LLM experience, you can get something cool working in a day, such as an LLM-powered to-do list application.

The magic of Semantic Kernel feels remarkably similar to when I first touched LINQ and unlocked the ability to quickly and completely crush a database without writing a line of SQL code. And now, regardless of how developers feel about LLM integrations in applications, the barrier to entry is lower than it has ever been.

In this article, we’ll walk through an example that demonstrates how to integrate LLMs into C# code using Semantic Kernel. Recently, I ran a 45-minute demo — and I’m providing the example here based on that demo — showcasing Semantic Kernel. In it, I set up the LLM conversation, configured local memory for messages, integrated local C# code, and switched between models and providers.

A note on Semantic Kernel

Last year, Microsoft folded AutoML and Semantic Kernel into the shiny new Microsoft Agent Framework, an update that many in the industry may have missed.

My reflection on it today isn’t exactly [Obsolete(“Deprecated”)], however. The general approach, ideas, process, and concepts in our Semantic Kernel example are still applicable. Most of the naming conventions are the same. The shift to Microsoft Agent Framework is more of a reshuffle, refactor, and iterative improvement by Microsoft’s Semantic Kernel team, rather than a full rewrite. If you treat it like an upgrade, as Microsoft frames it, you should be good.

For reference, you can review these official docs, straight from the horse’s mouth: the Microsoft Agent Framework overview and migration guide.

What is Semantic Kernel?

Semantic Kernel is a software development kit (SDK) for LLM integration and orchestration. It supports C#, Java, and Python languages, but for our example, we’ll focus on C#. The open-source SDK was developed by Microsoft and licensed under MIT, which means it’s free for commercial use.

With Semantic Kernel, you can build your own LLM-enabled code, integrating multiple local and remote LLM providers and models, including multi-modal models. The integration allows you to use features such as text generation, embedding generation, vector stores, and Retrieval Augmented Generation (RAG). Semantic Kernel supports standard .NET logging and OpenTelemetry out of the box. It can also call external tools via OpenAPI, MCP, Azure Logic Apps, and, best of all, our lovely native C# code.

You can review the surprisingly decent documentation.

A case to explore Semantic Kernel in practice

Let’s imagine we’re working with a very feature-rich and amazing application – a to-do list manager. We’ve completed an imaginary launch, and it’s receiving rave reviews.

Here’s a conceptual architecture (we’re using the word “architecture” very loosely):

Our application has a console-based interface that accepts user input and uses it to call the three methods. For example, when a user enters “2” for “add a to-do list item,” we ask them to enter a value, then call the add method on the service with whatever they enter. If you’ve completed university coding projects, this pattern should be painfully familiar.

Why add an LLM to our application architecture?

Some stakeholders (and AI-motivated investors) in our imaginary company suggested it would be cool to have a simple, user-friendly interface backed by an LLM. With it, a user could write, “I want to watch all of the Matrix movies,” and the to-do list manager would generate a list of the films.

Now we’ll add the LLM to our conceptual architecture:

Easy, now let’s do it in practice.

The existing code

Our application has three files: a representation of a to-do item, a service that works with it, and the main UI file. For brevity, we won’t go through the original UI, since we’ll fully replace it.

As you can see, neither class is special in any way. Even better, we don’t need to modify them to integrate our LLM-based UI.

The new implementation

For simplicity’s sake, we’ll add all of the code below to our Program.cs sequentially.

Step 1: Creating the Kernel

The first thing we need to do is to create a Kernel. If we were using ASP.NET, we could also do this through dependency injection.

Key points for this step:

• Adding logging to our application uses the standard code we know and love, and it’s a one-liner.

• Our example uses Azure Foundry to both host and run the model.

• Switching between models and providers is incredibly easy and requires very little code. For example, if we wanted to add an Ollama model, we’d adjust the code accordingly:

That’s it; no other changes are needed. Ollama requires a few clicks to install, then it hosts and runs the model locally, exposing it through HTTP. With it, we can run a decently working model like Llama 3.2, which only requires a few GBs of storage and RAM. Besides Azure Foundry and Ollama, there are abstractions for other local and cloud LLM providers and models, including ONNX, OpenAI, Mistral, Google, and Amazon.

• While switching models is easy, selecting a model may be more difficult. The selected model will have a major impact on speed, cost, and the quality of the end result. Small local models are good enough to test the flow if you use explicit inputs. Some small local models might even be robust enough for your use case, such as embedding generation.

• They also offer the benefit of essentially free hosting. If you want to handle real user inputs, however, a more powerful model will work much better. Even for our imaginary case, Llama 3.2 would struggle to use the three methods correctly and hallucinate way too often to be usable in production. While some of those issues could be solved with better method descriptions for the LLM, eventually, we would need to throw money at the problem.

Step 2: Adding message history 

If we wanted a single-shot prompt, we could skip this step and pass the single message to the chat client. But in our case, we might want to refer to a past message in the conversation, so we need to introduce a new concept — message history.

Key points for this step:

• Our history is just a simple list of ChatMessage objects. This list will be sent on every HTTP request to the LLM and will keep growing. Some providers let you store the history on their end. Even if provider storage is an option, at some point, this will get unwieldy, and we’d still need to manage it ourselves. The simplest option would be to start a new conversation periodically. However, if we’re planning for users to run very long conversations, then periodically summarizing the history in a single message and, if needed, giving the LLM the ability to search (using a tool similar to what we’ll cover in step 3) might be the best option.

• There are a few types of ChatMessages (System, Tool, Assistant, User) that represent different actors in the chat. The system prompt should take priority and is arguably the most important one to get right. Doing that is a huge rabbit hole and not in this post’s scope.

Step 3: Integrating our C# code

Our next step is to somehow let the LLM know about our C# code. To accomplish this, we’ll use Tools. We’ll wrap that into ChatOptions and use auto ToolMode, which will allow the LLM to decide if it should use a particular tool. Adding AllowMultipleToolCalls allows the LLM to call methods in succession for the same request.

Key points for this step:

• A tool is essentially any C# method, but you can also use MCP servers, OpenAPI calls, and even Azure Logic Apps.

• Besides the method, we should also provide a description, which is very important for stable, high-quality LLM results. You should aim to cover what the method does, as well as when and how to call it properly.

Step 4: Putting it all together

Finally, let’s use what we’ve built in the previous steps to create a working UI. We’ll use the IChatClient and feed it with everything we built. The core of the code below is to get user input → add the input to the history → pass the history and chat options to our chat client → add the output to the history and print it out.

Key points for this step:

• All the complexity here comes from the choice to use GetStreamingResponseAsync, which streams the output and lets us write it out one word at a time. This is very nice for testing and is visually appealing at demos. For real-world cases, as Yoda would say, “There is another.” Consider using GetResponseAsync, which returns a string that simplifies everything.

• Concerning user input and output validation, our example is an incredibly naive implementation that assumes the user is never malicious. If you’re going to expose this to unknown external users, you’ll need to add significant validation layers on both ends.

Step 5: Reviewing the result

Finally, let us look at how our implementation appears to the end user:

Key points for this step:

• This implementation is the best of both worlds.

• For backing implementation and storage, we get the determinism and performance of the existing C# application we know and love.

• For the UI, we have the ability to communicate in natural language, speak abstractly, and rely on LLM context. We can ask for all the movies of a series and have them added to our to-do list in a single sentence. We can refer to the original trilogy. We can make small grammatical mistakes. The LLM can call multiple methods sequentially to achieve the desired result.

The good (and bad) of Semantic Kernel

Look at what we managed to accomplish with only around 100 lines of code in our imaginary program. Pretty magical, right?

However, with all the good, there are still a few challenges to consider before using Semantic Kernel:

• The overlap and lack of feature parity across Microsoft.Extensions.AI, Semantic Kernel, and now, Microsoft Agent Framework, can be very confusing. It can be difficult to figure out which library you’re supposed to use, as you can create implementations using abstractions from any of them.

• You can easily get lost in the documentation. It’s quite good, but it doesn’t solve the underlying duplication problem. Microsoft recommends using Extensions.AI as the primary library for abstractions, but not all features are supported there. Hopefully, they will figure it out.

• There are rapid changes in the underlying technology and abstractions, which can be expected given what we’re integrating.

Even with its downsides, it doesn’t take away from the magic of the SDK. For an experienced developer, that’s still impressively fast.

Whether you’re somewhat skeptical about LLM integrations, like me, or fully on board, Semantic Kernel is a perfect vehicle for exploration. It’s low stakes — an easy integration tool that allows you to test some of the latest LLM features in your applications.