Azure on Strathweb. A free flowing tech monologue.

LLM and SLM collaboration using the Minions pattern (with Phi-4-mini and Azure OpenAI)

Fri, 24 Oct 2025 07:06:14 +0000

In this post, we’ll explore a novel approach to optimizing AI workflows by strategically combining large language models (LLMs) with small language models (SLMs) using the “Minions pattern.” This technique, described in the research paper “Minions: Cost-efficient Collaboration Between On-device and Cloud Language Models” by Narayan et al., addresses one of the most pressing challenges in AI application development - the cost of processing large amounts of data with expensive, cloud-based language models. If you’ve ever built an AI system that needs to analyze extensive documents or datasets, you’ve probably felt the frustration of watching your API costs skyrocket as you process more and more content.

AI Agents with OpenAPI Tools - Part 2: Azure AI Foundry

Fri, 27 Jun 2025 07:06:14 +0000

In the previous part of this series, we explored how to attach OpenAPI-based tools to a Semantic Kernel AI agent. In this part, we will look at another SDK for building AI Agents, Azure AI Foundry SDK, to create an agent that can also interact with OpenAPI-based tools.

Using o-series Reasoning Models in PromptFlow

Mon, 24 Mar 2025 07:06:14 +0000

If you have tried to use the OpenAI o-series reasoning models, such as o1 or o3, with PromptFlow recently, you certainly ran into a nasty surprise. While PromptFlow supports a wide range of models and providers, the o-series models are not among them. This is of course quite a shame, especially if you’d like to benchmark or evaluate your flows against those models.

In this short post, we will look at a workaround.

Simplifying the AI workflow: Access different types of model deployments with Azure AI Inference

Fri, 22 Nov 2024 07:06:14 +0000

In this post, we will explore the flexibility behind Azure AI Inference, a new library from Azure, which allows us to run inference against a wide range of AI model deployments - both in Azure and, as we will see in this notebook, in other places as well.

It is available for Python and for .NET - in this post, we will focus on the Python version.

How GPT-4o-mini can be simultaneously 20x cheaper and 2x more expensive than GPT-4o

Fri, 25 Oct 2024 07:06:14 +0000

GPT-4o-mini is the small, cost-effective version of the GPT-4o model. It is a great default choice for developers who want a very capable and fast model, but don’t need the full power of the GPT-4o model. However, there are some important things to keep in mind when using GPT-4o-mini, especially when it comes to pricing - some of which is rather contradictory!

Speech-based retrieval augmented generation (RAG) with GPT-4o Realtime API

Mon, 14 Oct 2024 07:06:14 +0000

On October 1st, OpenAI and Microsoft (Azure OpenAI) announced the availability of the GPT-4o Realtime API for speech and audio. It is a new, innovative way of interacting with the GPT-4o model family, the provides a “speech in, speech out” conversational interface. Contrary to traditional text-based APIs, the Realtime API allows sending the audio input directly to the model, and receiving the audio output back. This is a significant improvement over the existing solutions to voice-enabled assistants, which required converting the audio to text first, and then converting the text back to audio. The Realtime API is currently in preview, and the SDKs for various languages have mixed-level of support for them, but it is already possible to build exciting new applications with it.

The low-latency speech-based interface also poses some challenges to established AI architectural patterns, such as Retrieval-Augmented Generation (RAG) - and today we will tackle just that, and have a look at a small sample realtime-voice RAG app in .NET.

Tool Calling with Azure OpenAI - Part 2: Using the tools directly via the SDK

Fri, 19 Apr 2024 07:06:14 +0000

Last time around, we discussed how Large Language Models can select the appropriate tool and its required parameters out of freely flowing conversation text. We also introduced the formal concept of those tools, which are structurally described using an OpenAPI schema.

In this part 2 of the series, we are going to build two different .NET command line assistant applications, both taking advantage of the tool calling integration. We will orchestrate everything by hand - that is, we will only use the Azure OpenAI Service API directly (or rather using the .NET SDK for Azure OpenAI) - without any additional AI frameworks.

Tool Calling with Azure OpenAI - Part 1: The Basics

Thu, 04 Apr 2024 07:06:14 +0000

One of the fantastic capabilities of the Large Language Models is their ability to choose (based on a predefined set of tool definitions) the appropriate tool and its required parameters out of freely flowing conversation text. With that, they can act as facilitators of workflow orchestration, where they would instruct applications to invoke specific tools, with specific set of arguments.

OpenAI announced the built-in capability called function calling in the summer of last year, and by now it is an integral part of working with and building applications on top of the GPT models. The functionality was later renamed in the API to “tools”, to better express their broad scope and nature.

Today I am starting a new multi-post Azure OpenAI blog series focusing specifically on the tool capabilities. We will build a client application with .NET, and explore tool integration from different angles - using the Azure OpenAI .NET SDK directly, using the Assistants SDK and finally leveraging various orchestration frameworks such as Semantic Kernel and AutoGen. In today’s part one, we are going to introduce the basic concepts behind tool calling.

Combining Azure OpenAI with Azure AI Speech

Fri, 08 Mar 2024 07:06:14 +0000

In my recent posts, I’ve been exploring various facets of the Azure OpenAI Service, discussing how it can power up our applications with AI. Today, I’m taking a slightly different angle - I want to dive into how we can enhance our projects further by integrating Azure OpenAI Service with Azure AI Speech. Let’s explore what this integration means and how it could lead to exciting, AI-powered applications.

Using your own data with GPT models in Azure OpenAI - Part 4: Adding vector search

Fri, 23 Feb 2024 07:06:14 +0000

For our Retrieval-Augmented-Generation (RAG) application, we setup AI Search in part 1, however so far we only used it using the basic keyword search.

In this part 4 of the series about bringing your own data to Azure OpenAI Service, we will go ahead and integrate vector search, as a more sophisticated way of performing the search across the Azure AI Search index within our RAG-pattern system.

I already covered vectorization and embeddings using the OpenAI embedding model on this blog, and we will be relying on the same principles here. I recommend reading through that article before continuing if you are not yet familiar with the concept of embeddings.

Running Swift Package tests in Azure DevOps

Wed, 31 Jan 2024 07:06:14 +0000

It is a common scenario to use Azure DevOps to build, sign and release iOS applications. Most of the tasks related to that can be handled by the Xcode@5 task, which provides support for all kinds of build activities around Xcode workspaces, and which is a de-facto shortcut for invoking xcodebuild command line tool.

The task is quite well documented, but it is not entirely obvious how to use it for building Swift Packages, as those come without Xcode workspaces. In this post we will quickly explore how to achieve that.

Using your own data with GPT models in Azure OpenAI - Part 3: Calling Azure OpenAI Service via .NET SDK

Mon, 18 Dec 2023 07:06:14 +0000

In the last post of this series we set up a demo .NET client application that was able to call and utilize a GPT model hosted in Azure OpenAI Service, which in turn was integrated with our own custom data via Azure AI Search. We did this using the bare bones REST API - and in part three, it’s time to shift gears and explore how to accomplish similar task using the .NET SDK, which offers a more streamlined and less ceremonious approach over calling the HTTP endpoints directly.

Using your own data with GPT models in Azure OpenAI - Part 2: Calling Azure OpenAI Service via REST API

Fri, 24 Nov 2023 07:06:14 +0000

In the previous part of this series, we have successfully set up Azure AI Search, to have it ready for integration with Azure OpenAI Service. The ultimate goal is to take advantage of the retrieval-augmented-generation pattern, and enhancing our interactions with the GPT model with our own custom data.

Let’s continue building this today.

Using your own data with GPT models in Azure OpenAI - Part 1: Setting up Azure AI Search

Fri, 10 Nov 2023 07:06:14 +0000

There is no question that the emergence of generative AI is going to significantly alter various aspects of our daily lives. At the same time, most of the large language models (LLMs) are designed as general-purpose black boxes and their utility is initially confined to the data they were trained on. However, it is possible to extend their functionality and reasoning to any custom data set, be it private or public, even without the massive effort that would be needed to retrain or even fine-tune them.

We are going to start exploring that concept today with a multi-part post series on “bringing your own data” to Azure OpenAI. In part one today, we will set up the necessary Azure resources and prepare the stage for a client application integration, which will follow in parts two and further.

Using embeddings model with Azure OpenAI Service

Wed, 13 Sep 2023 07:00:14 +0000

I recently blogged about building GPT-powered applications with Azure OpenAI Service. In that post, we looked at using the text-davinci-003 model to provide classification capabilities for natural text - more specifically, we categorized and rated scientific papers based on the interest area (note that the recommended model for this task now is gpt-35-turbo now).

In today’s post we are going to continue exploring Azure OpenAI Service, this time looking at the embeddings model, text-embedding-ada-002.

Building GPT powered applications with Azure OpenAI Service

Wed, 26 Apr 2023 10:06:14 +0000

In this post we will have a look at how we can utilize Azure OpenAI Service to build applications using various OpenAI models. At the high level, Azure OpenAI allows accessing GPT-4, GPT-3, Codex and Embeddings models using the security boundary of Azure, and while ensuring data privacy and residency and conforming to other common enterprise requirements such as private networking. In other words, it addresses one of the biggest worries of integrating AI services into own applications - the data is never shared with OpenAI.

Azure Blob Storage IFileProvider for ASP.NET Core

Thu, 16 Aug 2018 09:08:26 +0000

As part of my recent talks on ASP.NET core, I have been showing how to build a custom IFileProvider for ASP.NET Core. The example that I was using was Azure Blob Storage - and exposing files from there as if they were local files that are part of your application.

I have pushed that code to Github and decided to package it as Nuget package, which, hopefully, someone will find useful.

Running the OWIN pipeline in the new .NET Azure Mobile Services

Fri, 21 Feb 2014 09:05:59 +0000

Yesterday, a preview of the .NET Azure Mobile Services has been released. Despite the fact that I’d rather see a scripted C# support 🙂 - I am still very excited about this new .NET support, as ZUMO is one of my favorite Azure offerings.

The whole thing is in preview right now and runs on Web API (version 5.1 at the moment, so not the latest) but the team has made several very smart decisions, which I am sure the community will welcome with open arms. One of them is the ability to plug in your OWIN pipeline!

Adding high performance Windows Azure Cache Service to your ASP.NET Web API

Fri, 13 Sep 2013 10:06:59 +0000

Microsoft has recently announced the preview release of Windows Azure Cache Service - intended to allow you to easily deploy high performance, dedicated, distributed cache for your applications.

You can read more about the feature (and it does seem really awesome at first glance), in the thorough announcement post by Scott Guthrie.

Let’s look at how you can leverage this powerful service from ASP.NET Web API.

scriptcs and using Azure Mobile Services from your scripts

Mon, 13 May 2013 03:33:40 +0000

Hopefully by now you have already heard about the sriptcs project, which allows you to write script based applications with C# and Nuget.

If not, have a look at the readme and the great introduction post by Scott Hanselman to get started.

Actually, just today, we have released v.0.4 of scriptcs! To celebrate that, let’s revisit one of our favorite topics - Azure Mobile Services - and how you can use it with scriptcs.

Building real time applications for any client with Azure Mobile Services & Pusher

Tue, 29 Jan 2013 03:08:03 +0000

Azure Mobile Services is definitely one of the coolest technologies in the Azure family. One of the issues however, has been that it only has client libraries for Windows 8, iOS and Android, making it a bit more difficult for developers targeting other platforms (also web browsers!) to take advantage of its capabilities.

A while ago, I blogged about how to work around that - by using ZUMO REST API. However, you still didn’t have access to one of the nicest features of ZUMO for mobile devices - and that is push notifications. That changed recently, when ZUMO announced a partnership with Pusher, to provide push notifications for virtually any client.

Additionally, the team recently released another cool feature - scheduled tasks (cron jobs). Let’s see how you can use it all together to quickly build a realtime service.

More after the jump.

Using existing database with Azure Mobile Services

Wed, 12 Dec 2012 03:40:43 +0000

One of the topics we like to come back to on this blog is Azure Mobile Services (ZUMO) - and rightfully so, because that’s a terrific service, capable of smoothly fueling your application’s backend in a hassle-free and scalable manner.

One thing you might have noticed about ZUMO though, is that pretty much all the online tutorials and materials related to it will show you how to work with it from scratch. One of the unknown facts about it, is that, with some slight modifications, you can actually plug in your existing SQL Server (or rather SQL Azure) database (provided you have earlier migrated it to Windows Azure of course), and serve it for your application utilizing the power of Azure Mobile Services.

Let’s a have a look at how you’d configure that.

Send text messages (SMS) from Web API using Azure Mobile Services

Wed, 17 Oct 2012 01:52:17 +0000

Few weeks ago, I blogged about using Azure Mobile Services in your Web API applications and I think I managed to convince some of you that ZUMO with its REST-style API is really useful and super easy to work with.

The guys at Azure Mobile Services are not slowing down at all. Today Scott Gu announced a whole new super cool set of upgrades, and one of the coolest features is the integration to Twillio and the ability to send text messages (SMS) directly from the ZUMO script.

Let’s go ahead and extend our previous ZUMO example (managing a list of sports team) with SMS messaging.

Using Azure Mobile Services in your web apps through ASP.NET Web API

Mon, 03 Sep 2012 10:26:52 +0000

Azure Mobile Services is the latest cool thing in town, and if you haven’t checked it out already I really recommend you do, i.e. in this nice introduction post by Scott Gu. In short, it allows you to save/retrieve data in and out of dynamic tables (think no-schema) directly from the cloud. This makes it a perfect data storage solution for mobile apps, but why not use it in other scenarios as well?

For now Azure Mobile Services (a.k.a. ZUMO) is being advertised for Windows 8 only (the SDK is targeted for Windows 8 applications), but there is no reason why you couldn’t use it elsewhere.

Let’s do that and use Web API as a proxy.

Deploy your ASP.NET Web API application to Windows Azure in 3 minutes

Sun, 08 Jul 2012 11:43:37 +0000

Over the past months I have been blogging about ASP.NET Web API a lot. One question that I haven’t really addressed, is how to send this beast to production - because it’s one thing to develop something, and completely different to have it up and running in live environment.

While so many members of the community have really enjoyed Windows Azure since it has been unveiled in the new shape recently, a lot of people are still uncertain how to work with it - because, well, “cloud” has always sounded a little enterprise-like. I thought it might help people to have a quick step-by-step guide on how you could really easily deploy your app to Azure (using Git!).

More after the jump.