Mlx on Strathweb. A free flowing tech monologue.

Chain of Agents: Collaboration between local and remote language models with Agent Framework

Thu, 21 May 2026 07:06:14 +0000

This post continues a series on hybrid architectures that combine local Small Language Models (SLMs) with cloud Large Language Models (LLMs). We have already looked at the Minions pattern, which offloads bulk text extraction to a local model to reduce cloud API costs, and the SLM-default, LLM-fallback pattern, where a local model handles the majority of queries and only escalates to the cloud when confidence is low. Today we look at a third pattern: Chain of Agents (CoA), introduced in the paper “Chain of Agents: Large Language Models Collaborating on Long-Context Tasks” by Wang et al. (2024). Where the previous patterns were primarily motivated by cost, CoA is motivated by a different challenge - what happens when your document is simply too long to fit into any model’s context window?

Introducing the MLX Integration Library for Agent Framework

Thu, 11 Dec 2025 08:00:00 +0000

I’ve recently been working on setting up a bunch of Agent Framework samples, which would showcase the cooperation between cloud agents (backed by LLMs in the cloud) and local agents (running on your own machine). Since I primarily work on a Mac, the natural choice for me was to use MLX as the local model runner, which required a bit of bootstrapping - and felt quite tedious. So, the natural next step was to create a library that would make it easy to integrate MLX models into Agent Framework applications, since there wasn’t one available yet.

Today, I’m excited to announce the release of the MLX Integration Library for Agent Framework! This library simplifies the process of integrating MLX models into your Agent Framework applications, allowing you to leverage local Mac AI capabilities seamlessly alongside cloud-based agents.

SLM-default, LLM-fallback pattern with Agent Framework and Azure AI Foundry

Fri, 05 Dec 2025 08:00:00 +0000

When building AI workflows, we often face a choice: do we use a massive, expensive cloud model for everything (to ensure best reasoning capabilities), or do we cut costs with a smaller local model (and risk hallucinations)? In this post, we’ll explore a “best of both worlds” architecture, as described in the recent survey “Small Language Models for Agentic Systems” Sharma & Mehta, 2025.

We call this the “SLM-default, LLM-fallback” pattern. The premise is simple: route all queries to a fast, private, on-device Small Language Model (SLM) first. Only if that model cannot confidently answer the query, do we escalate the request to a paid cloud model (LLM).

LLM and SLM collaboration using the Minions pattern (with Phi-4-mini and Azure OpenAI)

Fri, 24 Oct 2025 07:06:14 +0000

In this post, we’ll explore a novel approach to optimizing AI workflows by strategically combining large language models (LLMs) with small language models (SLMs) using the “Minions pattern.” This technique, described in the research paper “Minions: Cost-efficient Collaboration Between On-device and Cloud Language Models” by Narayan et al., addresses one of the most pressing challenges in AI application development - the cost of processing large amounts of data with expensive, cloud-based language models. If you’ve ever built an AI system that needs to analyze extensive documents or datasets, you’ve probably felt the frustration of watching your API costs skyrocket as you process more and more content.

Running Phi models on iOS with Apple MLX Framework

Mon, 10 Mar 2025 08:30:12 +0000

As I previously blogged a few times, I have been working on the Strathweb Phi Engine, a cross-platform library for running Phi model inference via a simple, high-level API, from a number of high-level languages: C#, Swift, Kotlin and Python. This of course includes the capability of running Phi models on iOS devices, and the sample repo contains a demo SwiftUI application that demonstrates how to do this.

Today I wanted to show an alternative way of running Phi models on iOS devices, using Apple’s MLX framework. I previously blogged about fine-tuning Phi models on iOS using MLX, so that post is a good read if you want to learn more about the MLX framework and how to use it.

Fine tuning Phi models with MLX

Fri, 17 Jan 2025 07:06:14 +0000

Recently, I dedicated quite a lot of room on this blog to the topic of running Phi locally with the Strathweb Phi Engine. This time, I want to focus on a different aspect of adopting small language models like Phi - fine-tuning them. We are going to do this with Apple’s MLX library, which offers excellent performance for ML-related tasks on Apple Silicon.

We are going to do LoRA fine tuning of a Phi model, and then invoke it using Strathweb Phi Engine.