Of late, I've been involved in work to integrate APIs into LLM interactions, using Semantic Kernel. This post is something of a brain dump on the topic. Given how fast this space is moving, I expect what is written here to be out of date, possibly even before I hit publish. But nevertheless, I hope it's useful.
I've recently been working on building retrieval augmented generation (RAG) experiences into applications; building systems where large language models (LLMs) can query documents. To achieve this, we first need a strategy to chunk those documents and make them LLM-friendly. Kernel Memory, a sister project of Semantic Kernel supports this.
This post grew out of my desire to improve the metadata for my blog posts. I have been blogging for more than ten years, and the majority of my posts lack descriptions. A description is meta tag that sits in a page and describes the contents of the page. This is what this posts description meta tag looks like in HTML:
<meta
name="description"
content="Use the TypeScript Azure Open AI SDK to generate article metadata."
/>
Descriptions are important for search engine optimisation (SEO) and for accessibility. You can read up more on the topic here. I wanted to have descriptions for all my blog posts. But writing around 230 descriptions for my existing posts was not something I wanted to do manually. I wanted to automate it.
We're currently in the gold rush period of AI. The world cannot get enough. A consequence of this, is that rationing is in force. It's like the end of the second world war, but with GPUs. This is a good thing, because it means that we can't just spin up as many resources as we like. It's a bad thing, for the exact same reason.
If you're making use of Azure's Open AI resources for your AI needs, you'll be aware that there are limits known as "quotas" in place. If you're looking to control how many resources you're using, you'll want to be able to control the capacity of your deployments. This is possible with Bicep.
This post grew out of a GitHub issue around the topic where people were bumping on the message the capacity should be null for standard deployment as they attempted to deploy. At the time that issue was raised, there was very little documentation on how to handle this. Since then, things have improved, but I thought it would be useful to have a post on the topic.
