I like to subject myself to agony from time to time. So, I tried to build my SQL Server 2025 test environment in Docker from scratch. I thought I was getting behind with the good stuff about the native AI integration into the Microsoft database platform – vector search and the concept of embedding in particular. Spinning a SQL Server container in Docker is easy peasy.
But setting up SQL Server to connect to the Large Language Model over a Secure Proxy is not for the faint of heart. Just the angst of getting the spaces aligned in the YAML file can toss you into existential crisis. After almost a couple of hours of pulling my hair trying to fix one error after another, I gave up.
If you are like me who’s itching to try the SQL Server 2025 features and its promises of good fortune and brighter tomorrow, especially the vector search and embedding stuff, there’s a repo out there that you can clone to your personal laptop to easily and quickly set up SQL 2025, LLM (Ollama), and Secure Proxy (NGINX), which is required by SQL Server – all in one setup via Docker Compose. Well, needless to say, the only thing that you need is the Docker Desktop app installed in your personal laptop (and Git).
And aren’t we so lucky to have the people who dedicate their precious time to making our life easier?
Enter Ollama SQL FastStart.
Ollama SQL FastStartstreamlines the deployment of SQL Server 2025 with integrated AI capabilities through a comprehensive Docker-based solution. This project delivers a production-ready environment combining SQL Server 2025, Ollama’s large language model services, and NGINX with full SSL support—all preconfigured to work together seamlessly
This is huge in my humble opinion! This is helping to bring the boon of the first AI-ready enterprise database platform to the hand of the masses. Microsoft you failed at this noble commission! To be fair, Microsoft has a getting-started blog on this subject here. But that’s the long way. Ollama SQL FastStart is the short cut.
Thanks Anthony Nocentino for this Getting Started with Vector Search in SQL Server 2025 Using Ollama kit. This will save the world a lot of headaches. And might just avert a worldwide catastrophe caused by existential crises.
Anthony’s Ollama SQL FastStart kit just works out of the box. It really makes it easy and quick to spin the whole environment not in minutes but seconds! I’m exaggerating there a bit of course, but a couple of Bash commands is all you need to set the whole thing up. You’ll be up and running those vector searches in no time!
Install Git & Clone the Repo
If you haven’t done so, download and install Git. And the Docker Desktop app if that’s not installed yet also.
I have a folder in my local called C:Docker. This is where I usually save all my Docker related stuff. You may want to create a folder anywhere that is convenient for you. Open Git Bash in your folder.
Then clone the Ollama SQL FastStart project from GitHub.
git Clone https://github.com/nocentino/ollama-sql-faststart.gitChange dir to ollama-sql-faststart:
cd ollama-sql-faststartThen build and start the services:
docker compose up --detachGive it a couple of minutes to let it do its magic.
If you’ve setup your Docker correctly, you should now see the ollama-sql-faststart Docker compose stack, like so:
You may want to slide that Only show running containers option to have a cleaner look in the compose stack.
Install and Open SSMS 22
This is probably the best time to test SQL Server Management Studio v22 (Preview as of this writing) if you haven’t done so. Download it here.
Install SSMS v22 locally to your laptop.
Login to the SQL Server instance in Docker:
Server Name: host.docker.internal,1433
The password detail is found in Anthony’s blog linked above. And of course, you may change this after you complete your setup.
Open the SQL script called configure_model.sql from C:<your dir here>ollama-sql-faststartscriptsconfigure_model.sql. Then execute it
And everything else that you need to get started is in the configure_model.sql script found in C:<your dir here>ollama-sql-faststartscriptsconfigure_model.sql.
And that’s it! It works like magic!
And of course, aside from the vector search, SQL Server 2025 has a lot to offer. Find them here.
The post How to Set Up SQL Server 2025, Ollama and NGINX in Docker Without the Headaches first appeared on SQL, Code, Coffee, Etc..