Using Ollama on Windows 11 as an Alternative to Public LLMs

In my ongoing war against shadow AI, I’ve been testing out alternatives that most everyone can use, no matter what your technical expertise.  Although somewhat limited depending on your resources available, (laptop CPU/GPU, memory, etc.) If you want to try the newest open models without sending your prompts to the cloud, or you just want a controllable sandbox for demos, the new offering of Ollama on Windows 11 is a great way to run LLMs locally. You’re going to trade some performance and convenience compared to Copilot or ChatGPT, but you gain privacy, offline capability, and a lot of tinkering power.  The truth is, if you need some privacy, this is the way to go and if you’re learning, playing around on a laptop isn’t the worst way to go if you’re working with company or private data.

Below is a practical guide: what to expect, how to install, which models to try, and how to squeeze the most out of a Windows laptop or desktop.

Why local instead of public:

• Privacy & control: Everything runs on your machine; prompts and documents don’t leave the PC. That’s attractive for regulated, company data and internal prototypes. Windows-focused outlets have been making the same case, why not use local tools, which can be a smart alternative for many scenarios.
• Offline + cost: No subscription required; you can experiment even without internet.
• Caveat: Copilot (and Microsoft 365 Copilot) integrates deeply with your tenant’s permissions and data boundaries, which is useful in enterprise, if you’re okay with cloud inference.

What you’ll need (realistic expectations)

• CPU-only works, especially with small models (0.5B–7B). A recent article even shows usable results on a 7-year-old laptop, but just temper expectations (think “helper,” not “superhuman coder”).
• GPU helps a lot. Ollama supports NVIDIA GPUs on Windows (CUDA); AMD Radeon acceleration has been introduced for Windows and Linux and continues to mature.
• Model size matters. Smaller, quantized models (e.g., Q4_K_M) load and respond faster but may lose some quality vs. higher-precision variants like Q8_0.
• Context length costs RAM/VRAM. Huge context windows (e.g., 32k–64k tokens) can tank performance; dial them back to keep things a bit snappier.

Once we run through the install, the GUI interface is quite intuitive if you’re used other generative text AI in the past, so I’m going to add the CLI options as well.  I appreciate them and to be honest, Ollama performs better when run from just the CLI, which should be expected when running on a local laptop.

Install Ollama on Windows 11

The easiest to install from the command line is either Windows Installer or WinGet.  Here’s the WinGet command:

1. Installed via WinGet

winget install --id=Ollama.Ollama -e

Note: Ollama runs a local service at http://localhost:11434 and gives you a CLI (ollama) plus a friendly Windows GUI app that makes chatting with local models easy (no terminal required).

Pull a model and run your first prompt

Browse the Ollama model library to pick something light (e.g., llama3.2:3b, phi4:14b, qwen2.5:7b, or a coding model like qwen2.5-coder:7b). Then:

2. Example: pull a small general model

ollama pull llama3.2:3b

Output:

3. Chat from the CLI

ollama run llama3.2:3b "Summarize ACID in relational databases in two sentences."

Output:
ACID stands for Atomicity, Consistency, Isolation, and Durability, which are four fundamental principles that
ensure the reliability and integrity of database transactions in relational databases. By adhering to these
principles, ACID guarantees that database operations are processed reliably, even in the presence of failures or
concurrency issues, ensuring data consistency and accuracy.

I was curious how long it took, so ran it again in my Powershell console, (I’m such a terminal girly…) and this is what I received, sans the larger values in time, which had 0 values:

PS C:\WINDOWS\system32> Measure-Command {ollama run llama3.2:3b "Summarize ACID in relational databases in two sentences."}

Seconds           : 12
Milliseconds      : 383
Ticks             : 123839645
TotalMinutes      : 0.206399408333333
TotalSeconds      : 12.3839645
TotalMilliseconds : 12383.9645

So how did this compare to running it in ChatGPT 5? I ran it from a browser, and updated the prompt to request the time in milliseconds:

ACID means each transaction is Atomic (all-or-nothing), Consistent (preserves rules/invariants), Isolated (concurrent work doesn’t interfere),
and Durable (once committed, it survives crashes). Together these properties make reads/writes reliable and predictable—even under concurrency, rollbacks, or failures.

I can’t directly and accurately measure your end-to-end chat round-trip time from inside this interface (network + browser).

Per ChatGPT, it brought back more than requested, (over achiever!) and I ended up having to take it back to Powershell if I wanted to get a real timing outside of the 59 seconds reported internally to the browser.  With a quick check, came up with the following command to verify the time outside of the additional browser times:

$sw = [System.Diagnostics.Stopwatch]::StartNew()
$body = @{ prompt = "Summarize ACID in two sentences." } | ConvertTo-Json
Invoke-RestMethod -Method Post -Uri http://localhost:8000/chat -ContentType "application/json" -Body $body | Out-Null
$sw.Stop(); "$($sw.ElapsedMilliseconds) ms"

The total time consumed by ChatGPT 5 to do the same from a public LLM is 4283 ms, in other words,

Yes, ChatGPT was quicker to respond when removing the browser and additional output, but it should be expected.  We’re just running this on our laptop and I don’t have a GPU in sight.

Tuning for speed on Windows

If responses feel sluggish, try the following in order.  There’s no guarantee you’ll get break-neck speeds, but it will help if Ollama isn’t performing at acceptable speeds:

1. Pick the right model size & quantization.

   Start with ~3B–7B parameter models quantized to Q4_K_M for a good size/quality balance; step up only if you need more reasoning.

2. Reduce the context length.

   Don’t default to 32k or 64k unless you need it; 4k–8k often performs much better on typical PCs.

3. Use your GPU (if NVIDIA).

   Install current NVIDIA drivers; Ollama will automatically accelerate with CUDA if supported. You can also set Windows “Graphics settings” to force the Ollama app to use the high-performance GPU.

4. Keep memory under control.

   Limit simultaneous loaded models and how long they stay in memory:

   • OLLAMA_MAX_LOADED_MODELS=1
   • OLLAMA_KEEP_ALIVE=5m

     These are standard server env vars you can set in Windows “Environment Variables” before launching Ollama.

5. Use the new Windows GUI for quick toggles.

   The app exposes basic options (like context window) without editing files, which I found handy when I was experimenting.

When to Not Use Ollama

Consider Copilot when:

• Deep Microsoft 365 integration: If you need tenant-aware grounding across SharePoint, Outlook, Teams, etc., Copilot’s data-boundary logic and permissions are built in.
• Bigger models / turnkey accuracy: Cloud services still win for the largest, most capable frontier models.  There’s no local VRAM constraints.

Consider ChatGPT, Perplexity, Claude, etc. (PAID versions) when:

• Generic questions regarding business/organizational work. No critical data or PII is involved.
• Help reformatting or creating more crisp/concise wording in correspondence/documents.
• Create what I call, “filler content” around any proprietary or intellectual property.
• Remember to never upload or paste critical data/intellectual property into public generative AI.

Troubleshooting quick hits

• “It’s all CPU.” Make sure you’re on a supported NVIDIA GPU with current drivers whenever possible; AMD acceleration on Windows exists but is newer and may vary by model/driver.
• “The app works, but API calls fail.” Check the service at http://localhost:11434/api/tags. If it’s not responding, start Ollama and wait a few seconds.
• “Model downloads are huge.” Choose smaller/quantized variants (e.g., :3b, Q4_K_M) and lower the context window.

Summary

If Copilot/ChatGPT(Paid, not free), OpenAI, Perplexity, etc. is your daily driver for Generative AI, think of Ollama as a local test bench. It’s perfect for validating prompts, exploring new open models, and building small end-to-end prototypes, both privately and offline. Start with a 3B–7B model, keep the context window modest, and let the new Windows app handle the basics while you get hands-on with the API for deeper experiments

References

These are my go-to I used to understand, to install and to troubleshoot.  They’ve been the most helpful and I’d be lying if I didn’t say I’ve asked ChatGPT and Intellisense to help me when stuck with some of the challenges a Google search failed helping me on.

• Download Ollama for Windows (installer & CLI/GUI). Ollama
• API quickstart (generate/chat examples). Ollama
• Model library (browse/choose models). Ollama
• NVIDIA GPU support (what’s supported). ollama.qubitpi.org
• AMD on Windows (support introduced and evolving). Ollama
• PowerShell Docs (support as I learn) Microsoft Learn
• PowerShell Timing using the Measure-Command
• Context length and performance tips. Windows Central
• Why can local AI be a smart choice. Windows Central

Peace out,

DBAKevlar

Join the debate, and respond to the editorial on the forums

This article shows how to define environments in your TOML files, use resolvers to provide secure connection details, and configure per-environment overrides and placeholders. It explains how this approach simplifies automation, makes CI/CD pipelines easier to manage, and helps teams work consistently and securely across multiple databases.

Redgate Test Data Manager’s latest release brings significant performance improvements and new masking capabilities. These updates reduce friction in your workflows, whilst maintaining security and data quality standards.

I’m pleased to announce that Flyway continues to improve its support for Oracle databases with every release, focusing on performance, reliability, and developer efficiency. The most recent updates to Flyway Desktop and Flyway CLI specifically target a long-standing challenge for Oracle users: unnecessary table rebuilds when migrating to partitions or when managing partitions. This post highlights the changes in the latest version, new behaviors, and upcoming features that will benefit teams working with Oracle partitioned tables.

Generative AI is profoundly revolutionizing the cr...

There is a really great series of online events hi...

Here’s an interesting story about a failure being introduced by LLM-written code. Specifically, the LLM was doing some code refactoring, and when it moved a chunk of code from...

As I was waiting to start a recent episode of Live...

We have been working with AI models for development a lot lately (yes, just like everyone else). And I’m seesawing between “damn, that’s impressive” and “damn, brainless fool” quite...

The following is Part 1 of 3 from Addy Osmani’s original post “Context Engineering: Bringing Engineering Discipline to Parts.” Context Engineering Tips: To get the best results from an...

Let me give you some introduction, and for some of you flashbacks to your early computer science classes. There is an initial state, flow lines that show what are the next legal states, and one or more termination states. These diagrams also called finite state automata, and you’ll see it in freshman computer science classes. To date, we might implement them with a graph database. What is worth relearning are our old tricks from SQL.

Using a varchar column to reference another table can be a huge performance killer and stands in the way of a well-normalized database, but changing the schema of a table in use is a challenging problem. A lengthy downtime might be the quickest solution to come to mind, but this technique will allow you to do the work in smaller bites without taking a downtime, all while eliminating the risk and stress of a long rollback process in the case of an error or a full transaction log.

David Seis digs into another dbatools cmdlet: As ...

Here’s a homework challenge:

When I was in the world of EMS, you rarely started a course for the next level of license without a chapter or two on the legal side. They...

Corruption isn’t a “maybe someday” problem �...

When deploying SQL Server in enterprise environments, choosing the right service account model is critical for security, manageability, and scalability.

Compression was introduced in SQL Server version 2008. While it helped, the compression ratios were frequently too low to justify the...

As DBAs we install SQL Server for various reasons regularly. If you could save time for each installation for more critical tasks, would you? In this blog post, we...

Yesterday on the r/SQL subreddit there was a post about someone, who’s boss was using ChatGPT to generate queries against an operational database, and in shocking news, the queries sucked and were causing large amounts of resource contention.

Join us live as we answer your questions about Pow...

With just under 100 days to go before PASS Data Community Summit in Seattle, we wanted to share with you a summary of all the Redgate sessions you’ll be...

On the last day of my Caribbean cruise, as we pull into port in Miami, let’s go through your top-voted questions from https://pollgab.com/room/brento. Here’s what we covered: 00:00 Start...

When managing storage infrastructure at scale, one...

This month, we’re getting practical and tactical to answer … What should you know about de-identifying data? Data de-identification is where technical data protection methods intersect with legal and...

Hugo Kornelis dives into the arcane: But what you probably don’t know is how that hash table is structured. How is the data stored? Where…

I realized I never created a post to show how to deploy Terraform from VS Code. I haven’t done that in a while because I don’t do it at... The...

Here's how DevSecOps 2.0 transforms security from a last-minute hurdle into a core enabler of resilient, fast, and compliant software delivery.

Lukas Eder makes a recommendation: Why, yes of cou...

Phil Yang lays out how to make a migration: Apache Kafka has made a landmark shift in KIP-500 with the introduction of Kafka Raft (KRaft) mode, eliminating…

Marco Russo and Alberto Ferrari share some thought...

Marco Russo and Alberto Ferrari’s example goes sideways: DAX calculation items do not provide full recursion. However, a limited form of recursion is available, known…

This article describes the sideways recursion triggered by invoking a calculation item from another calculation item, explaining why it should be avoided to steer clear of unexpected results. DAX...

Andy Leonard works in Microsoft Entra: My older so...

Elizabeth Oldag announces a pricing change: We’r...

Matt Basile grabs some data: AzCopy is a powerful and performant tool for copying data between Azure Storage and Microsoft OneLake, and is the preferred…

In today’s data-driven world, organizations are ...

Configuring Azure and Fabric security to allow read access to Fabric Data Factory pipelines and notebooks.

Microsoft bought GitHub for $7.5 billion in 2018.

This article outlines the key areas to monitor in Oracle ASM, why they matter, and what SQL or tools to use for visibility and alerts.

Imagine that you are given the following task, with a file like this:Name,Department,Salary,JoinDate John Smith,Marketing,75000,2023-01-15 Alice Johnson,Finance,82000,2022-06-22 Bob Lee,Sales,68000,2024-03-10 Emma Davis,HR,71000,2021-09-01You want to turn that into a single list of all the terms...

Curiosity often leads to the most interesting technical adventures. This time, I decided to explore something off the beaten path: running Debian GNU/Hurd inside a virtual machine on my...

Mercy Bassey grabs a certificate: Read on to see how.

People often ask “How can I automatically rebuild by indexes regularly?” or “When should I rebuild my indexes in PostgreSQL?”.

Learn how to run PostgreSQL in Docker with production-grade SSL encryption, certificate-based authentication, and custom configuration files - perfect for secure development environments.… The post Secure PostgreSQL in Docker: SSL,...

Ben Richardson looks at fonts: Want your Power BI ...

Gilbert Quevauvilliers wants to download a report: I am sure we have all had it where there is Power BI report in the service which…

What if you could zoom in on your data one day at a time?

Continuing my (already very long) series on what information you should be adding to the AI Instructions of your semantic model and why, in this post I’ll show you...

Explore the benefits and hidden risks of Power BI’s TMDL View. Learn how to safely use TMDL scripts with expert tips on security and best practices.… The post Power BI...

I’m late to the party this month. Taiob Ali has ...

A senior developer seeks leadership opportunities. Here's practical advice on positioning yourself for a promotion while maintaining technical credibility.

Imagine this situation, someone edits a stored pro...

Louis Davidson continues a series on regular expre...

In a tech landscape dominated by distributed systems, serverless architectures, and real-time analytics, one might assume that SQL, a language born in the 1970s, would be fading into obscurity.... The...

Headlines one-upping each other on the number of passwords exposed in a data breach have become somewhat of a sport in recent years.

Exploits allow for persistent backdooring when targets open booby-trapped archive.

In this article I want to point out some important evasion-methods for anyone concerned about Auditing data access.

The fourth part of this five-part series on stopping prompt injection attacks describes how dangerous such attacks can be.

Learn T-SQL With Erik: SELECT INTO Going Further I...

Ivan Palomares Carrascosa takes us through a simple natural language processing problem and solution: It’s no secret that decision tree-based models excel at a wide range of…

Learn T-SQL With Erik: Identity Annoyances Going Further If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75%...

Erik Darling walks through something that can slow down your temp table loads. The video covers a scenario in which a user creates a temp…

Learn T-SQL With Erik: Ordered Deletes and Table Expressions Going Further If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m...

Yes, SSMS 21 uses the Visual Studio installer. No, you don’t need to download it every time. No, you don’t need a Visual Studio license to use it. The post...

Randolph West clears up a few misconceptions about SQL Server Management Studio 21’s installer: There’s been some confusion lately about the SQL Server Management Studio…

Josephine Bush deploys some resources: I realized I never created a post to show how to deploy Terraform from VS Code. I haven’t done that…