SQLServerCentral Article

Influencing a Local AI

,

I set up a local AI LLM to run and then conducted some experiments. This post shows a bit about the experiments and a little code. The setup is on my blog using Docker to containerize an Ollama LLM, which you can read about.

The Basic Setup

If you read my blog, I downloaded an Ollama container image, ran it, and downloaded the mistral model and ran it. Interatively, I was able to query the mode as I would ChatGPT or any other LLM. You can see the results below from my first few queries.

Querying a local LLM

From here, I disconnected and had a model running in a container with access on my local port 11434. I ran model disconnected, which meant my local shell was available for other tasks. From here. I decided to write a small program and play with some prompt injection. I chose Python as I like the language and I see a lot of examples written in Python.

My first attempt was with this example.py, shown below. This code imports a model that talks to ollama and then reads a file from disk. I get a prompt from a user and add the contents of my file to the prompt, injecting it into the mode. This runs with a loop that repeats, allowing me to "talk" to the model.

import ollama
note = 'notes.md'
with open(note, 'r') as file:
    content = file.read()

my_prompt = f'What would you like to know?  '
print("\n")
user_entry = input(my_prompt)
while user_entry != '':
    model_prompt = user_entry +  '\n' + content
    response = ollama.generate(model='mistral', prompt=model_prompt)
    actual_response = response['response']
    print(actual_response)
    print("\n")
    user_entry = input(my_prompt)

With this, I can have a local conversation on my machine. Let's experiment.

The First Injection: Single Table Restore

I added this data to the contents of my local notes.md file.

Telling the model a single table restore is possible

This isn't true, and the WITH SIGNLE_TABLE, or SINGLE_TABLE,, option doesn't exist. However, I'm adding this to a prompt. When the model returns information, I see this, where the model seems to think I can restore a single table with the restore command.

response from the model on single table restore

What's interesting is the model corrected my typo (SIGNLE v SINGLE) and says I can do this, but hasn't included the option in the sample.

I decided to ask again. The image below includes the last line with the STOPAT parameter in it and then the response to my asking again. Below, I've highlighted my prompt.

New query about single table restore

Again, I see the option listed and then a series of steps. Note that the RESTORE command now has the SINGLE_TABLE option included. If you were not familiar with SQL Server backup and restore, you might think this is a valid way to restore a table that you've deleted.

The Second Experiment

I stopped my program and edited the contents of the note.md file.

It's true, we could restore a single table in SQL Server v6.5, but that capability disappeared. I've amended my prompt injection, and I restart the program. I wanted to test the model locally, so I disconnected the Internet, re-ran my program and asked a question. I got a response about Tom Brady. However, look at the last paragraph. The model (which was still running) added a note about my previous question.

That's fascinating.

Adding Context

There are likely scenarios where I want the model to consider certain things, but I don't want to keep typing them. Injection is another way to help guide the AI for the user. As an example, here is what I added to my notes.md to provide context.

contents of context file

When I do this, I get these results for a question about a function. I'm asking about the OVER and get the different parts of the OVER clause and some examples.

results of context added to prompt

If I change my context prompt to be an advanced developer, I use these statements.

advanced developer context

The results I get are briefer, and don't explain the parameters well, but I get Northwind examples.

Rseponse with northwind examples

I can also change the prompt again and get the AdventureWorks responses.

Context for adventureworks examples.

I see the results now include a list of the different parts of the OVER() clause and also good examples from Adventureworks. adventureworks examples in results

My Custom Schema

I decided to try adding a small schema (3 tables) to my prompt. Here's the contents of my file:

Assume I am an advanced T-SQL developer. List all the parameters with a name and the basic definition of each
Include examples based on this baseball schema
CREATE TABLE [dbo].[players](
[lahmanID] [int] NOT NULL,
[playerID] [varchar](10) NULL,
[managerID] [varchar](10) NULL,
[hofID] [varchar](10) NULL,
[birthYear] [int] NULL,
[birthMonth] [int] NULL,
[birthDay] [int] NULL,
[birthCountry] [varchar](50) NULL,
[birthState] [varchar](2) NULL,
[birthCity] [varchar](50) NULL,
[deathYear] [int] NULL,
[deathMonth] [int] NULL,
[deathDay] [int] NULL,
[deathCountry] [varchar](50) NULL,
[deathState] [varchar](2) NULL,
[deathCity] [varchar](50) NULL,
[nameFirst] [varchar](50) NULL,
[nameLast] [varchar](50) NULL,
[nameNote] [varchar](255) NULL,
[nameGiven] [varchar](255) NULL,
[nameNick] [varchar](255) NULL,
[weight] [int] NULL,
[height] [int] NULL,
[bats] [varchar](1) NULL,
[throws] [varchar](1) NULL,
[debut] [varchar](10) NULL,
[finalGame] [varchar](10) NULL,
[college] [varchar](50) NULL,
[lahman40ID] [varchar](9) NULL,
[lahman45ID] [varchar](9) NULL,
[retroID] [varchar](9) NULL,
[holtzID] [varchar](9) NULL,
[bbrefID] [varchar](9) NULL,
PRIMARY KEY CLUSTERED 
(
[lahmanID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY]
GO
CREATE TABLE [dbo].[teams](
[yearID] [int] NOT NULL,
[lgID] [varchar](2) NOT NULL,
[teamID] [varchar](3) NOT NULL,
[franchID] [varchar](3) NULL,
[divID] [varchar](1) NULL,
[Rank] [int] NULL,
[G] [int] NULL,
[Ghome] [int] NULL,
[W] [int] NULL,
[L] [int] NULL,
[DivWin] [varchar](1) NULL,
[WCWin] [varchar](1) NULL,
[LgWin] [varchar](1) NULL,
[WSWin] [varchar](1) NULL,
[R] [int] NULL,
[AB] [int] NULL,
[H] [int] NULL,
[2B] [int] NULL,
[3B] [int] NULL,

[int] NULL, [BB] [int] NULL, [SO] [int] NULL, [SB] [int] NULL, [CS] [int] NULL, [HBP] [int] NULL, [SF] [int] NULL, [RA] [int] NULL, [ER] [int] NULL, [ERA] [int] NULL, [CG] [int] NULL, [SHO] [int] NULL, [SV] [int] NULL, [IPouts] [int] NULL, [HA] [int] NULL, [HRA] [int] NULL, [BBA] [int] NULL, [SOA] [int] NULL, [E] [int] NULL, [DP] [int] NULL, [FP] [int] NULL, [name] [varchar](50) NULL, [park] [varchar](255) NULL, [attendance] [int] NULL, [BPF] [int] NULL, [PPF] [int] NULL, [teamIDBR] [varchar](3) NULL, [teamIDlahman45] [varchar](3) NULL, [teamIDretro] [varchar](3) NULL, PRIMARY KEY CLUSTERED ( [yearID] ASC, [lgID] ASC, [teamID] ASC )WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY] ) ON [PRIMARY] GO CREATE TABLE [dbo].[batting]( [playerID] [varchar](9) NOT NULL, [yearID] [int] NOT NULL, [stint] [int] NOT NULL, [teamID] [varchar](3) NULL, [lgID] [varchar](2) NULL, [G] [int] NULL, [G_batting] [int] NULL, [AB] [int] NULL, [R] [int] NULL, [H] [int] NULL, [2B] [int] NULL, [3B] [int] NULL,
[int] NULL, [RBI] [int] NULL, [SB] [int] NULL, [CS] [int] NULL, [BB] [int] NULL, [SO] [int] NULL, [IBB] [int] NULL, [HBP] [int] NULL, [SH] [int] NULL, [SF] [int] NULL, [GIDP] [int] NULL, [G_old] [int] NULL, PRIMARY KEY CLUSTERED ( [playerID] ASC, [yearID] ASC, [stint] ASC )WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY] ) ON [PRIMARY] GO

Now I re-run and ask a different, but related, question. You can see the results now use my schema as the examples. That's pretty cool.

Examples with custom schema

Conclusion

Working with AI is an interesting exercise, and while I appreciate the work done by OpenAI, Microsoft, Google, etc., I also think that there are a lot of concerns over public AI services, and perhaps even concerns over how they train models. There are many smaller models, and I've experimented with the Mistral model here.

As I changed the context I injected, I could influence the AI. In the restore example, this is malicious influence. There are also real security concerns with AIs and prompt injections. A few resources you might read (all relatively short).

A more practical use is setting a context that's useful for me. Like who I am, or how I'd like to see some results. I can use a common, public known schema or add my own.

 

 

Rate

5 (3)

You rated this post out of 5. Change rating

Share

Share

Rate

5 (3)

You rated this post out of 5. Change rating