Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


Add to briefcase

Question about included columns in nonclustered indexes: Where exactly are they stored? Expand / Collapse
Author
Message
Posted Wednesday, October 1, 2008 11:55 AM
SSC Veteran

SSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC Veteran

Group: General Forum Members
Last Login: Today @ 7:00 AM
Points: 279, Visits: 1,114
I figured an indexing question would best be answered by folks who focus on performance tuning, if there was a better forum for this please let me know.

Anyway I was looking at: http://msdn.microsoft.com/en-us/library/ms177484(SQL.90).aspx and an article talking about indexing.

With included columns you can add them to a NONclustered index so that when a query uses that index additional columns the query SELECTs can be "included columns" and returned as part of the INDEX SEEK instead of having to hop over to the table, lookup the table row, and then scan that row of the table to get whatever columns are needed.

Classic example:
nonclustered index on: FirstName, LastName.
included columns: EmailAddress, PhoneNumber

The table would also of course have a PK which is part of the clustered index.



The clustered index might have other columns, maybe LastName or "whatever".
It sorts the table and is used to quickly locate records.

Here is what I don't get:

I do my query: SELECT FirstName, LastName, EmailAddress, PhoneNumber
FROM table
WHERE FirstName = 'John' AND LastName = 'Doe'

It then uses my NONclustered index to perform my query AND it would then locate the record(s) that match my criteria.

What happens then?

It looks like it would then have a pointer to the clustered index which it would use to locate the record in the table, and retrieve the data.

Except in this case it is a COVERING INDEX... and it has Email and Phone stored in the NONclustered index's leaf level.

So... what does it do?

Does it grab Email and Phone and THEN still jump to the clustered index to locate the row in the table so it can pull out First and Last Name?

What I'm getting at is this:
CLUSTERED indexes store the actual table at their leaf level, is that correct?

NONclustered stores pointers to the clustered index (the table) or to artificial keys in a heap (tables with no clustered index), right?

So... how does the included column on a NONclustered index allow the query to avoid having to lookup the data in the table itself?

That's the part I don't get.

It seems like it grabs Phone and Email from its leaf level, but still has to jump to the clustered index to get the Fname and Lname since it doesn't actually store those data values in the index, just pointers to them...

So what am I not following/understanding?

Thanks!
Post #579285
Posted Wednesday, October 1, 2008 12:18 PM


SSCoach

SSCoachSSCoachSSCoachSSCoachSSCoachSSCoachSSCoachSSCoachSSCoachSSCoachSSCoach

Group: General Forum Members
Last Login: Today @ 4:57 PM
Points: 15,661, Visits: 28,047
The columns defined in the INCLUDE statement are stored at the leaf level of the non-clustered index. There's no need to go to the cluster or a heap to retrieve the data. Storing the data increases the size of the index, but it doesn't change the size and levels of the B-Tree since the index keys are still the same. That means, for the most part, the index seek operations should be as fast as ever.

----------------------------------------------------
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood..." Theodore Roosevelt
The Scary DBA
Author of: SQL Server 2012 Query Performance Tuning
SQL Server 2008 Query Performance Tuning Distilled
and
SQL Server Execution Plans

Product Evangelist for Red Gate Software
Post #579302
Posted Wednesday, October 1, 2008 12:59 PM


SSC-Forever

SSC-ForeverSSC-ForeverSSC-ForeverSSC-ForeverSSC-ForeverSSC-ForeverSSC-ForeverSSC-ForeverSSC-ForeverSSC-ForeverSSC-ForeverSSC-ForeverSSC-ForeverSSC-ForeverSSC-Forever

Group: General Forum Members
Last Login: Today @ 3:07 PM
Points: 42,812, Visits: 35,931
Maxer (10/1/2008)
It seems like it grabs Phone and Email from its leaf level, but still has to jump to the clustered index to get the Fname and Lname since it doesn't actually store those data values in the index, just pointers to them...


FName and LName are the index keys, so they are stored at all levels of the index. They have to be, how else would SQL be able seek on those columns.

In a nonclustered index, the nonclustered index keys and the clustered index keys are stored at all levels of the index. The include columns are only stored at the leaf level. The clustering keys are there as a 'pointer' to the row, in case a bookmark lookup is required.



Gail Shaw
Microsoft Certified Master: SQL Server 2008, MVP
SQL In The Wild: Discussions on DB performance with occasional diversions into recoverability

We walk in the dark places no others will enter
We stand on the bridge and no one may pass

Post #579330
Posted Wednesday, October 1, 2008 1:34 PM
SSC Veteran

SSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC Veteran

Group: General Forum Members
Last Login: Today @ 7:00 AM
Points: 279, Visits: 1,114
GilaMonster (10/1/2008)
Maxer (10/1/2008)
It seems like it grabs Phone and Email from its leaf level, but still has to jump to the clustered index to get the Fname and Lname since it doesn't actually store those data values in the index, just pointers to them...


FName and LName are the index keys, so they are stored at all levels of the index. They have to be, how else would SQL be able seek on those columns.

In a nonclustered index, the nonclustered index keys and the clustered index keys are stored at all levels of the index. The include columns are only stored at the leaf level. The clustering keys are there as a 'pointer' to the row, in case a bookmark lookup is required.


Ah, of course. Indeed otherwise how would it be able to seek on those columns :)

Well that explains that.

Thanks for pointing out my obvious logical error there. I knew I was failing to take something into account, and that would certainly be it!

Post #579350
Posted Tuesday, November 23, 2010 1:01 PM
SSC Journeyman

SSC JourneymanSSC JourneymanSSC JourneymanSSC JourneymanSSC JourneymanSSC JourneymanSSC JourneymanSSC Journeyman

Group: General Forum Members
Last Login: Monday, April 7, 2014 6:15 PM
Points: 86, Visits: 216
Actually, the included columns make the leaf index entries larger. Therefore, fewer entries per page, which translates to more B-tree above. So there is a size cost for the entire tree structure based on the percent increase in size in the index entry. Increase in tree depth and seek time will be logarithmic increase.

Still, well defined include columns can yield a big improvement.


David Lathrop
DBA
WA Dept of Health
Post #1025428
Posted Friday, November 16, 2012 8:13 AM
Valued Member

Valued MemberValued MemberValued MemberValued MemberValued MemberValued MemberValued MemberValued Member

Group: General Forum Members
Last Login: Tuesday, June 3, 2014 12:21 PM
Points: 53, Visits: 251
I know it's an old topic but how is included columns affecting memory footprint? Some say higher use to put index in.memory cache. Some say lower since less pages are read and cached since it is in the index.

K
Post #1385721
Posted Saturday, November 17, 2012 10:28 AM
SSCommitted

SSCommittedSSCommittedSSCommittedSSCommittedSSCommittedSSCommittedSSCommittedSSCommitted

Group: General Forum Members
Last Login: Saturday, August 16, 2014 5:48 AM
Points: 1,862, Visits: 3,602
Kristian Ask (11/16/2012)
I know it's an old topic but how is included columns affecting memory footprint? Some say higher use to put index in.memory cache. Some say lower since less pages are read and cached since it is in the index.

K


There is less memory footprint because the idea behind a non-clustered index with INCLUDE columns is to help avoid bookmark lookups in the query plan. Bookmark lookups are operations that go and fetch data from the clustered index (ie. the actual table) to help return columns that are in the SELECT portion of the query. Imagine a very wide table with tens or hundreds of columns being loaded into memory for the bookmark lookup. The memory footprint will be higher than if only a select few columns are instead loaded into memory as include columns in a noncl. Index.

Does my somewhat muddled explanation make sense?


__________________________________________________________________________________

Turbocharge Your Database Maintenance With Service Broker: Part 2
Turbocharge Your Database Maintenance With Service Broker: Part 1
Real-Time Tracking of Tempdb Utilization Through Reporting Services
Monitoring Database Blocking Through SCOM 2007 Custom Rules and Alerts
Preparing for the Unthinkable - a Disaster/Recovery Implementation
Post #1385952
Posted Monday, November 19, 2012 3:01 AM
Valued Member

Valued MemberValued MemberValued MemberValued MemberValued MemberValued MemberValued MemberValued Member

Group: General Forum Members
Last Login: Tuesday, June 3, 2014 12:21 PM
Points: 53, Visits: 251
I got it... sound logical. One more thing though.

If the included columns is not creating a covering index, will the included columns still be read or is it then only using the table?
Post #1386253
Posted Monday, November 19, 2012 4:03 AM


SSC-Forever

SSC-ForeverSSC-ForeverSSC-ForeverSSC-ForeverSSC-ForeverSSC-ForeverSSC-ForeverSSC-ForeverSSC-ForeverSSC-ForeverSSC-ForeverSSC-ForeverSSC-ForeverSSC-ForeverSSC-Forever

Group: General Forum Members
Last Login: Today @ 3:07 PM
Points: 42,812, Visits: 35,931
The included columns will still be read.


Gail Shaw
Microsoft Certified Master: SQL Server 2008, MVP
SQL In The Wild: Discussions on DB performance with occasional diversions into recoverability

We walk in the dark places no others will enter
We stand on the bridge and no one may pass

Post #1386281
Posted Monday, November 19, 2012 4:35 AM


SSCrazy

SSCrazySSCrazySSCrazySSCrazySSCrazySSCrazySSCrazySSCrazy

Group: General Forum Members
Last Login: Tuesday, August 19, 2014 12:26 AM
Points: 2,840, Visits: 3,963
Kristian Ask (11/19/2012)
If the included columns is not creating a covering index, will the included columns still be read or is it then only using the table?
yes they will be retrieved with the help of keylook up or RID lookup.


-------Bhuvnesh----------
I work only to learn Sql Server...though my company pays me for getting their stuff done
Post #1386295
« Prev Topic | Next Topic »

Add to briefcase

Permissions Expand / Collapse