SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Books for Query Optimizer inner workings?


Books for Query Optimizer inner workings?

Author
Message
ducon
ducon
Old Hand
Old Hand (327 reputation)Old Hand (327 reputation)Old Hand (327 reputation)Old Hand (327 reputation)Old Hand (327 reputation)Old Hand (327 reputation)Old Hand (327 reputation)Old Hand (327 reputation)

Group: General Forum Members
Points: 327 Visits: 286
Hi there,

I'm looking for a book to read about the inner workings of the query optimizer for SQL 2005 and 2008. I read about it whenever I can find stuff (eg. http://blogs.msdn.com/sqlqueryprocessing/) but it's not updated very often. BOL is a good start but I would love to find a book dedicated to the query optimizer that explains how it works, how it chooses certain plans over others etc.

Is there something out there that's dedicated to it and does not just have one chapter that talks about it in the middle of a more generale performance tuning book?

Thanks

Greg
steveb.
steveb.
SSCarpal Tunnel
SSCarpal Tunnel (4.4K reputation)SSCarpal Tunnel (4.4K reputation)SSCarpal Tunnel (4.4K reputation)SSCarpal Tunnel (4.4K reputation)SSCarpal Tunnel (4.4K reputation)SSCarpal Tunnel (4.4K reputation)SSCarpal Tunnel (4.4K reputation)SSCarpal Tunnel (4.4K reputation)

Group: General Forum Members
Points: 4432 Visits: 7195
this book has some excellent information about the Optimizer

http://www.amazon.com/Microsoft-SQL-Server-2008-Internals/dp/0735626243/ref=pd_sim_b_2

infact the entire
Inside Microsoft SQL Server series is excellent in helping understand the internals of SQL server.
GilaMonster
GilaMonster
SSC Guru
SSC Guru (88K reputation)SSC Guru (88K reputation)SSC Guru (88K reputation)SSC Guru (88K reputation)SSC Guru (88K reputation)SSC Guru (88K reputation)SSC Guru (88K reputation)SSC Guru (88K reputation)

Group: General Forum Members
Points: 88266 Visits: 45277
I don't think there is a comprehensive book, as much of the internal workings, the costing algorithms etc, as far as I know are not public knowledge.

There's a large chapter in SQL 2008 Internals written by Conor Cunningham (one of the guys working on the optimiser), other than that there's some in Inside SQL Server 2005: Query Tuning and Optimisation, there's stuff scattered through Grant's book "SQL 2008 Query performance tuning distilled"

Also, while it's not specifically SQL server, it may be worth doing a search for academic papers on Query Optimisation. Try scholar.google.com as a starting point. It's really theoretical but if you are interested in this stuff it's worth a look.

Gail Shaw
Microsoft Certified Master: SQL Server, MVP, M.Sc (Comp Sci)
SQL In The Wild: Discussions on DB performance with occasional diversions into recoverability

We walk in the dark places no others will enter
We stand on the bridge and no one may pass


Shawn Melton
Shawn Melton
SSCrazy
SSCrazy (2.3K reputation)SSCrazy (2.3K reputation)SSCrazy (2.3K reputation)SSCrazy (2.3K reputation)SSCrazy (2.3K reputation)SSCrazy (2.3K reputation)SSCrazy (2.3K reputation)SSCrazy (2.3K reputation)

Group: General Forum Members
Points: 2285 Visits: 3513
ducon (1/4/2010)
Hi there,

I would love to find a book dedicated to the query optimizer that explains how it works, how it chooses certain plans over others etc.
Greg


ISQL Server Execution Plans

It does give a background of the query optimizer and goes into detail about execution plans. Don't know if it is the detail you are looking for. Gail is right in that the true "internals" of the query optimizer are Microsoft trade secrets probably and not available to public. I have never come across anything detailed, but Grant's e-book is a good start.

Shawn Melton
PS C:\>(Find-Me).TwitterURL
@wsmelton
PS C:\>(Find-Me).BlogURL
meltondba.wordpress.com
ducon
ducon
Old Hand
Old Hand (327 reputation)Old Hand (327 reputation)Old Hand (327 reputation)Old Hand (327 reputation)Old Hand (327 reputation)Old Hand (327 reputation)Old Hand (327 reputation)Old Hand (327 reputation)

Group: General Forum Members
Points: 327 Visits: 286
Hi,

Thanks for all the info, I will check these out. I'm pretty sure there's a fine line between giving enough information for people to make informed decisions and revealing your trade secrets as pointed out. However, I think it'd be pretty nice to know more about it as it may shape a lot in my indexing, data allocation strategies etc.

Greg
GilaMonster
GilaMonster
SSC Guru
SSC Guru (88K reputation)SSC Guru (88K reputation)SSC Guru (88K reputation)SSC Guru (88K reputation)SSC Guru (88K reputation)SSC Guru (88K reputation)SSC Guru (88K reputation)SSC Guru (88K reputation)

Group: General Forum Members
Points: 88266 Visits: 45277
There's enough information around for indexing strategies and the like. My blog's got some, all of the books I mention have lots on that. That's not what I'd call deep internals though.

Gail Shaw
Microsoft Certified Master: SQL Server, MVP, M.Sc (Comp Sci)
SQL In The Wild: Discussions on DB performance with occasional diversions into recoverability

We walk in the dark places no others will enter
We stand on the bridge and no one may pass


ducon
ducon
Old Hand
Old Hand (327 reputation)Old Hand (327 reputation)Old Hand (327 reputation)Old Hand (327 reputation)Old Hand (327 reputation)Old Hand (327 reputation)Old Hand (327 reputation)Old Hand (327 reputation)

Group: General Forum Members
Points: 327 Visits: 286
Yes, I agree with the fact that there is information online, but it's scattered all around and can be inconsistent between sources. That's what I want when I buy a book dedicated to a particular topic. I'm not looking into finding information that is impossible to find online.

Knowing how the query optimizer works will help shaping strategies regarding indexing etc. It's not necessarily deep internals but in my humble opinion:

1) it sure does not hurt (I always end up learning something new everytime I try something, even though I thought I knew how that worked)
2)I'm not sure how far you can get before hitting the "trade secret" wall in this topic
3) the definition of something "deep" is probably very relative to everyone's knowledge in this kind of area (and their needs as well)

I have enough database theory books in my shelves to understand how it's "supposed" to work from a maethematical point of view, but we've all seen cases where some queries are done in a certain way by the optimizer instead of the one we thought it ought to. The main drawbacks of these books is that they most of the time of course are RDBMS-independent, which is also their strength.

I'm pretty sure a book like the SQL Internals mentioned earlier is not entirely dedicated to that but I think that it can give some valuable information that may affect these decisions. Mind you, I haven't read it yet, nor do I own it.

Regards,

Greg
Grant Fritchey
Grant Fritchey
SSC-Forever
SSC-Forever (40K reputation)SSC-Forever (40K reputation)SSC-Forever (40K reputation)SSC-Forever (40K reputation)SSC-Forever (40K reputation)SSC-Forever (40K reputation)SSC-Forever (40K reputation)SSC-Forever (40K reputation)

Group: General Forum Members
Points: 40487 Visits: 32665
Tossing another penny down the well, I'd go with the Internals books. I'd especially focus on T-SQL Querying book. It has a couple of chapters on how the optimizer works before it gets into all the other stuff of writing queries.

----------------------------------------------------
The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood...
Theodore Roosevelt

The Scary DBA
Author of: SQL Server Query Performance Tuning and SQL Server Execution Plans
Product Evangelist for Red Gate Software
blake colson-405790
blake colson-405790
SSC Rookie
SSC Rookie (40 reputation)SSC Rookie (40 reputation)SSC Rookie (40 reputation)SSC Rookie (40 reputation)SSC Rookie (40 reputation)SSC Rookie (40 reputation)SSC Rookie (40 reputation)SSC Rookie (40 reputation)

Group: General Forum Members
Points: 40 Visits: 146
Check out Benjamin Nevarez if you're still pursuing this. I had several exchanges with him a while back about how calculations were derived by the optimizer. This knowledge helped me figure out expected execution times. My last exchange if you're interested: http://sqlblog.com/blogs/ben_nevarez/archive/2009/09/11/how-the-query-optimizer-uses-statistics-part-ii.aspx

He ended up writing a book: http://www.amazon.com/Inside-SQL-Server-Query-Optimizer/dp/1906434603/ref=sr_1_11?ie=UTF8&qid=1301970045&sr=8-11

Good luck !

-Blake a.k.a CQLBoy

CQLBoy
Grant Fritchey
Grant Fritchey
SSC-Forever
SSC-Forever (40K reputation)SSC-Forever (40K reputation)SSC-Forever (40K reputation)SSC-Forever (40K reputation)SSC-Forever (40K reputation)SSC-Forever (40K reputation)SSC-Forever (40K reputation)SSC-Forever (40K reputation)

Group: General Forum Members
Points: 40487 Visits: 32665
blake colson-405790 (6/17/2011)
Check out Benjamin Nevarez if you're still pursuing this. I had several exchanges with him a while back about how calculations were derived by the optimizer. This knowledge helped me figure out expected execution times. My last exchange if you're interested: http://sqlblog.com/blogs/ben_nevarez/archive/2009/09/11/how-the-query-optimizer-uses-statistics-part-ii.aspx

He ended up writing a book: http://www.amazon.com/Inside-SQL-Server-Query-Optimizer/dp/1906434603/ref=sr_1_11?ie=UTF8&qid=1301970045&sr=8-11

Good luck !

-Blake a.k.a CQLBoy


This thread is about 18 months old.

Benjamin's book is really excellent, and I'm saying that despite being the technical editor on it.

----------------------------------------------------
The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood...
Theodore Roosevelt

The Scary DBA
Author of: SQL Server Query Performance Tuning and SQL Server Execution Plans
Product Evangelist for Red Gate Software
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search