Search billions of records from multiple tables

  • If you can give us table / index DDL, sample data (we don't want your real data, feel free to fake it), and a sample query, we can help you code your indexes and query. Anything we give you will probably need to be tweaked, but it will be better than nothing.

    Brandie Tarvin, MCITP Database AdministratorLiveJournal Blog: http://brandietarvin.livejournal.com/[/url]On LinkedIn!, Google+, and Twitter.Freelance Writer: ShadowrunLatchkeys: Nevermore, Latchkeys: The Bootleg War, and Latchkeys: Roscoes in the Night are now available on Nook and Kindle.

  • Artefact(ArtefactId,EntityTypeId,UserId,GroupId,ArtefactAttachmentId,Name,Summary,Description,TenantId...etc)

    User(Userid,OrganizationID,TenantID,UserPassword,SecretQuestion,SecretAnswer,FirstName,MiddleName,LastName,Gender,DateOfBirth,DateOfJoining,Email etc...)

    Rows (Artefact)

    :(1,1,1,1,1,'Title','Summary','Desc',1,..etc)

    Rows(User)

    :(1,1,1,'password','Secrete Question','Secrete Ans','FirstName', 'LastName','MiddleName','LastName','M','14-04-2012','3-03-2010','email@email.com' etc...)

  • Sorry, but that's not detailed enough. We need CREATE TABLE statements (that's what DDL is) and INSERT statements with a longer list of sample data. One line per table is not enough data either. We need more than that in order to do proper testing.

    Please provide that and we can do our best to give you more complete answers.

    Brandie Tarvin, MCITP Database AdministratorLiveJournal Blog: http://brandietarvin.livejournal.com/[/url]On LinkedIn!, Google+, and Twitter.Freelance Writer: ShadowrunLatchkeys: Nevermore, Latchkeys: The Bootleg War, and Latchkeys: Roscoes in the Night are now available on Nook and Kindle.

  • >>EDIT: And if none of what I said makes sense, it's time to bring in an outside consultant who can help you with this. This is too large of a thing to easily condense down into a few posts on a forum.

    That is the best advice given thus far on this thread.

    Best,
    Kevin G. Boles
    SQL Server Consultant
    SQL MVP 2007-2012
    TheSQLGuru on googles mail service

  • You mentioned that you've been using Text Search engine in the past, so this advice assumes that at least some of your routinely searched terms are contained in text columns. That's an inefficient method of working with billions of rows. I would suggest that all routinely searched terms be split out into properly normalized and indexed columns.

    "Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho

Viewing 5 posts - 16 through 20 (of 20 total)

You must be logged in to reply to this topic. Login to reply