This way is a few hundred times faster (on large datasets)...NOTE:the details are missing but it still identifies the key holes in the sequence, except beiging and end which are obvious...?

SELECT s1.SeqNumber

FROM #SequenceTable s1

LEFT JOIN #SequenceTable s2

ON s1.SeqNumber = s2.SeqNumber -1

WHERE s2.SeqNumber IS NULL