This doesn't really add much value to Peter's version, but just for my own amusement:

DECLARE @MissingNumbers

TABLE (n INTEGER NOT NULL);

INSERT @MissingNumbers (n)

VALUES (1),(2),(4),(5),(7),(8),(11),(12),(13),(15),(17),(19),(20);

SELECT N.n

FROM dbo.Numbers((SELECT MAX(n) FROM @MissingNumbers)) N

EXCEPT SELECT n FROM @MissingNumbers;

(Using my previously posted function)