Hm. On thinking about it a bit more, I guess I could use a subquery:

SELECT

A.ID

FROM TableA A

WHERE

(

CASE

WHEN @TableB_ID IS NOT NULL THEN

CASE

WHEN EXISTS (SELECT 1 FROM LinkTable WHERE TableA_ID = A.ID AND TableB_ID = @TableB_ID) THEN 1

ELSE 0

END

ELSE 1

END

) = 1

Also very ugly though...

And one other solution I've come up with:

SELECT

A.ID

FROM TableA A

LEFT JOIN LinkTable ONTableA_ID = A.ID

WHERE (CASE WHEN @TableB_ID IS NOT NULL THEN @TableB_ID ELSE ISNULL(TableB_ID, -1) END) = ISNULL(TableB_ID, -1)

This one of course is reliant upon using a number for the ISNULL function which is guaranteed to never occur. Sure, it might work in some cases, but it still feels more like a hack than anything else.