Sorry.. I didnt read the post completely, so here is the modified query,

SET STATISTICS PROFILE ON

SELECT aa.* FROM @mySampleTable aa

INNER JOIN (SELECT a.CaseId, a.latestUpd FROM (SELECT CASEID, COUNT(*) CCnt, MAX(CreatedDate) LatestUpd FROM @mySampleTable GROUP BY CASEID) A OUTER APPLY (SELECT TOP 1 CASEID, STUDENTID, COUNT(*) SCnt FROM @mySampleTable B WHERE A.cASEiD = B.cASEiD GROUP BY CASEID, STUDENTID) AS BWHERE A.CCNT <> b.scnt) bb on aa.CaseId = bb.CaseId

ORDER BY latestUpd desc, aa.CaseId

I tried to avoid using Max and Min function on Char column as this may back fire with huge data. and used Outer Apply instead with Counting No Of Rows against each CaseID and StudentId