The fact that the larger files do not have the problem suggests that this is a timing issue rather than a logic/coding issue.
You could try calling .Dispose() rather than .Close(). You could also try adding a Thread.Sleep(nnnn) to test the possibility of it being down to timing.