When to switch to new zoom-level using deepZoom

I am trying to write an renderer for a very large image, and I've generated a set of image tiles using DeepZoom, with an overlap of 2 and a tilesize of 1024 (original image is 16200x8100).

Currently, my renderer switches to the next zoom level when the scale-transformation doubles, or switches out to when it halves. However, when the next zoom-level is switched to, the camera is offset by some pixels to the left (even if the offset transform has not changed), as though the images are bigger. How can I compensate for this or is there some particular time at which I should switch to the next zoom level?