Chrome on mobile renders elements in wrong position after URL bar is shown/hidden

I have a fullscreen modal (position: fixed, top: 0, bottom: 0, ...) on my page. Scaling and scrolling doesn't cause any problems and everything works as expected. However, sometimes (I haven't really figured out the circumstances yet), Chrome will not re-render the modal correctly after scrolling up or down when hiding/showing the address bar.

This is the page before scrolling when the URL bar is hidden. The header is rendered where it should be:

After scrolling down, Chrome shows the URL bar (the tiles in this case) and knows where the content of the modal should be, but it doesn't render it in the new position. This also applies to the body of the modal.

Anyone have an idea how to solve this? Thanks!