When I enable the maintenance mode page, it appears for anonymous users, however users that were already signed in before the maintenance mode page was enabled, can continue to browse the website while maintenance mode is enabled.
How can I set it up so that when maintenance mode page is enabled, only administrator users can continue to access the site?
You need to create a page with the Maintenance mode content, set the Maintenance template in the page settings, and enable the Maintenance mode in the Theme Settings > Maintenance mode.
Screenshot for clarification: https://ibb.co/bdHQjgJ
I have already set that up. As mentioned, the maintenance mode page is showing for anonymous users. This issue is, users that are already signed in before the maintenance mode page is enabled, can continue to browse the website while maintenance mode is enabled.