We tested with the default theme and it also takes sometimes 4-6 seconds to load the page. There is definitely something wrong either with your server configuration or with WordPress/WooCommerce in general. Here are screenshots.

https://gyazo.com/228268ca2ecf20972701af6ad661fc8f
https://gyazo.com/8a27fc77fdee464b6f4de5f6f0c4ba7a