Navigate to Woocommerce > Settings > Product tab > Advanced tab and disable Use the product attributes lookup table for catalog filtering and check the issue: https://gyazo.com/637a2e9265c643fd5cb960587d543e41
No matter if I select to SHOW 9, 12, or 18 products in one page, there are always 4 products per page in the first 3 pages + 3 products in the last (4th) page.