How to reduce crawler load on Woodmart filters
-
Hi,
we are using Woodmart on a WooCommerce store with a fairly large number of product filters.
Our problem is that when bots, scanners, or fake browser crawlers start crawling filtered URLs, they quickly generate a huge number of filter combinations. This creates many expensive archive requests and can noticeably slow down or even overload the site.
We want normal users to keep using filters, but we want to reduce the risk of scanners generating endless filter combinations and slowing down the server.
Do you have any recommended Woodmart settings or best practices to reduce this kind of crawler load while keeping filters usable for real users?
Thanks.
Attachments:
You must be
logged in to view attached files.
Hi there,
This is a common issue. Many bots crawl filter URLs, which are usually not cached and can generate heavy database queries, increasing server load.
These filtered URLs are generated by WooCommerce rather than the theme, so WoodMart does not have a setting that can control or limit crawler access to them.
The usual solution is to configure a firewall service such as Cloudflare to detect and block bots from accessing filter URL patterns. This can significantly reduce unnecessary requests before they reach your server, while regular visitors can continue using the product filters normally.
Kind regards,
XTemos Studio
Hi there,
In addition to the recommendation above, please refer to our article with detailed instructions for reducing bot traffic to WoodMart product filter URLs using Cloudflare:
https://xtemos.com/blog/how-to-stop-bot-traffic-to-woodmart-product-filters-with-cloudflare/
Kind regards,
XTemos Studio
.
