Hello,

The category page inherits the settings of the shop page. So you need to set the sidebar for the shop page in the Theme Settings > Shop > Sidebars and Titles. Then you need to add filter widgets to Appearance > Widgets > Shop page widget area. You can apply WOODMART WooCommerce Layered Nav which has the option to determine which categories it should be shown at.

Best Regards