WooCommerce Sort by Popularity Not Working in Product Filters

When I use product filters for my WooCommerce shop, they are not sorted by popularity (sales) as I have chosen from settings. Instead, it shows products sorted by date, with the most recently posted at the top. This issue only occurs when I use WooCommerce product filters. If I don't use any filters, it sorts correctly by popularity.

How can I sort products by popularity while using product filters for the product's attributes like colour, size, etc.? Any help on this will be appreciated.