Unfortunately, it is not possible to display the label there. It displays all the variations and not separate attributes there. If you have two attributes, for example, the list will be shown as following:

Red, Medium
Red, Small
Red, Large
Blue, Medium
Blue, Small
and so on..