I’ve stolen that expression from Darcy at egardengo and I hope he won’t sue me for copyright infringement. The principle goes like this: the more valuable space a plant occupies in your garden, the more it has to perform for you and the less you should tolerate its shortcomings.
For example: Here is the exquisite Rosemary Harkness rose:
I’ve enjoyed her blooms for over 10 years. But at her very best she bloomed little, and at her worst she had gangly canes and was covered with black spot. Never an attractive shrub. And while I didn’t need the space she occupied, I put up with her deficiencies for the sake of the few but beautiful blooms.
No more. I wanted to plant a ‘Bluebird’ Hibiscus syriacus (Hardy Hibiscus) that my neighbour was discarding, and the best place for it was right about where Rosemary was. So Rosemary is no more. Shovel pruned, as they say. There are lots of other, better performing apricot-shaded roses, so why have one that has never earned her keep.
So look around at your garden objectively to see if the plants are fulfilling their designated function. If not, and they don’t “put a smile on your face”, consider replacing them with better choices.