When a nest box stays empty
An empty box is not a failed experiment. Review its setting over a season before deciding what to change.

What does empty mean?
Not seeing a bird enter is different from knowing a box has never been visited. A short look each evening may miss daytime prospecting, while a box can also be used as a roost without a nest being built. Keep your conclusions as modest as the observations allow.
You do not need to open the box to settle every uncertainty. Watching from a normal indoor position over several days can reveal patterns without repeated close approaches. Note the time, weather and what you actually saw, rather than filling a diary with assumptions about why the birds stayed away.
Review the obvious constraints
Start with the intended species. Is the chamber suitable, and do those birds occur nearby? Then look at the approach, exposure and activity around the site. A branch may have grown across the entrance since installation. A feeder hung later might now bring constant traffic to the same corner.
Consider the whole season. The position that seemed shaded in winter could receive afternoon sun after a neighbouring tree was removed. A quiet wall may become the setting for a frequently used barbecue in spring. These changes are more useful clues than the colour of the box.
There may be no visible fault
Territory boundaries and alternative natural cavities can influence nesting decisions in ways that are hard to recognise from the garden. A correctly chosen box is an opportunity, not a contract with a pair of birds. Immediate occupancy is not the only measure of whether installing it was worthwhile.
Avoid adding food inside, lining it with household fibres or repeatedly enlarging the entrance to tempt occupants. Those changes can create new problems without addressing the reason it was unused. If the structure is sound and the setting sensible, leaving it in place may be the best decision.
Make one considered change later
After several unused seasons, a different position may be worth considering. Plan the move for a time when the box is definitely inactive and there are no birds using it to roost. Evaluate the new location as carefully as the old one, including safe access and exposure.
Keep a note of what changed. Moving the box, replacing its front and clearing nearby shrubs all at once makes it impossible to know which change mattered. More importantly, it may turn a quiet site into a repeatedly disturbed one just as birds begin to consider it.