One entry, ten minutes above a village that has been photographed to death, and it is entirely worth the ramparts.
Avoid August entirely. I mean this more than I mean most things.
Late September is the answer. The light has stopped shouting, the coast has emptied, and the pines still hold the heat out of the courtyards. In August the road up is a queue and the whole argument of the place is lost in it.
It is uphill from the village and further on foot than the map suggests, in the sort of shoes people bring to the Côte d'Azur. Walk it anyway; arriving through the trees is half of what the architect was doing.
Give the outdoor courts as long as the galleries. Most people do the rooms properly and the sculpture at a trot, which is exactly the wrong way round here.