The leaves and tender stem growth of the upper 10 cm (4 inches), along with the flowers and ripening seeds, are harvested for medicinal purposes. The berries are harvested by gently rubbing the berries loose from the stem. The leaves, flowers, and/or berries may be consumed as a decoction, traditional tincture, cider vinegar tincture, syrup, elixir, or simply eaten straight off the plant as a medicinal food. A popular way of taking Vitex is on awakening as a simple 1:1 fluid extract, which is said to interact with hormonal circadian rhythms most effectively.

The berries are considered a tonic herb for both the male and female reproductive systems. The leaves are believed to have the same effect but to a lesser degree.

In ancient times it was believed to be an aphrodisiac, hence the name chaste tree. Pliny, in his Historia Naturalis, reports the use of stems and leaves of this plant by women as bedding "to cool the heat of lust" during the time of the Thesmophoria, when Athenian women left their husbands' beds to remain ritually chaste. At the end of the thirteenth century John Trevisa reports of it "the herbe agnus-castus is always grene, and the flower thereof is namely called Agnus Castus, for wyth smel and vse it maketh men chaste as a lombe". Chaucer, in "The Flower and the Leaf," refers to it as an attribute of the chaste Diana, and in the 16th century the English herbalist William Turner reports the same an aphrodisiac properties of the seed, both fried and not fried. More recently, this plant has been called monk's pepper in the thought that it was used as anti-libido medicine by monks to aid their attempts to remain chaste. There are disputed accounts regarding its actual action on libido, with some claims that it is an aphrodisiac and others that it is aphrodisiac. Because of the complex mechanism of action it can be probably both, depending on concentration of the extract and physiologic variables