Milk gets trapped in my Avent breast pump's diaphragm

Make sure you hold the pump upright; as if you bend it milk will leak from the joints where the pump and bottle connect.