Training a Keras multi-task model with subsets of labels in each training example

Keras allows you to freeze layers, which helps with fine-tuning models. However, if you have a multi-task problem where each training example only contains a subset of the labels (e.g. only one label) we need to unfreeze a single path through the model, to avoid updating weights for heads that we don't have a label for in this example. How can this be achieved using Keras?