Influence of the data generating distribution
To see this, first we have to mention that, neither by using Batch gradient descent (using the whole dataset to compute the gradient) nor using mini-batch gradient descent, we are computing the true (exact) value of the gradient.
To compute the true value of the gradient we would have to use the set of all possible values of the features, $x$, (and thereby the outputs $y$).
More formally, and refering to the quantity that we want to minimize as the expected value of the per-example loss function ($J(x,y,\theta)$, where $\theta$ are the parameters) w.r.t all the possible $x,y$ values, the true gradient $g$ is given by:
$$g = \frac{\partial}{\partial \theta}\mathbb{E}_{x,y\sim p_{data}}(J(x,y,\theta)) $$
And if we assume certain conditions We have that:
$$g = \mathbb{E}_{x,y\sim p_{data}}\left(\frac{\partial}{\partial \theta}J(x,y,\theta)\right) $$
Where $p_{data}$ is the data generating distribution (the distribution from which the values of $x$ and $y$ are drawn). However, this data generating distribution is usually unkown. We just know the dataset that we are given.
Because of this, to update the parameters using all the information given (the training set), we instead use the empirical ditribution defined by the training data ($\hat{p}_{data}$) which puts a probability of $1/m$ on each of the $m$ samples $(x^{(1)}, y^{(1)}), \,(x^{(2)}, y^{(2)}),\,...\,,(x^{(m)}, y^{(m)})$ of the dataset. So the gradient is approximated by:
$$ \begin{aligned}
\hat{g}&=\frac{\partial}{\partial \theta}\mathbb{E}_{x,y\sim \hat{p}_{data}}(J(x,y,\theta))\\&=\frac{\partial}{\partial \theta}\left(\sum_{i=1}^m \frac{1}{m}J_i(x^{(i)},y^{(i)},\theta)\right)\\
&= \frac{1}{m}\sum_{i=1}^m\frac{\partial }{\partial \theta}J_i(x^{(i)},y^{(i)},\theta)
\end{aligned}
$$
Ending up with the Batch gradient descent.
But what happens with mini-batches?
By using mini-bath updates, we are continuously seeing new data (assuming that we compute just one epoch). So in this case, using mini-batches, we are using the data generating distribution.
This means that on each mini-batch update, by sampling this data generating distribution, we end up with an estimation ($\hat{g}$) of the true gradient ($g$) that is unbiased i.e. $\mathbb{E}_{x,y\sim p_{data}}(\hat{g})=g$. To see this, and considering $\text{s-sized}$ mini-batches:
$$\begin{aligned}
\mathbb{E}_{x,y\sim p_{data}}(\hat{g})&=\mathbb{E}_{x,y\sim p_{data}}\left(\frac{g^{(1)}+...+g^{(s)}}{s}\right)\\
&=\frac{1}{s}(\mathbb{E}_{x,y\sim p_{data}}(g^{(1)}+...+g^{(s)}))\\
&=\frac{1}{s}s\,\,g=g
\end{aligned}
$$
Thereby, making succesive mini-batch updates we would be tending in average (as shown by $\mathbb{E}_{x,y\sim p_{data}}(\hat{g})$) to updating our parameters with the true value of the gradient. And this is what I think the authors refer to in the quote of the question.
Great references:
Deep Learning book, Ian Goodfellow et. al Chapter 8.1
Answers from here