Does Bias always decrease when Complexity increase?

(I'm just starting learning about ML stuff and so please don't be rude if the following question is to stupid or totally wrong)

I'm reading about Bias-Variance Trade off and I don't understand the (probably) most important part: why its a tradeoff?

I totally get that the generalization error can be decomposed in 3 parts, an irreducible error due to the noise in our data, a Bias term and a Variance term. In some cases I have a model with low Variance (very simple model) and as a consequence high Bias (im not targeting at all the true function behind our data). And also I totally get that increasing the complexity of the model can decrease the Bias of our model (on average the prediction of my model will be correct) at the price of increasing its Variance (the model generated will vary a lot depending on the training data set).

So my question is: does Bias always decrease when Variance increase??. And my answer to that question is: No.

Suppose that for example my data are generated by a quadratic function ($f(x)=x^2$) plus some random noise $y = f(x) + \epsilon$. The model I'm fitting is a polynomial regression with K = degree of my polynomial: $\hat{f}(x) = x^k + ... $. Now I start from K=0, very simple model with low Variance but of course has a lot of Bias since I'm totally missing the true function behind my data, and in fact in the end I get a pretty big generalization error, in this case mainly because of the Bias of my model.

Now if I jump directly to K = 20 my model will have a lot of Variance, i.e. it will change a lot depending of what training set I'm using to build it, but does have low Bias? I don't think so, since also in this case I'm totally missing the true function behind my data, the model that I'm building is totally different from the true function.

So this is my doubt, because what I perceived reading article online is that blindly increasing the Variance of the model will magically decrease the Bias, and I think that this is only true if I'm increasing the Variance of the model in right direction. If I'm totally missing the true function of my data I will end up with a lot of Variance but at the same time a lot of Bias.

EDIT

Ok I think I got it now. The thing I was missing is that increasing the complexity of our model means to add more parameters to the previous model. In this way every model incorporate the previous version of the model, and in this way the Bias can only decrease.

Using my example, the Bias of a model that use polynomials up to K = 4 degree cannot be worse than the Bias of one that only use polynomials up to K=2, since in K=4 we have ALSO the polynomials with degree equal K=2.