Tableau year-to-year change in grand total

New to Tableau and having some problems with a calculation I thought would be easy.

I have data with one value per month running across a few years. I would simply like to display a table with the change from month to month in percent and the change from start of year to end of year in percent as a grand total.

The input data looks like this:

And I would like to create a table like this:

The month to month change is easy enough using a table calculation with Percent difference from and across then down, but the yearly change has me frustrated by now as I keep getting weird values. I tried to create a calculated field using offsets in the table, but of course it doesn't work when applied in grand total using the 'Size()' trick.

Any tips?