Matrix calculation (in R) - least squares

I have a matrix in R as per the following:

(72 -- 90 -- 60)
(60 83 -- -- --)
(50 -- 77 83 --)
(-- 40 -- 84 65)

Reproducible as follows:

B = matrix(c(72,60,50,NA,NA,83,NA,40,90,NA,77,NA,NA,NA,83,84,60,NA,NA,65),nrow=4,ncol=5) 

It is a sparse matrix in the sense that not all elements are populated. What i want to do is find an adjustment matrix (or vector, that is added to each column) so that the average of the rows is equal to the average of columns. The answer is:

(13.04, 10.93, -14.68, -22.70, 5.90)

I computed this iteratively and is also rounded to 2dp so doesn't quite equate, but, for example, it should be the case that if you add the adjustments to the original matrix:

A = t(t(B)+c(13.04,10.93,-14.68,-21.70,5.9))    

then the average of the row averages should equal the average of the column averages:

mean(colMeans(A, na.rm = TRUE, dims = 1))
mean(rowMeans(A, na.rm = TRUE, dims = 1))

However, I think this is a least squares problem and can be solved algebraicly but i don't know how to do it or how to set it out. The question is, is there a way to find the adjustment algebraicly using matrix calculation rather than iteratively. Can anyone help?