Comparisons of quiescent galaxies in the distant universe with their counterparts in the local volume show that the structure of galaxies continues to change well after they stop forming stars. Very compact galaxies, that have the stellar mass of the Milky Way and the size of the Milky Way bulge, seem to disappear between redshifts z~2 and z~0. The average size of quiescent population increases by a factor of a few over the same redshift interval. In contrast to the existing surveys of high-redshift universe, at intermediate redshift (0.2 < z < 1) we trace quiescent population by combining high-resolution imaging and dense redshift surveys that map large areas of the sky. We utilize these spectro-photometric datasets as a missing link that completes the picture of galaxy size growth in the quiescent phase. I will describe how information-rich surveys of the universe at intermediate redshift enable us to test models of galaxy mass assembly by accurately characterizing the correlations between structural evolution of galaxies, their stellar population properties, and the environment they inhabit.