"BTW, in the 2010 census, the downtown/midtown census tracts actually saw a sizable net loss of people."

Not true. If you look at the NY Times Census tool for downtown and midtown tracts, you see a number of tracts that saw significant percentage increases in population. You also saw some tracts lose significant population percentage-wise. It's a mixed bag but the numbers don't back up a claim of a significant population loss in those areas.