There is nothing wrong with a tall development there as long as it interacts with the street well. Empire State Bldg. is surrounded by 5-20 story buildings but you have to look up to notice anything unusual.

The most important thing is a well executed, well constructed development ASAP, and an end to the unsettling, gaping hole downtown.