If you're looking to stay in the NYC region but don't need to be actually in NYC: Boston and Philadelphia have great startup communities and great transportation options.
Closer in to NYC: Stamford (lots of empty office space), Hartford (now semi–accessible to NYC by rail, plus an international airport), White Plains (easily accessible to NYC plus regional airport). NJ has a lot of office space along I–95 once you get out of the NYC metro area, but am uncertain on the rent.
Closer in to NYC: Stamford (lots of empty office space), Hartford (now semi–accessible to NYC by rail, plus an international airport), White Plains (easily accessible to NYC plus regional airport). NJ has a lot of office space along I–95 once you get out of the NYC metro area, but am uncertain on the rent.