Julio Acevedo, 44, was found outside a convenience store in Bethlehem.
Police say he was travelling at twice the speed limit when he hit a taxi carrying Nachman and Raizy Glauber, both 21, on Sunday in Brooklyn.
Both were pronounced dead at hospital and the baby died a day after being delivered by caesarean section.
The couple were part of the borough of Brooklyn’s close-knit Orthodox Jewish community.
More than 1,000 people turned out to mourn the couple at a funeral at Congregation Yetev Lev D’Satmar synagogue.
The crash happened just after midnight on Sunday, a few blocks from the Glaubers’ home in the Williamsburg area of Brooklyn.
Raizy Glauber was seven months’ pregnant and had not been feeling well, the family said, so the couple called a taxi to take them to hospital. Their cab collided with a BMW car.
The four-pound (1.8kg) baby boy was buried with his parents on Monday. He died of extreme prematurity, according to the city medical examiner’s office.
Acevedo was arrested by a fugitive unit of the New York Police Department as he walked towards the store shortly after 17:00 (22:00 GMT) on Wednesday, the New York Daily News reported.
“Plans are underway now to extradite him to New York in connection with the car crash that took the lives of the young couple and their child,” police spokesman Paul Browne said.
The 44-year-old had told local news outlets that he intended to turn himself in on Tuesday.
He was arrested last month on a charge of driving while under the influence, and that case was still pending on Sunday. Acevedo has also served about a decade in prison for manslaughter.