The fatalities also included police officer Joseph Seals, a member of the municipal force since 2006, Efe news reported.
"This is an investigation that will take weeks, maybe months," Jersey City police chief Mike Kelly told a press conference on Tuesday, pointing to the challenge of piecing together events that unfolded over the course of hours at multiple locations.
Preliminary reports indicate that Seals was killed "while trying to interdict the bad guys," Kelly said of the 39-year-old father of five.
Law enforcement sources said that the incident began after Seals, a detective working with the gun squad, and another officer spotted a van suspected of being linked to a homicide last weekend in Bayonne, New Jersey.
As the police approached the vehicle, one suspect emerged and fired shots, hitting Seals in the head and the other officer in the shoulder.
The suspects then drove off and holed up inside the Jersey City Kosher Supermarket, located in the city's mainly Jewish Greenville neighborhood.
Police followed the suspects to the store and a standoff ensued.
"Our officers were under fire for hours," Kelly said, adding that authorities believe the three civil slain inside the store were killed by the "bad guys."
Despite the location of the shooting, officials say the violence did not appear to have connection to terrorism or to have been motivated by hatred toward Jews.
Two police officers wounded in the incident, Ray Sanchez and Maruela Fernandez, were treated at Jersey City Medical Center and released.
All the municipal schools were put on lockdown.
Police from seven other New Jersey departments and from New York City, just across the Hudson River, came to the aid of their colleagues in Jersey City, along with FBI agents based in nearby Newark.
"The police department was immediately on the scene interjecting," Jersey City Public Safety Director James Shea said. "Most active shooters end when the police arrive. In this case, one of our officers gave his life immediately trying to prevent what subsequently occurred. Two more officers were shot trying to prevent it and the Jersey City Police Department continued to respond and engage those people to minimize any possible of damage."
( With inputs from IANS )