Baha Abu Al Ata, a senior commander of Islamic Jihad, and his wife Asmaa died in an Israeli airstrike on their home in the Gaza Strip on Tuesday, officials said, Efe news reported.
At least 70 rockets were fired at Israel from Gaza after Abu Al Ata's killing, leaving 17 Israelis injured.
Israel carried out further strikes that left two Palestin Mohammed Attiyah Musleh Hammouda, 20, and Zaki Ghanama, 25, dead.
They were killed in a series of Israeli bombings which, according to military sources, targeted Islamic Jihad positions including rocket-launching units in the northern part of the area, a military training complex, and sites where armament was manufactured and stored.
Israeli officials said that both men were militants.
Ashraf al Qedra, spokesman for the Gaza Ministry of Health, said another 25 people were injured in the attacks.
Israeli Prime Minister Benjamin Netanyahu said in a press conference in Tel Aviv: "Israel is not interested in escalation, but will do everything necessary to defend ourselves.
"And I say in advance: This could take time. Patience and composure are required."
The Israeli Defence Force said it carried out a "surgical strike" on a property in Gaza, targeting Abu Al Ata.
"Al Ata was directly responsible for hundreds of terror attacks against Israeli citizens and soldiers," the IDF said.
Jonathan Conricus, IDF spokesman, said the Islamic Jihad group had been planning an imminent attack.
Alleged members of the Palestinian Islamist group responded to the Israeli military action with a barrage of rockets launched from the Gaza Strip into Israeli territory.
Anti-aircraft sirens could be heard in southern and central Israel, forcing the closure of educational centres and other security measures.
The IDF said the militants had launched approximately 50 rockets, roughly 20 of which had been intercepted by Israeli's Iron Dome defence system.
Conricus said: "Alarms have now sounded in Modiin, an Israeli city of 90K, which is more than 70km from Gaza, in what appears to be a serious escalation by Islamic Jihad.
"Iron Dome has so far performed well. IDF remains ready to defend our civil against terror threats."
In a statement, Islamic Jihad said Israel had crossed "a red line".
In a similar incident on Tuesday morning another commander of Islamic Jihad was targeted in Damascus.
The militant group has accused Israel of carrying out that strike too, although officials have yet to comment on the situation.
( With inputs from IANS )