The Egypt-brokered ceasefire reportedly began at 22:00 (19:00 GMT), but within minutes there were exchanges of fire.
At least 33 Palestinians have been killed since Tuesday in Gaza, where Israel says it has struck PIJ targets.
Palestinian rocket fire into Israel has killed two people, one Israeli and a Palestinian working in the country.
A Palestinian source familiar with the ceasefire negotiations told the BBC that Israel and PIJ had agreed a ceasefire to end five days of hostilities.
The source added that the agreement included a commitment by Israel to stop so-called "targeted assassinations". Israel was, meanwhile, said to have demanded a complete halt to rocket fire by PIJ at its civilians.
Egyptian officials called on both sides to adhere to the agreement, but the Israeli military said militants fired rockets at over the cities and towns in southern Israel not long after the ceasefire was meant to start.
The military then announced that it had carried out air strikes in response, hitting two PIJ rocket launchers in Gaza.
Israel began its military operation in Gaza before dawn on Tuesday, killing three leaders of the militant group in their homes as well as at least 10 civilians, including relatives and neighbours of the men.
PIJ fighters then fired barrages of rockets at southern and central Israel, which they said were intended to avenge the dead.
Over the past five days, Israel has carried out air strikes across Gaza, killing three more senior figures in PIJ and targeting what it said were the group's rocket launch sites and command centres.