B'Tselem said detailed research with careful cross-checking showed 1,387 Palestinians died, over half of them civilians and 252 of them children.
This contradicts an Israeli army report stating fewer than 300 civilians died in fighting in December and January.
Israel launched the assault to halt rocket attacks from Hamas-run Gaza.
The overall B'Tselem total broadly tallies with the official Palestinian death toll and the findings of other non-governmental organisations, although the proportion of civilians it identifies is lower.
The group says the extent of civilian deaths does not prove, in itself, that Israel violated the laws of war.
However, it says it raises grave concerns about the military's behaviour when taken in the context of "numerous testimonies" from troops and Palestinians.
Amnesty International has already accused Israel of committing war crimes during its offensive.
The Israeli army has admitted "rare mishaps" during the campaign but denies troops violated international humanitarian law.
'Serious introspection'
B'Tselem said the findings had been compiled during months of research, including visits to the families of those killed.
It said it was unable to compare its figures with the official Israeli ones because the military refused to provide its list of fatalities.
The group said the results should compel the Israeli government to launch an independent investigation into its three-week offensive.
Earlier this year the Israeli army said that 1,166 Gazans were killed in the conflict, a quarter of whom were civilians.
Its figures indicated that the toll included 709 militants from Hamas and other groups, and 295 non-combatants.
According to B'Tselem, 1,387 Palestinians were killed by the Israeli military, including 773 civilians, 330 combatants and 248 civilian police - whom Israeli officials classify as militants.
B'Tselem has counted 252 children under the age of 16 who were killed - the military puts that figure at 89 - and 109 women over 18.
"The extremely heavy civilian casualties and the massive damage to civilian property require serious introspection on the part of Israeli society," B'Tselem said, adding that it considered the army's internal probe insufficient.
The group acknowledged the challenges of combat in the densely crowded Gaza Strip, and criticised the "illegal and immoral actions" by Palestinian militants accused of hiding among the civilian population.
But this "cannot legitimise such extensive harm to civilians by a state committed to the rule of law", B'Tselem added.
The Israeli military had no immediate comment on the report, but has previously rejected such criticism.
It has said the aim of the campaign "was target the Hamas terror organisation and not citizens of the Gaza Strip".
Front Page Features