The four men killed 63-year-old Terry Oldham after deciding his perverted past made him "fair game", a jury was told .
A murder investigation was launched when police found extensive groin injuries on Mr Oldham's battered body.
A post mortem revealed the assault with the broom handle had injured Mr Oldham internally causing poisoning which led to organ failure.
Aaron Mallen, 24, Michelle Casterton, 43, and Daniel Quinn, 23, all from Falmouth, Cornwall, and Richard Rosevear, 33, of St Austell, Cornwall, all deny murder.
Paul Dunkels, QC, told Truro Crown Court evidence did not show which defendant inflicted the fatal injury.
But he said it was the work of "more than one person" between March 28 and April 1 last year.
Mr Dunkels said: "He (Mr Oldman) was a small frail man in his 60s.
"He was routinely bullied and assaulted by them, they took his money and they stole his possessions.
"They knew he was a convicted sex offender so viewed him as fair game."
The court heard Mr Oldham had been convicted of indecently assaulting a girl aged under 14 in 1987 and was subject to a sexual offences prevention order as a result.
Mr Dunkels listed a series of "extensive injuries" to the victim's head, chest, abdomen, hands, arms and legs.
He said there were a large number of bruises to many parts of Mr Oldham's body of varying ages meaning he had been assaulted on numerous occasions.
He said: "Terry Oldham became increasingly isolated and depressed as the defendants took over his house and his life."
He said that none of the defendants had done anything to help the 5ft 2ins man, who weighed just 8.5 stone, following the assault despite the fact that it "must have been obvious that he was seriously ill".
The four were asked to give statements following the discovery of Mr Oldham's body.
Mr Dunkels said: "Each maintained that all was normal and Terry Oldham was well until he was found dead that morning.
"Mallen, Casterton and Quinn were laughing and said that Richard Rosevear woke them up and said that Terry was dead and they all went back to sleep before getting up to see what had happened.
"They said they kept poking him to see if he really was dead." Mr Dunkels added that Casterton, Mallen and
Quinn told a different story when talking to friends later that day.
The trial, expected to last up to five weeks, continues