Baby taken to hospital with skull fracture 'days after father seen shaking him'
Social services staff had begun an investigation when alerted, said Judge Ross Duggan.
Medical checks showed no sign of harm and the little boy's father had denied behaving "as alleged".
Judge Duggan had been asked to make decisions about where the little boy should live.
He concluded that the youngster should be taken from his parents' care and placed for adoption.
The judge said both the little boy's parents had learning difficulties.
His father had autism traits and his mother an "autistic spectrum condition".
The youngster had been living with his parents and his mother's parents.
His grandparents had given his parents "guidance and help" and most nights his grandmother had taken charge.
Judge Duggan concluded that the little boy had suffered a fractured skull when "accidentally dropped" by one of his four carers.
He concluded that the two arm fractures had been "inflicted... in frustration" by one of the little boy's parents.
Evidence pointed to a "momentary loss of control", the judge said.
Judge Duggan said the family involved could not be identified.
But he said social services bosses at Lancashire County Council had asked him to make decisions about the little boy's future.
The little boy's parents and grandparents had denied causing harm.
They said injuries had been caused by hospital doctors or during hospital treatment.
The judge indicated that the little boy had been temporarily placed with foster carers after being hurt - pending decisions on his long-term future.
Judge Duggan said he had heard evidence from a number of people at the family court hearing - including the neighbour who said he had seen the baby being shaken by the little boy's father.
"Four days before the hospital admission social services received a report from a neighbour about the father's activities during the previous day," said the judge.
"The neighbour told me that from the kitchen window he saw the father trying to calm a crying baby but this involved shaking and what he described as a push-throw which took the baby out of his field of vision.
"The neighbour explained that what he saw made him angry.
"He felt he had to intervene.
"He went outside and shouted to other members of the father's family, who clearly had not seen what he had seen."
Judge Duggan added: "His wife contacted social services and investigations ensued. The neighbour's observations were challenged but I accept the overall accuracy of what he says."
The judge said a "medical examination, albeit without x-rays, showed no sign of harm" and went on: "The episode only directly involved the father, and he denies that he acted as alleged."