Michael Cephas is charged with raping a four-year-old girl and her 12-year-old sister. Cephas lived on Chestnut Avenue in Newport News.
NewsChannel 3 spoke with a neighbor who didn't want her face shown, but says she knew something was up.
"I don't know, Mike is just strange," the neighbor said. "He's always luring little girls over there."
Police say Cephas raped the four-year-old in late January.
Cephas is the boyfriend of the victim's mother.
The victim's 12-year-old sister told investigators she had been sexually abused since she was nine.
"I don't care what time of day or night, she was with him."
The neighbor says she told her granddaughter to never go over to Cephas' house.
She adds Cephas had a lot of women cycle in and out of the house with their children and she warned them about the 57-year-old.
"We were trying to tell the young ladies that kept moving in and moving out, 'do you know about him? oh no child, he's not going to do anything to my child,'" the neighbor said. "But you know about him. I think the last three found out; the first two didn't know until they really found out and then they moved."
As prevalent as the neighbor says the abuse was, experts like Brenda Garrett with Kids Priority One say too often the victims are too afraid to tell anyone.
"Childhood sexual abuse is terribly under-reported, and most of the time, it is someone the child knows," Garrett said.
Garrett says 90 percent of the time the abuser is someone who is taking care of the child.
Police aren't saying how often Cephas watched the two victims.
But the neighbor says Cephas was always hanging around young children.
"It's strange for a man to be attached to a kid that's not yours," the neighbor said. "I don't trust men like that."