(CNN) — On the tough streets of Compton in the late 1980s, a crowd of school children looked on as an imposing middle-aged man berated two young black girls.
The city, in Los Angeles County, was riven with internecine gang warfare at the time, and verbal conflict could quickly escalate to violence.
Yet the man doing the shouting, and the girls on the receiving end of his invective, were not acting out one of Compton’s then common tragic scenes.
For a start, the schoolkids had been bused in specifically to witness the event at the local tennis courts.
This was Richard Williams doing his worst to toughen up his beloved daughters Venus and Serena, who in just a few years would begin their domination of women’s tennis.
“In order to be successful you must prepare for the unexpected,” Williams recalls in an interview with CNN’s Open Court. “Criticism can bring the best out of you.”
On one occasion, he says local gang members tried to intervene, having been so unsettled by what they were seeing. It was “everything that white people shouted,” the 73-year-old explains.
“When (they) came to me and said, ‘You can’t talk to Venus in that way … I said, ‘Watch out. I’m going to do what I want to do.’
“Criticism is one of the greatest things, I think, that we’ve been trained to live through.”
Richard Williams was no stranger to racism. Growing up in Shreveport, Louisiana, in the 1940s and ’50s, he says he witnessed a friend being lynched.
Another died after being run over by a white woman who claimed it was the victim’s fault. “There was no investigation, there was no police car,” he says.
“But that was life. I was close to being killed so many times. A hell of a lot of times.”
‘A genteel lynch mob’
When he became a father, Williams wanted his kids to be ready to overcome adversity, racial or otherwise.
In 2001, the unorthodox training he provided his daughters would prove prescient when they were abused at one of the most prestigious tournaments on the world tennis circuit.
As Serena walked on court for the final of the Indian Wells event in the well-heeled Californian desert community, the watching crowd let rip.
Boos rained down on the then 19-year-old, while Venus and her father received a similar welcome as they took their seats in the stand.
The crowd was unhappy that Venus had pulled out of the semifinal clash against her sister a day earlier due to injury. Some suspected that matches between the duo were fixed by their father to maintain family harmony.
Accusations of racism would follow, with Serena writing in her autobiography that “all I could see was a sea of rich people — mostly older, mostly white — standing and booing lustily, like some kind of genteel lynch mob.”
Richard Williams would also state in an interview with USA Today that “one guy said, ‘I wish it was ’75, we’d skin you alive.'” Although nobody was arrested, tournament director Charlie Pasarell said he didn’t discount that Williams Sr. was racially abused.
Despite the poisonous nature of the occasion, Williams knew his youngest daughter was ready for what unfolded that day.
Serena would go on to defeat fellow teen Kim Clijsters 4-6 6-4 6-2 despite being treated like what he describes as a “a dog, or frog or hog.”
“The whole crowd turned against her and all she had to do was remember the training that she had been through,” he says.
These challenging episodes have only bolstered the bonds that have bound the Williams family on its way to the top of the tennis world.
Serena has 21 grand slam singles titles to her name while Venus has seven, despite suffering from illness and injuries in recent years.
Richard Williams says sticking close together was something he learned from his own mother at a very early age. “She instilled in me so many positive things,” he says. “That’s probably my earliest memory, (my) greatest memory.”
His mother’s many pearls of wisdom included advice on how to survive in a world where he was beaten up as a five-year-old for “handing a dollar to a white store clerk.”
“People say to their kids or to themselves that they would be the thing they want,” Williams says. “But she said that’s very difficult. It’s easy to say, but very difficult to be done. She was showing me ways of how it could be done.”
That Williams channeled this advice to help his two daughters become leading players is the stuff of tennis legend.
When up late watching TV one night in 1980, he turned the channel to see Romania’s Virginia Ruzici receive a check for winning a tennis tournament that he describes as “a hell of a lot of money for four days’ (work).”
“I went to my wife and said that we have two kids, and we’ll become rich. They’re going to be tennis players.”
A 78-page blueprint was soon prepared, describing how Venus and Serena would reach the top. A key plank of the plan was that it would be enacted from one of America’s roughest areas.
The Williams family was not poor and could have afforded to live in more well-to-do areas. But it was decided that Compton was the best place for their early education.
“There was no place in the world that was rougher than Compton,” Williams said. “The ghetto will make you rough, it’ll make you tough, it’ll make you strong.
“And so that’s why I went to Compton with them.”
Teaching Serena to paint
Although Venus and Serena left Compton to attend the Rick Macci tennis school in Florida in the early 1990s, Williams believes they never forgot those lessons.
Serena’s unquestionable power, aggression and on-court mental fortitude certainly attest to that. Yet those deep wells of strength have been required in more troublesome circumstances too.
The Indian Wells debacle was followed by the 2003 death of Venus and Serena’s half sister Yetunde Price in a drive-by shooting in Compton.
In 2011, meanwhile, Serena received the shock news that she had a number of blood clots in her lungs, while still recovering from a serious foot injury caused by treading on broken glass at a restaurant the previous year.
She required immediate surgery, and those closest to her feared for her life.
“I just thought she was going to die,” her father says, recalling a painful memory he describes as “just too much.”
An operation saved her, and she would not play on tour again until June 2011.
Yet Williams believes one of his unorthodox suggestions played its part in the recuperation.
“She helped me paint a fence,” he says. “Tennis wasn’t on my mind, but hoping that my daughter would live was on my mind and getting her to do something.
“My wife and Serena painted the fence. And Serena felt better, I believe, from that day on.”
Although world No. 1 Serena remains the dominant force in women’s tennis and Venus has recaptured some of her best form in recent years as she copes with Sjogren’s syndrome, their father and one-time coach now struggles to watch his girls play.
He’s seldom in the players’ box at tournaments these days and feels his presence is no longer necessary on the biggest occasions.
“I would never want to be at another tennis tournament ever,” he said. “I’ve been there for too long.”
But that doesn’t mean the strength of feeling between father and daughters who have experienced so much together has tapered.
“Serena often tells me that if it wasn’t for me and Jehovah she wouldn’t be where she is. That’s enough to make you cry. I tried to get them to believe that they have pushed themselves, not me,” Williams says.
“Too many parents are pushing their kids the wrong way, and that could have been me at one time.
“But I hope that the two of them, whenever they retire, whatever they’re going to do, they can accept what they have.”
™ & © 2015 Cable News Network, Inc., a Time Warner Company. All rights reserved