She said that one home and five away fans were arrested for sectarian abuse.
Budge was "appalled" at vandalism she says "caused the destruction of close to 100 seats" at Tynecastle, plus "disgraceful" acts of graffiti.
She also complained of "intimidation" of Hearts fans and staff, with some young fans being "traumatised".
Budge told her club website she had discussed the problems with Celtic, but the Scottish Premiership club have yet to make a public response.
Celtic won the fourth-round tie 4-0, with the Championship leaders' chances having been hampered by a red card shown to Morgaro Gomis within 10 minutes.
"The match was a disappointing football spectacle for both sets of supporters," said Budge. "It does not justify bad behaviour - on or off the field.
"I was appalled at the vandalism carried out by our visiting supporters on Sunday.
"Having recently spent significant sums to replace broken seats in the Roseburn Stand, for the benefit, primarily, of our away supporters, I was hugely disappointed to witness the destruction of close to 100 seats.
"Similarly, the graffiti on walls, tiles, doors - basically all types of surfaces - was quite disgraceful and this is not to mention the damage to toilets.
"I have received a number of emails/letters from parents stating that their children were terrified/had to be removed part way through the game/were traumatised - and as such would be unlikely ever to want to return to a game. This is appalling."
Budge said she had spoken to Celtic about the incidents and planned to raise concerns with the police.