Lib Dems say they are "disappointed" in the Prime Minister, while a Labour MP claims Mrs May is afraid of facing Jeremy Corbyn.
John McDonnell told Sky News that live debates during the Labour leadership contest had helped transform Mr Corbyn into "a candidate who could win" - and claimed the Prime Minister was worried about history repeating itself in the General Election.
"Once we get into a debate about the real issues - the policies about what's happening to the health service, education, the economy, care for the elderly and our future relationship with the EU - I think you'll see the polls narrow and the prospect of a Labour government," Mr McDonnell said.
Mrs May's decision to get "out and about" meeting voters instead of facing off with party leaders in a TV studio has also drawn criticism from the Liberal Democrats.
Tim Farron said he was "very disappointed" in Mrs May for refusing to commit to a televised event, adding: "It seems to me she feels she has got everything to lose by going on television and debating myself and others.
"When all is said and done, she has chosen this election, she presumably has some confidence in her position, why won't she go out there and argue with people like me who want to challenge her?"
And Leanne Wood, whose profile as Plaid Cymru leader received a considerable boost after she appeared in two of the 2015 broadcasts, said: "Theresa May should be empty chaired if she doesn't show up to any planned TV debates."
In the hours after Mrs May's surprise call for a snap election on Tuesday, Mr Corbyn had urged the Conservative leader to agree to a live TV event - describing them as "what democracy needs and what the British people deserve".
However, during an interview on BBC Radio 4's Today programme, Mrs May said: "We won't be doing television debates.
"I believe in campaigns where politicians actually get out and about and meet with voters… that's what I'm going to be doing around this campaign."
Although a mainstay of US presidential elections for decades, live TV debates during UK general elections are a fairly new concept.
In 2010, David Cameron, Nick Clegg and Gordon Brown took part in three such events - but there was much wrangling over the formats for the debates which followed in 2015.
In the end, one debate took place between the leaders of seven parties: the Conservatives, UKIP, Labour, the Lib Dems, the Green Party, Plaid Cymru and the Scottish National Party.
Another debate involved the five parties not involved in the coalition government.
Although David Cameron and Ed Miliband did take part in a 90-minute programme before the 2015 vote, they were questioned separately and did not debate face-to-face.
In past elections, Sky News has joined other broadcasters in urging all major political parties to commit to televised debates.