A DUP MP says they are not bluffing over their Brexit threat and the government needs to take a "hard look" at their position.
DUP leader Arlene Foster is steadfastly against any agreement with the EU that would leave Northern Ireland being treated differently to the rest of the UK.
The support of the Unionist party is crucial to Mrs May, who agreed a "confidence and supply" deal with the DUP in the wake of losing her parliamentary majority in last June's snap election.
Defeat on the Budget could be a mortal blow to the PM's authority and spark a no-confidence vote in the government.
Under the Fixed-term Parliament Act, if Mrs May lost a no-confidence vote, an election would be held if no alternative government was confirmed by the Commons within two weeks.
Downing Street has insisted defeat on the Budget would not amount to a vote of no confidence in the government, but there is no doubt the rhetoric of recent days has ratcheted up the Brexit brinkmanship.
Speaking to Sky's political correspondent Tamara Cohen, DUP MP Jim Shannon insisted the party was not bluffing.
"In the next two weeks we'll be deciding what we're going to do and I think, in all honesty Tamara, that everything is on the table, and the implications of that are clear," he said.
"We're happy on this principle that we want to be treated the same as rest of the United Kingdom, to fight any election on that should that happen.
"I hope it won't happen, we don't want to embarrass the government, but we want the government to take a long hard look at our position."
The threat to vote down the Budget will put pressure on the Cabinet to deliver a deal in which the whole of the UK remains in a temporary customs union with the EU until a solution is found which would avoid regulatory or customs checks on the Irish border.
But such an arrangement would provoke outrage from Conservative backbenchers, who say Britain would find it very difficult to end such an arrangement as it would leave our exit date in the hands of the EU.
Steve Baker, Conservative MP for Wycombe, said he believed "at least 40" of his colleagues would vote against a deal along the lines the PM has proposed which would include this "backstop" for Northern Ireland.