In Johnson's first electoral test since he took office late last month, the party was defeated for the parliamentary seat of Brecon and Radnorshire in Wales by Jane Dodds, 55-year-old leader of the anti-Brexit Liberal Democrats, who won with 13,286 votes, a majority of 1,425, the UK media reported.
The Brecon vote was triggered when Conservative Chris Davies was unseated by a petition of constituents after being convicted and fined for expenses fraud. Davies, who fought to retain the seat, came second with 12,401 votes.
It was a success for the Liberal Democrats, which had previously held the seat from 1997 to 2015, when it was won by Davies.
The Brexit Party, launched in April and led by eurosceptic Nigel Farage, came third with 3,331 votes. Britain's main opposition Labour Party, whose leadership is divided over Brexit, was fourth with 1,680 votes.
Two other anti-Brexit parties Plaid Cymru (the Party of Wales) and the Green Party decided not to run in the by-election to give Dodds a better chance to win.
"My very first act as your new MP when I get to Westminster will be to find Boris Johnson, wherever he's hiding, and tell him to stop playing with the future of our community and rule out a no-deal Brexit now," Dodds said in her victory speech.
"There is no time for tribalism when our country is faced with a Boris Johnson government and the threat of a no-deal Brexit," she added.
"Boris Johnson's shrinking majority makes it clear that he has no mandate to crash us out of the European Union (EU)," said Liberal Democrats Leader Jo Swinson.
The Conservatives have already lacked an overall majority in the House of Commons and rely on an alliance with 10 lawmakers from the Democratic Unionist Party of Northern Ireland. Losing the Brecon seat has left the alliance with only 320 of the 639 parliamentary seats, Xinhua news agency reported.
It means Johnson's government may struggle to pass any plan to take Britain out of the EU on October 31, deal or no deal, when the British Parliament reassembles on September 3 after summer holiday recess.
Still, Roger Awan-Scully, head of Politics and International Relations at Britain's Cardiff University, had his understanding of the by-election.
"Until a few days ago, people were talking seriously about the Brexit Party pushing the Conservatives into third place. They've done well resisting the pressure from the Brexit Party," the professor said.
"It hasn't quite been good enough this time round but whenever we get a general election, which might not be far away, this seat is very much in play for the Conservatives," he added.
( With inputs from IANS )