The 60-year-old ran on a promise of shaking up traditional party politics in the Central American nation.
With almost all votes counted, he had a lead of more than five percentage points over ex-President José María Figueres.
Turnout was the lowest in decades as voters expressed discontent with Costa Rica's struggling economy.
José María Figueres, who ran for a second term as president after having held the top job from 1994 to 1998, admitted defeat once his rival's lead had become too big to be overcome.
Many Costa Ricans said they were looking for a break with the politicians of the past, a trend which went against Mr Figueres, who comes from one of the country's most influential political families.
While Mr Chaves is not entirely new to politics - he did serve in the outgoing government of President Carlos Alvarado as economy minister for six months - he has emphasised his plan to break with what he calls "privileges" of Costa Rica's established political parties.
Rodrigo Chaves has moved away from the country's traditional political parties and presented himself as a maverick and an outsider to the established order.
He has said he is prepared to use referendums to bypass Congress and shake up the usual stable political system in the Central American nation.
In large part, he benefited from widespread discontent in the country, as Costa Ricans are frustrated at what they see as worsening corruption in a country whose tourism-based economy has been badly affected by the coronavirus pandemic.
Following an at times acrimonious campaign, Mr Chaves struck a conciliatory note as his win became clear.
"Tonight, let's set aside the party colours that can easily divide us," he told his supporters. "I humbly ask you that we unite beneath the blue, white and red of our national flag."
Supporters of Mr Chaves took to the streets of the capital, San José, to celebrate their candidate's win.
One of those celebrating told Reuters news agency that there was a "hunger for real change" in Costa Rica. "We need new faces to save the country," the man said.