The veteran left-wing politician, known widely as Lula, also led the country between 2003 and 2010 - and defeated Jair Bolsonaro in October's poll.
In his first speech, Lula vowed to rebuild a country in "terrible ruins".
He decried the policies of his predecessor, who went to the US on Friday to avoid the handover ceremony.
A sea of Lula supporters gathered in front of Congress since early in the morning - decked out in the red colour of his Workers' Party. They travelled to see their leader sworn in - but also for a celebration.
More than 60 artists - including Samba legend Martinho da Vila - were booked to perform on two giant stages decorated in the national flag as part of a music festival dubbed "Lulapalooza".
"Love has won over hate," read one banner carried by a man dressed as Lula - complete with a presidential sash.
"Brazil needed this change, this transformation," said another backer of the incoming leader as she queued for Sunday's festivities.
Juliana Barreto - who is from Lula's home state Pernambuco - told the BBC that her country was "a disaster" previously.
Lula and incoming Vice-President Geraldo Alckmin paraded through the city on an open-top convertible before proceeding to the Congress building - where the swearing-in occurred at the start of the formal inauguration ceremony.
The men have spent the past days selecting their cabinet and appointing supporters to key state-owned businesses.
Shortly after being sworn in, Lula sought to instil a sense of hope in the people of Brazil and promised to "rebuild the nation and make a Brazil of all, for all".
There were several instances when he got out his hanky. His most emotional moment came when speaking to the Brazilian people after the swearing-in ceremony - he started sobbing when talking about those who beg at traffic lights, desperate for food.
Probably not even Lula thought this day would ever come - a return to the top job after two decades, despite a spell in prison after being convicted of corruption. The convictions were subsequently annulled in 2021.
Much of his speech to Congress was about unity and reconstruction. The two words are crucial in such a deeply divided country, hit hard by the pandemic and hugely polarised politically.
Lula knows that his ultimate challenge will be to convince those who feel he is a corrupt politician who belongs in jail that he does now belong in the presidential palace again and can be their leader too.
He pledged to undo the legacy of his predecessor's government, which he said involved depleting funding for education, health and the conservation of the Amazon rainforest.
To huge cheers from those watching in Congress, he also promised to revoke Mr Bolsonaro's controversial gun laws immediately.
Lula went on to state that his government would not be motivated by "a spirit of revenge", but that those who had made mistakes would answer for their errors.
In particular, he singled out Mr Bolsonaro's Covid-19 policies, accusing him of causing a "genocide" of deaths in Brazil during the pandemic, which would need to be fully investigated.
In another noted change of policy from the Bolsonaro administration, Marina Silva - one of Brazil's best known climate activists - was re-appointed to head the environment and climate ministry. She will be expected to achieve Lula's pledge - which was repeated during his speech - to reach "zero deforestation" in the Amazon by 2030.
The atmosphere in Brasilia couldn't be more different than when Mr Bolsonaro was in power. Lots of people were waving banners or wearing T-shirts with the words "Love conquers hate," a reference to the narrative many felt came from Mr Bolsonaro.
But diversity and inclusion too was a big part of today's inauguration. With Mr Bolsonaro abandoning his final official duty of passing on the presidential sash, it was left to Eni Souza, a rubbish picker, to do the honours. And standing next to Lula was an indigenous leader, a black boy and a disabled influencer. In this country where racism is all too common, it was an important image that will endure.
Watch: Crowds celebrate in Brazil as Lula is sworn in as president
Lula broke down in tears as he talked about poverty in his country