The attackers also set fire to shops and homes, along with the residence of the traditional leader, forcing the residents of Nuku and Woro to flee, local lawmaker Saidu Baba Ahmed told BBC Hausa.
Officials have blamed jihadist groups and deployed an army battalion to Kwara's Kaiama constituency.
Tuesday's attack, one of several across Nigeria in the last few days, comes as the country's defence minister confirmed to the BBC that a small team of US troops was in the country to help with intelligence and training.
Nigerian President Bola Tinubu said Islamist militant group Boko Haram was behind the attack. Local lawmaker Mohammed Omar Bio said Lakurawa, an armed group affiliated with Islamic State, was responsible, according to the Associated Press.
Kwara State Governor AbdulRahman AbdulRazaq said 75 local Muslims "were massacred in cold blood simply for refusing to surrender to extremists who preached a strange doctrine".
Ahmed initially told BBC Hausa that at least 35 people had died, but human rights groups have since reported more than four times that number of deaths.
A Red Cross official in Kwara, Babaomo Ayodeji, told AFP that "reports said that the death toll now stands at 162, as the search for more bodies continues".
Amnesty International said in a statement that over 170 people had died, noting many were shot at close range and some burnt alive.
Several people were abducted in the attack that was undertaken in a "stunning absence of any form of security for the protection of lives", the human rights group said while calling for an investigation.
A further 21 people were killed in an attack on Doma village in the state of Katsina, they added.
The attacks come alongside the first official acknowledgement from Nigeria of an American troop presence since US President Donald Trump ordered the military in November to prepare for action in Nigeria to tackle Islamist militant groups.
Defence Minister Chrisopher Musa did not provide details on the team's size, arrival date, location or duration of stay.
His comments follow those made on Tuesday by Gen Dagvin Anderson of US Africa Command (Africom), who said the deployment followed a Nigerian request and was focused on intelligence support.
"Our partnership with Nigeria is a great example of a very willing and capable partner who requested the unique capabilities that only the US can bring," he said.
Nigeria faces an array of security challenges including criminal gangs - known locally as "bandits" who loot and kidnap for ransom - an Islamist insurgency, clashes over land and separatist unrest.
