“The Prude family and the Rochester community have been through great pain and anguish,” attorney general Letitia James said in a statement about Mr Prude’s death, which has sparked nightly protests and calls for reform.
She said the grand jury would be part of an “exhaustive investigation”.
Mr Prude’s death after his brother called for help for his erratic behaviour in March has inflamed New York’s third-largest city since video of the encounter was made public earlier this week.
Protesters have demanded more accountability for how the death happened, and legislation to change how authorities respond to mental health emergencies.
“This is just the beginning,” Ashley Gantt, a protest organiser, told the Associated Press by email after Ms James’ announcement. “We will not be stopped in our quest for truth and justice.”
Protesters gathered on Saturday for a fourth night on the street where Mr Prude, naked and handcuffed, was held face-down as snow fell.
Police body camera video shows officers covering Mr Prude’s head with a “spit hood” designed to protect police from bodily fluids, then pressing his face into the pavement for two minutes.
Mr Prude died on March 30 after he was taken off life support.
The Monroe County medical examiner listed the manner of death as homicide caused by “complications of asphyxia in the setting of physical restraint”.
Excited delirium and acute intoxication by phencyclidine, or PCP, were contributing factors, the report said.
A police internal affairs investigation cleared the officers involved of any wrongdoing, concluding in April that their “actions and conduct displayed when dealing with Prude appear to be appropriate and consistent with their training”.
Ms James’ office opened its investigation the same month. Under New York law, deaths of unarmed people in police custody are often turned over to the attorney general’s office, rather than handled by local officials.
Governor Andrew Cuomo earlier this week called on Ms James to expedite the probe.
“Today, I applaud Attorney General Tish James for taking swift, decisive action in empanelling a grand jury,” Mr Cuomo said in a statement.
“Justice delayed is justice denied and the people of New York deserve the truth.”
