Hundreds of New Yorkers who have been found with small amounts of marijuana, or who have simply admitted to using the drug, have been accused of child neglect, even though they did not face criminal charges.

In some of these cases, parents have lost custody of their children, according to The New York Times. The city’s child welfare agency said marijuana use by parents often indicates other serious problems in how a parent cares for their child. New York’s approach contrasts with that of California, which demands that child welfare officials show harm to a child from marijuana use before a neglect case can proceed.

Lawyers from legal services groups that defend parents told the newspaper they see hundreds of marijuana cases every year, mostly involving recreational users. They said they are handling more than a dozen cases that involve parents who have not faced neglect allegations, yet whose children have been placed in foster care because of marijuana charges.

City health data show the rate of marijuana use is twice as high among whites as among blacks and Hispanics. However, defense lawyers say child neglect cases involving a parent’s use of marijuana are rarely filed against white parents.

Michael Fagan, a spokesman for the city’s Administration for Children’s Services, told the newspaper, “Drug use itself is not child abuse or neglect, but it can put children in danger of neglect or abuse. We think the argument that use of cocaine, heroin or marijuana by a parent of young children should not be looked into or should simply be ignored is just plain wrong.”