Faulty AI that told people to break the law, championed by the mayor of New York
An AI chatbot recently introduced in New York City as a way for business owners to access “actionable and trusted information” 24/7 has been defended by Mayor Eric Adams after being called out for encouraging entrepreneurs to break the law violate.
The MyCity AI chatbot uses Microsoft’s major Azure AI language models, but is plagued by the same issues that plague most AI models, including providing false and misleading information.
In response to the criticism, Mayor Adams stated that the AI tool is “wrong in some areas, and we need to fix it,” further defending the chatbot by saying, “Any time you use technology, you need to put it in the real environment to iron out the kinks.”
Break the law, it’s completely legal
It was previously reported that the chatbot told landlords they were free to discriminate based on income, and that business owners could take employee tips as their own — despite both practices being illegal in New York.
Subsequent investigations into the chatbot’s responses on Thursday found that it was still spreading misinformation Reuters report that the AI had said that stores no longer had to accept cash as a form of payment, despite the fact that this is against New York law.
NYC Hospitality Alliance Executive Director Andrew Rigie said, “I commend the city for its efforts to use AI to help businesses, but it has to work. “If, when I ask a question, I have to go back to my lawyers to know whether the answer is correct or not, that defeats the purpose.”
The MyCity chatbot has since been updated to note that the AI’s answers “may sometimes be inaccurate or incomplete” and that the answers given to questions should not be relied upon as legal or professional advice.