[Team45] Kaki-2o: an AI-Powered Multi-lingual Chatbot to access Municipal Services

Problem statement

Only 56% of residents surveyed were aware of OneService digital channels.

Without a structured feedback process and data, the ability of agencies and town councils to effectively identify, triage, and address resident’s concerns is hindered, leading to dissatisfaction with municipal services.

To facilitate public engagement and streamline the reporting of municipal issues, the Municipal Services Office has developed several digital platforms, however:

• 👎Awareness of digital platforms remains low, especially amongst residents >55y/o
• ⛔Technical and language barriers may exclude certain groups of residents from using MSO's digital offerings
• 🧓Residents who choose to access Municipal services through hotlines or physical touch points -- face inconveniences, e.g. time wasted traveling or finding the right agency; and -- create additional handling and staffing for our agencies and town councils

Our Solution

Our team's solution focuses on the utilization of GenAI models to transcribe speech to text and to process vernacular inputs in Chinese, Malay, or Tamil, so as to enable a more natural way for our residents to access Municipal services through a highly conversant and low-friction Chatbot user experience.

• 📲There is a high penetration of mobile devices in Singapore, with WhatsApp, Telegram, and Instagram being the most popular messaging platform
• 🤖GenAI models like Whisper and gpt-4o have matured to become more conversant and facilitate multi-lingual (e.g. Chinese, Malay, Tamil) and multi-modal (Speech and Text) inputs.

How it works

Raw speech recording and text are first transmitted from the Telegram front end to the Kaki-2o model. For speech, this is transcribed by the Whisper model and passed to the gpt-4o LLM where the chatbot prompts are stored.

The chatbot is designed to capture the necessary details for case submission, such as the description of the issue, location, date and time, name and contact, by interacting with the user in a natural and organized way. The LLM is also able to infer the case category based on the user's description, which is used to determine the correct agency for routing.

These details, along with the case category, are then stored in the data persistence tier, which can then be interfaced with other systems through APIs/data exchanges.

Test our Proof of Concept

Experience our chatbot first hand by visiting us at Level 11, Booth DA4 (near the Stairs to Level 10)

Impact and outcomes

We tested Proof of Concept first with 7 testers, some of whom were introduced from GovTech's Tech Kaki network. We iterated based on the user feedback to refine our initial model. We found that the Chatbot:

• ⌛User were able to submit a case in an average of 4 minutes 15 secs.
• 5/7 (71%) rated Kaki-2o at 4-5 stars ⭐
• 5/7 (71%) users would recommend Kaki-2o to especially seniors 🧓
• Depending on user interaction, each submission costed about 🪙4-8 cts ( can be further managed by using gpt-4o-mini)
• There are some nuances between local languages and translations, e.g. Selamat Jalan instead of Selamat Hari

From discussion with visitors to our booth at the Feedback Bazaar

• Kaki-2o could be scalable to other agencies, e.g. STB, MHA, ICA.
• Kaki-2o could also facilitate access to Government services for persons with disability, e.g. unable to type, visual impairment.

Future steps for your project

We envision that the Kaki-2o AI model will introduce a new way for Citizens to access Government services beyond Municipal Services as the first of its kind if it can be incubated to reach production.

However, there is still more work required to:

• Localise the chatbot to understand Singaporean vernacular terms and addresses (possibly with MDDI/SLA)
• Integrate the chatbot with the OneService@SG(OSSG) CRM system for case routing
• Further test the chatbot with more vernacular testers to identify any nuances that need to be addressed (possibly GovTech Kakis)
• Secure the model using GovTech's AIGuardian Products (Litmus & Sentinel)

To achieve this, we will require resources such:

• Hosting of the Kaki-2o model on GovTech's existing SGTechStack infrastructure (possibly with VICA)
• 2 AI and 2 Backend Engineers to work with the MSO product team
• Setup tester panel with Chinese, Malay and Tamil language proficiencies (TechKaki)
• Development costs for LLM tokens (est. about $1500) and Infra hosting cost (est. about $500)

Kaki-2o Team members and respective divisions

Kaki-2o was bought to you by the following team of engineers and analysts who are passionate about AI products! 🙌

Written with StackEdit.