Building A Privacy Preserving Voice Assistant

Summary

Being able to control a computer with your voice has rapidly moved from science fiction to science fact. Unfortunately, the majority of platforms that have been made available to consumers are controlled by large organizations with little incentive to respect users’ privacy. The team at Snips are building a platform that runs entirely off-line and on-device so that your information is always in your control. In this episode Adrien Ball explains how the Snips architecture works, the challenges of building a speech recognition and natural language understanding toolchain that works on limited resources, and how they are tackling issues around usability for casual consumers. If you have been interested in taking advantage of personal voice assistants, but wary of using commercially available options, this is definitely worth a listen.

Announcements

• Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
• When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With 200 Gbit/s private networking, scalable shared block storage, node balancers, and a 40 Gbit/s public network, all controlled by a brand new API you’ve got everything you need to scale up. And for your tasks that need fast computation, such as training machine learning models, they just launched dedicated CPU instances. Go to pythonpodcast.com/linode to get a $20 credit and launch a new server in under a minute. And don’t forget to thank them for their continued support of this show!
• You listen to this show to learn and stay up to date with the ways that Python is being used, including the latest in machine learning and data analysis. For even more opportunities to meet, listen, and learn from your peers you don’t want to miss out on this year’s conference season. We have partnered with organizations such as O’Reilly Media, Dataversity, and the Open Data Science Conference. Go to pythonpodcast.com/conferences to learn more and take advantage of our partner discounts when you register.
• Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
• To help other people find the show please leave a review on iTunes and tell your friends and co-workers
• Join the community in the new Zulip chat workspace at pythonpodcast.com/chat
• Your host as usual is Tobias Macey and today I’m interviewing Adrien Ball about SNIPS, a set of technologies to make voice controlled systems that respect user’s privacy

Interview

• Introductions
• How did you get introduced to Python?
• Can you start by explaining what the Snips is and how it got started?
• For someone who wants to use Snips can you talk through the onboarding proces?
  • One of the interesting features of your platform is the option for automated training data generation. Can you explain how that works?
• Can you describe the overall architecture of the Snips platform and how it has evolved since you first began working on it?
• Two of the main components that can be used independently are the ASR (Automated Speech Recognition) and NLU (Natural Language Understanding) engines. Each of those have a number of competitors in the market, both open source and commercial. How would you describe your overall position in the market for each of those projects?
• I know that one of the biggest challenges in conversational interfaces is maintaining context for multi-step interactions. How is that handled in Snips?
• For the NLU engine, you recently ported it from Python to Rust. What was your motivation for doing so and how would you characterize your experience between the two languages?
  • Are you continuing to maintain both implementations and if so how are you maintaining feature parity?
• How do you approach the overall usability and user experience, particularly for non-technical end users?
  • How is discoverability handled (e.g. finding out what capabilities/skills are available)
• One of the compelling aspects of Snips is the ability to deploy to a wide variety of devices, including offline support. Can you talk through that deployment process, both from a user perspective and how it is implemented under the covers?
  • What is involved in updating deployed models and keeping track of which versions are deployed to which devices?
• What is involved in adding new capabilities or integrations to the Snips platform?
• What are the limitations of running everything offline and on-device?
  • When is Snips the wrong choice?
• In the process of building and maintaining the various components of Snips, what have been some of the most useful/interesting/unexpected lessons that you have learned?
  • What have been the most challenging aspects?
• What are some of the most interesting/innovative/unexpected ways that you have seen the Snips technologies used?
• What is in store for the future of Snips?

Keep In Touch

• LinkedIn
• adrienball on GitHub
• @adrien_ball on Medium
• @adrien_ball on Twitter

Picks

• Tobias
  • Chrome OS
• Adrien
  • Google I/O
  • Facebook F8
  • User Privacy

Links

• Snips
• 2048 Game
• Smart Cities
• Raspberry Pi
• WikiData
• MQTT
• Google Assistant
• Amazon Alexa
• Microsoft Cortana
• Mozilla Common Voice
• Rust Language
• Snips Hermes messaging library

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA