Build Your Own Domain Specific Language in Python With textX

Summary

Programming languages are a powerful tool and can be used to create all manner of applications, however sometimes their syntax is more cumbersome than necessary. For some industries or subject areas there is already an agreed upon set of concepts that can be used to express your logic. For those cases you can create a Domain Specific Language, or DSL to make it easier to write programs that can express the necessary logic with a custom syntax. In this episode Igor Dejanović shares his work on textX and how you can use it to build your own DSLs with Python. He explains his motivations for creating it, how it compares to other tools in the Python ecosystem for building parsers, and how you can use it to build your own custom languages.

Announcements

• Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
• When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With the launch of their managed Kubernetes platform it’s easy to get started with the next generation of deployment and scaling, powered by the battle tested Linode platform, including simple pricing, node balancers, 40Gbit networking, dedicated CPU and GPU instances, and worldwide data centers. Go to pythonpodcast.com/linode and get a $60 credit to try out a Kubernetes cluster of your own. And don’t forget to thank them for their continued support of this show!
• You listen to this show to learn and stay up to date with the ways that Python is being used, including the latest in machine learning and data analysis. For more opportunities to stay up to date, gain new skills, and learn from your peers there are a growing number of virtual events that you can attend from the comfort and safety of your home. Go to pythonpodcast.com/conferences to check out the upcoming events being offered by our partners and get registered today!
• Your host as usual is Tobias Macey and today I’m interviewing Igor Dejanović about textX, a meta-language for building domain specific languges in Python

Interview

• Introductions
• How did you get introduced to Python?
• Can you start by describing what a domain specific language is and some examples of when you might need one?
• What is textX and what was your motivation for creating it?
• There are a number of other libraries in the Python ecosystem for building parsers, and for creating DSLs. What are the features of textX that might lead someone to choose it over the other options?
• What are some of the challenges that face language designers when constructing the syntax of their DSL?
• Beyond being able to parse and process an arbitrary syntax, there are other concerns for consumers of the definition in terms of tooling. How does textX provide support to those end users?
• How is textX implemented?
  • How has the design or goals of textX changed since you first began working on it?
• What is the workflow for someone using textX to build their own DSL?
  • Once they have defined the grammar, how do they distribute the generated interpreter for others to use?
• What are some of the common challenges that users of textX face when trying to define their DSL?
• What are some of the cases where a PEG parser is unable to unambiguously process a defined grammar?
• What are some of the most interesting/innovative/unexpected ways that you have seen textX used?
• What have you found to be the most interesting, unexpected, or challenging lessons that you have learned while building and maintaining textX and its associated projects?
• While preparing for this interview I noticed that you have another parser library in the form of Parglare. How has your experience working with textX informed your designs of that project?
  • What lessons have you taken back from Parglare into textX?
• When is textX the wrong choice, and someone might be better served by another DSL library, different style of parser, or just hand-crafting a simple parser with a regex?
• What do you have planned for the future of textX?

Keep In Touch

• Website
• igordejanovic on GitHub
• @dejanovicigor on Twitter

Picks

• Tobias
  • wemake-python-styleguide
• Igor
  • Interactive Fiction genre
    • Awesome Interactive Fiction
    • The Interactive Fiction Database
    • TADS
    • Inform 7

Closing Announcements

• Thank you for listening! Don’t forget to check out our other show, the Data Engineering Podcast for the latest on modern data management.
• Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
• If you’ve learned something or tried out a project from the show then tell us about it! Email [email protected]) with your story.
• To help other people find the show please leave a review on iTunes and tell your friends and co-workers
• Join the community in the new Zulip chat workspace at pythonpodcast.com/chat

Links

• textX
• U of Novi Sad
• Serbia
• DSL course
• Secondary Notation
• Django
• Xtext
• Eclipse
• PLY
• SLY
• PyParsing
• Lark
• PEG Grammar
• Language Workbench
• Language Server Protocol
• Visual Studio Code
• textX-LS
• Arpeggio Parser
• Context-Free Grammar
• pyTabs
• Guitar Tablatures
• Parglare
• GLR parsing
• TEP 1
• Evennia
  • Podcast Episode

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA