295: Is Your Type System Leaking?

0:00Hello, and welcome to the Thinking Elixir podcast, where we cover the news of the community and learn from each other. My name is Mark Erickson, and I'm David Byrne Heisel. Let's jump into the news. Oh, actually, before we get to the news, I got one little call out. Let's start with this. I'm going to put this in the perspective a little bit for us. Most of the work for companies that use Elixir, you know, in its community and ecosystem you get to benefit from all of that effort, in many cases, for free. So a lot of us may personally sponsor some good work and folks, you know, in the ecosystem, personally, I do that for the LSP folks. It's a simple principle, right? Put your money where your mouth is if you need that tool to be better, put some funding to it, right? Vote with your wallet, as they say. And so it wasn't much, but it was something, and at a personal level, it really is not as much as it should be, in my opinion. I know that, and I know who's doing that. And so I know what to do to go sponsor them, right?

1:00Then they've put out little calls of, you know, I need funding or whatever. And that takes work for them to even do. And a lot of other cases, a lot of us don't know who or what to sponsor either. We might share that feeling that, yeah, I appreciate that work here and I want to contribute somehow. I can't contribute my time, but I can contribute my money. That's what the Irling ecosystem foundation is here to help with, right? That's not their sole purpose, but that is a major purpose of what they do. They do collect funds and redistribute that to the community in various ways. It is, in my opinion, the primary place for the goodwill to be distributed to the community. More so, maybe this is just the anti-American part of me that's very American, by the way. But I think it's a very American attitude that all responsibility relies, you know, ends up on the individual and that there's like no community responsibility. You know what I mean? Compared to the Japanese culture, the Japanese culture is like the opposite, right? It's like, you do everything for the community, nothing for yourself, but in America, it's like, no, you do nothing for your community, you do it all for yourself.

2:03I don't like that philosophy. Your most active and immediate electric community is most likely, you are truly, because you listen to our podcast and we thank you for that. That's just a joke. The real, most immediate electric community, if you're listening to this, is most likely your workplace and your workplace probably has other folks that, you know, at least know a little bit about a lecture or a write a lecture or that kind of stuff, your slack rooms, your discourse, whatever. So here's the ask, let me get down to it. If your company is not sponsoring the E.E.F., it probably should. I'll say that. Yeah, it probably should, but in my opinion, it probably should. Your company is most likely benefiting from a lot of the goodwill of the elixir community and the folks that are in it for probably virtually free, you know, if they are not sponsoring the E.E.F. in some way. And I know companies range from small to large to contractor to solo to enterprise, right? So don't even worry about amounts.

3:05The ask is all that we're looking for. So maybe today, you would consider asking someone in your company that has access to the purse. If they are able to go to rlf.org, e-r-l-e-f.org, that's urling ecosystem foundation.org, and they can find out some more information there about how to sponsor the ecosystem and let them decide how much would be appropriate. You know, maybe they'll ask you even what would be appropriate because, yeah, communities in need. And there's lots of efforts that the foundation wants to fund. And we can only do that if they get funding themselves. All right. That's it for that. I was on the top because I know a lot of folks will tune in for the news, you know, and then maybe they're, they're commute prevents them from hearing the tail end of the news. So I want to make sure you guys hear that the urling ecosystem foundation is looking for funding. Go to e-r-l-e-f.org and see if you can help. And first up for the news, Jose Valin published a new technical blog post on the elixir Lang org website.

4:06It is actually titled lazy BDDs with eager literal intersections. So this is actually really deep academic sort of really going in on how this type system stuff is working. So for you deep language geeks out there, Ashton, I'm thinking of you, we will try and keep this brief and high level for everyone else. But the post is quite technical. It's discussing how elixir changed its set theoretic type representations from DNFs or disjunctive normal forms to BDDs, which is binary decision diagrams and how that solved one set of problems and then introduced another set of problems and then how they solve that. It changed it to be a lazy system instead of having to pre compute and flatten all of these types together. The end result for the blog post, it just to quote some of it here is the initially implemented eager literal intersections as part of a elixir version 1.20 release, which reduced the type checking time of one of the worst cases from 10 seconds to 25 milliseconds.

5:10So you know, this worst case situation, if you like, did it absolutely the weirdest way you could to create these type system issues, it could really take 10 seconds to resolve and compile some of this and they were able to solve that and get down to 25 milliseconds. And the blog post they continue, however, their initial implementation caused a performance regression as they did not distinguish between open and closed maps, which they explain what that is in the article. And this regression was addressed by applying the optimization only to closed maps as they discussed in a previous article and they link to a commit where you can see that in action. The TLDR, if you don't, if you're like, wow, I'm glad that this kind of stuff is being published and written about, but the TLDR for it is they did some really cool stuff deep in how the type system works and it will work better and faster now. So that's cool. This episode is sponsored by paraxiel.io, the only security platform with true elixir support. I sat down with founder Michael Lubus to ask this week's security question.

6:11Lots of companies and teams are really focused on velocity and shipping features. And is that fundamentally incompatible with good security? No, I would say that companies that ship software regularly tend to have much better security because if you're only doing release every month or year or so in some cases, it usually means the security is very bad, packages are not being updated. You're running outdated software. When I do plantation tests, I've done them professionally, even before elixir. When you get a modern application, almost the user interface, you kind of feel when it was coded and then user interfaces old, it doesn't mean that the application is insecure, but you could tell that it's kind of been neglected. They maybe haven't been keeping up with best practices or the running out day library that you can exploit. So I would say shipping features regularly and having a very fast development pace, I associate that with better security in the majority of cases. Reach out to Michael at paraxiel.io to get personalized security feedback for your projects

7:12and your business. That's at paraxiel.io. Next up, so just following up on the technical blog post, right? There's another little bit of technical stuff about type systems. Josie did post a link or a blog post about how type systems are leaky abstractions. And so I was like, hmm, I don't know what that means. I had to think about it. I'm like, hmm, I don't know what that means. I had to go read this. It's basically talking about a story of a pull request to add a function map take bang. So we already have map take, but map take bang is what the proposal was about. The bang part of this means that it would raise if it didn't have the keys in there, right? So map take works that way. You pass in a map and you know list of keys that you want to take out of it. So it kind of just doesn't split it, but it just well takes those keys out of it and just leaves you a new map with those only those keys in there and values. Map take bang would say we'll do the same thing. But if it didn't have those keys, then you get your error.

8:14I won't rehash the whole article, but that's that's the origin story of this whole thing. And he intersects it with how types of system should work because well, structs are maps with dressing on it, right? Additional guards around it. And so what if what would happen if you put in a struct in there, you know, with the bang, what should happen? You should already have guarantees, right, that those keys should be there, knowing that it's a struct. And it talks about typescripts solution to this, which is key of, I've actually had to deal with that. It's casting via typescript is a very weird world. I never quite could understand what the heck was going on here. But we already know that typescript is a gradual type system too. It is unsound in the runtime. And Jose has repeatedly pointed to that as like, yes, this is good. And yes, it's popular, but it's incomplete. And we're trying to get a more complete theory here. Hence the set type theoretic theory that they've been espousing through this whole process. It's a very interesting read. I highly recommend that you check it out.

9:15This is not one of those blog posts that says, hey, we release something or hey, the compiler's 10 times faster now or, you know, that kind of stuff. This is just like a, did you know? And interesting to think about, there's no results on the other side. Yeah, it's a good learning opportunity. If you want to learn a little bit more about the type system and the edge cases, the things that you don't always think about, you know, basically what they're doing is hard. And here's a good blog post that describes one of those little hard points that you just really had to think on. Fascinating, right? How do you recommend it? We got a link in the show notes. Yeah, I will point out that in the social media post where Jose was talking about it, he said, spoiler alert, that the article is not about which is better dynamic versus static types, but rather an attempt to shine a light on the underlying trade-offs. And he is in no illusion that he will change anyone's mind on the topic of whether they prefer static or dynamic. So it's not about that. It's just like, hey, understanding some of this. And when you get into some of it, then you realize when you're trying to do something that would

10:17otherwise make sense, like I should just be able to make refactor my code in this way. And then the type system says you can't that's the leaky abstraction where you have to understand the internals of how that type system works just to make it so your refactoring of your code is valid when logically and runtime wise, it's all valid. So it's just interesting to kind of get some appreciation. And next up, Zach Daniel shared a new usage rules feature. So usage rules is the package that we've talked about recently that helps bring LLM rules from your dependencies into like a unified cloud MD or agent MD file. So your developer experience with the application and all of its dependencies is a smoother experience. So what his new feature is is that usage rules now supports copying skills from your package in addition to constructing skills from the usage rules files. And package authors can now provide entire skills for your project. And when you update your depth, you can get the new version of those skills.

11:19So if you've used something like cloud code where you can do like a slash and you can create custom skills. So you can say, this is how I want to create my PR comments. And this is the structure and all the rules around it. So I can just easily have a shortcut to doing a comment task. So I thought, no, that's pretty cool. And I could actually even imagine someone creating a package that has no code that is only a set of skills. Like, hey, this is like the best set of elixir skills. You know, that you could do that or you could just do that for your own company and have it as a dev only dependency. But it's a way of versioning, distributing, and upgrading a set of skills across a team. Or a group of people. So pretty cool idea. Yeah, it is nice. I wrote my first skill the other day. Not nice. Yeah. Or is it a command or a skill? I don't know what it is anymore. I think they've just kind of all merged into skills. Yeah. But I have a slash command now of ship. And so yeah, it'll run cloud codes own skill called simplify. How you recommend that, by the way.

12:20I often end up with like duplicated code, you know, repeated things or just bad practices generally. It's just slop. You know, aka slop. Simplify is their built-in skill to help with that. I think it's interesting. I haven't tried that one. The creator of cloud code uses it all the time. So he says. But then I also created two sub-agents, a elixir reviewer. It says more than this, but it basically says, you are Jose William. Review this code. Among other things. And then lastly, I do a terrible job with this of self-marketing sometimes. So the other sub-agent it'll do is a technical product manager. I guess it'll review the branch or the diff or whatever and tell me the impact of this and like what the good things are out of this. So that way I can kind of copy and paste a little bit of that like into a slack message or a PR description of that kind of stuff. Very helpful, right? Essentially anytime you are finding yourself repeating things,

13:21that's the whole purpose of these plugins and skills and all. Wrap it up in a skill. Yeah, good. All right, next up, it's just a tease. Soren two, they're handled at Parker and Shannon Silbert of OpenFame. They teased some upcoming, I presume, OpenPro workflow improvements, workflow plus web improvements actually. So we've got to link to their tease. So just to describe where they currently are with this, right? If you installed OpenPro and then OpenWeb, OpenPro gives you a whole module here for tying and correlating multiple jobs together. It's called workflows. And so you can kind of like create whatever workflow you want. This job and then that job and then that job and then fan out and then have the penises and all that kind of stuff, right? Build yourself a little dag basically, right? One of the shortcomings, though, has been seeing that represented in one place in OpenWeb. OpenWeb is open source, but it's made, I think, primarily for just jobs, right? So if your workflow has 10 jobs and they're all, you know, leading into each other in a simple sense, if they're all just serial one after another,

14:22you can't see that in one page, you know, on OpenWeb. Now comes the tease. That's going to be a solved problem. That's very excited about that. So the tease shows a new workflow search page. I'm assuming this is going to be OpenWeb, who knows, we'll see. It shows a workflow progress bar. You get search filters, the various stats on it. When you click into the workflow, you get like a graph of the nodes and by nodes, I mean like the jobs, the individual jobs. And everybody's in like a canvas, you know, with like the nodes, with the connections and all. It seems like everybody's using that nowadays. Very helpful for just illustrating the whole workflow. So wonderful that it's there. And then of course, more in-depth stats on the workflow progress and all. Very happy to see that coming up. Can't wait to throw that into my mix. My mix depths, because I need that like yesterday. So looking forward to that. When it does come out, tune in here. I'm sure we'll announce it when we see it. And next up, mdex version 0.11.6 was published.

15:27So mdex is the markdown support for elixir. It's a great library. We've talked about it multiple times. So I just wanted to highlight a couple cool things with this release. So this adds a new code fence renderers option. And it also has some fixes in the syntax highlighter. And the streaming parser to recognize more patterns and make it more reliable. So you'd want to update it for that if nothing else, just the stability and the fixes. But these new renderers are interesting because they allow you to create custom code fences. You know, code fences are like the triple back tick, markdown, or a custom one like alert, where it will be styled uniquely. And you have a separate area where you say the code fences for alerts are handled this way. But it also supports some other code fences for things like picture. It's P-I-K-C-H-R, which is a format that turns into a rendered SVG. Then there's also chart, which takes just regular SVG. There's CSV and others.

16:27So if you're looking for opportunities and ways to say, I would like to have a prettier representation of maybe CSV data in a markdown file. Hey, that's already built in through these code fences. And you can create custom ones. So I just wanted to call that out. Thought it was really neat. I'm actually installing MDX for a project of mine. So it's very cool. I've enjoyed it so far. I love all the different ways you can export the markdown, right? And then you never have to worry about syntax highlighting anymore, like whether that's happening server side or client side. I remember that being a big gap the long time ago. All right, next up, oh, oh, this is me. I released a NeoVim plugin to help with XPM completion results. This has been bugging me for years. I'm trying to get around to doing it. Here's what this does. Okay, for anyone that's not using NeoVim, ignore me for the next minute. Or maybe not. Maybe you should consider adopting NeoVim because you're either editor overlords. You have to rely on them to give you these features. With NeoVim, you can program it yourself like I did here.

17:29All right, so what is this doing? This is definitely a NeoVim plugin. And not just NeoVim, but a blank completion plugin, which is as far as I know, the most popular completion plugin. For NeoVim, and it has a plugin system for allowing you to auto-complete various sources. Things like I've got these files on my file system or LSP, you know, information, and it just, you know, gives you the typical auto-complete, you know, experience that most editors have. I added one here called Hex-CMP. CMP is complete, is short for completion. It's only enabled within the mix EXS file and leverages tree-sitter parsing to find that you are currently your cursor as in you, or as in the function called DEPS. And you're inside of a tuple, right? So convention, right? So we know that mix EXS is totally a script. You can do whatever you want in there. It could, it doesn't have to be called DEPS. You could call it FUBAR, and it would still work, right? It's a script, but conventionally, I've never seen that. It's always DEPS, right? Every generator out there is DEPS.

18:29So I think it's a safe bet. So given that safe bet, if you are in the DEPS function in the mix EXS file only, doesn't operate anywhere else. And you're in a tuple. The tuple is three positions, right? The first one is the package, the second is the fuzzy version matcher, and the third is all the options for that dependency. And so given that knowledge, via the tree-sitter parsing, which is also the popular way now, the modern way of most editors to perform syntax highlighting, but also just know where you are for more intelligence and how to jump three things. HexCMP will call HexS API for listing and searching for packages. It does all the disk caching here, so we're not hammering HexS servers all the time. Yeah, so it'll search for packages. It'll get metadata on those packages so you can see what that package even is, what it's latest version is, how popular it is, but by way of downloads. And then the second position is that fuzzy matcher, so it'll also auto-complete or list those versions for you for that package.

19:29So it's exact versions, right? Conventionally, I think everyone does the little till-day arrow one, at least version one, I don't care what patch or what minor, right? So the completion engine does its own sorting, but at least the plugin will provide it at least with the fuzzy ones, the popular ones at top. So hopefully those show up first in the list, so that's the idea. All right, and then of course the third option is all the options that the dependency can take, like runtime false, only in prod or dev or whatever. So it gives you a little bit of that intelligence as well. And it does that by acting like a lightweight little in-process LSP that provides the hover functionality. So it's both a completion engine and a little bit of a LSP, a very narrow focus to LSP. So that way you hover over the dependency you hit in NeoVem, you'd probably hit K. You hit K over it and you see the pop-up, the signature of the dependency, the row, and then also the information about the package itself. Incredible that it works.

20:29I love it. I feel so happy having that out there now. I've used it so many times. If you're using NeoVem, you got to go check it out. All right, there it is. It's self-plugged, sorry, but it's short of life-changing. Nice. And next up, Mike Binns announced that version 1.0 of Flame On, which is the Flame Charts in Live Dashboard has been released. The release is mostly just version upgrades and some minor bug fixes. But since Flame On has been out for over four years and it's been stable, they figured version 1.0 is appropriate. It's shown that it's stable. I thought that was funny. It's like, yeah, we should probably call it 1.0. So they also say stay tuned for some upcoming improvements and expansions on what Flame On can do. I've now seen some other situations where people may have been using Flame On. So it was accessible in Live Dashboard, but may have unloaded it or removed it from their project because of the version dependency issues

21:31because it wasn't being actively updated to keep on the latest updated depths. So if that's been the case for you, now would be a good time to plug that back in and be able to play with that again. All right, next up, Gleam has a new static site generator. It is called Blagato, Blagato. So here's how they describe themselves. It's a Gleam framework for building static blogs with Luster and Markdown. Oh, nice. Blagato generates your entire static site from a single configuration. So blog posts from Markdown with Front Manor, static pages from Luster Views, RSS feeds, site maps, robots, TXT files, all rendered via mod components. One thing that probably ought to be considered that baseline static site generation stuff now are LMS.txt now. Yeah, I don't see that listed here. Maybe they support it, maybe they don't, but it just came to mind because I know X-Doc recently started supporting that as well. And you see that when you generate your docs. Now you do mix docs and you see the outputs and by default it's EPUB and HTML.

22:33And now I see the LMS.txt file out there, which is pretty nifty. All right, well anyway, if you're in the Gleam ecosystem, maybe consider Blagato if you want to stay with your favorite language Gleam and still have a totally HTML artifact of your site. Looks pretty cool. And next up, Lucas Samson shared that he released a new library called Spark underscore EX. So Spark EX. And what this is is it's a native elixir Apache Spark connect client with livebook integration. If you're wondering what is Apache Spark TM, from the website it says Apache Spark TM is a multi-language engine for executing data engineering, data science and machine learning on single node machines or clusters. That's the niche. It's filling. This is something that helps you hook elixir into that. And I just thought it was funny because all over that website, they put a little Apache Spark TM, like every opportunity they could find is Apache Spark TM. So he should have called it like Spark TM EX or something,

23:36you know, just to... But we've got a link to a livebook demo notebook where you can quickly start to play with that and experiment with that. If you're using Apache Spark, that'll help work into the flow that you're wanting to play with. All right, next up, quicky. Livebook desktop now supports Linux. Yay. We've got a link to the announcement from Hugo Barat-Una. We reported about this last week. I think about a tide wave for what it's worth. So we won't repeat too much. But short version is that they're using, I don't know what it's technically called, but it's called Tari. And Tari is just a way to package an app to be cross platform. So tide wave, I think, did that first. And now it's happening in livebook desktop. So if you're a Linux nerd, you can have a livebook desktop now. Yay. And next up, easel can now render to the terminal as well. So if you remember, easel was the thing we talked about that Jason Stebes created, which you are creating graphics on the Phoenix Live view side. And then you're able to render that to WX,

24:39like WX Windows widgets or to HTML. And we were talking about it with in context of being able to play with the Boyd's B-O-I-D-S, the little life simulator with basics, little rules. Well, Jason wasn't done having fun yet. So he said, I'm going to see if I can make that render to the terminal. And that's exactly what he's done. It's just kind of interesting how he's accomplished this. So he's rasterizing off screen while using easel WX rasterized. So it is still requiring the WX library and runtime requirement to be able to do this. Once he's rasterized it, he extracts color silhouettes and fitting printable ASCII glyph masks per cell. And then he writes frames through a library called termite that handles low-level terminal APIs. Pretty interesting. So it also requires that you have an interactive TTY and that is run from a real terminal session to be able to actually have it render there. I haven't seen what this looks like. Have you seen this in action? What is this like?

25:40I haven't seen an action, but they've got a screenshot of it, our little video. Yeah, it's pretty funny. It looks pretty cool. Oh my gosh. It's quite a pipeline. Yeah, it's quite a pipeline. Yeah. So I'm just looking at this and think, it's like the Boyd simulator in ASCII characters that change in everything to find a best fit blended map. Like it's crazy. You just look at this and you think, wow, what else could I do with this? That could be interesting. Yeah, I wonder what performance is like there. If it has to rasterize everything like that. Anyway, true. Very cool. All right, last up. There is a new Elixir podcast on the block. It was started by Francisco Cesarini of Irling Solutions and Alan Weima. Alan Weima, I believe, has run another podcast already. So they're just teaming up and creating a new one. So we've got some links to their show in case you might enjoy that. It's called Beam, they're done that. Very nice name. Little jealous.

26:41So they launched their first episode where they talked to Andrea Leo Party about concurrency, OTP and the evolution of the Beam. And we've got links to all these to Spotify and Apple podcasts in the show notes. So you want to try a different pair of voices? Go try theirs. Well, that's all the time we have for today. Thank you for listening. We hope you'll join us next time on thinking Elixir.