The Road Ahead / Where’s This All Going

Featured

Hey all,

It’s been a while since my last post. As promised, I’ve spent the time tending to a variety of personal matters which I neglected during my seclusion producing ModVB Wave 1. One thing I got to do thankfully, was my semi-annual trip to Seattle/Bellevue/Redmond, which I say is for PAX but is really just an excuse to reconnect with my many friends there. I was super happy to see as many of them as I could in that limited time and to catch up with what’s been going on in their lives and catch them up with what’s been going on in mine.

I’m happy to share more of my personal exploits later but most relevant to this post is an individual interaction I had with a friend about my efforts around ModVB. Specifically, he asked (paraphrasing), “Is there are general theme to your future work? Is this all building around one big feature or what?”. Or, in my words, “Where’s this all going?”.

Which is an excellent question. I’m a mysterious hermit who lives in the woods of the South Side of Chicago and periodically I just show up and say some ominous stuff and then disappear. This summer I dropped some language features on you and crawled back into my hole for a bit. All fun, but maybe not as comprehensive as this effort deserves. Fortunately, I have (and have had for years) a massive itinerary (just outline of the language improvements is 10 pages to date) in my head and in OneNote and in Word docs that I’ve been building and following that could easily take a hundred or more of printed pages to detail so I’m not at a total loss for an answer.

It’s just a matter of giving you the forest and not the trees and so I set about sitting about and distilling that roadmap into this post. Hopefully it gives a better idea of the scope of things to come.

The VBest VB that ever VBed

No, seriously, if you take all of the good stuff that VB has been over its life and follow that throughline to its greatest expression in all aspects of the end-to-end experience without compromise, that’s what I’m trying to build. I don’t want there to be any lingering questions of what VB could have been if only things had been different. I want to build that thing, or at least my interpretation of that thing based on decades of experience using it, analyzing it, designing it, and communicating with other community members about their experiences, desires, and needs while using it too.

Still have questions about what that means or what it looks like? Well, over the next 1-2 years of building it I expect to design, discuss, initiate, or produce enough artifacts that you don’t. That’s the journey. Now let’s get a little more concrete about 3 layers being built.

  1. Core Language Experience: I want to build a massive, robust extension and modification of the vanilla VB.NET language and tooling that evolves, modernizes, and refines the language for current and future enthusiasts in a self-consistent manner.
  2. Extended Tools & Experiences: I want to build additional tools to enhance the VB experience beyond the core modified language to support certain platforms, technologies, and scenarios.
  3. Community: I want to build a self-sustaining community of VB enthusiasts around this modernized end-to-end experience.

These three layers make up the VB.NET UX I’m trying to build: language, tools, community.

Building for the enthusiast, not the user

Notice that I keep using the word enthusiast rather than user. Anyone can be a user just by using a thing. They don’t have to like or love the thing or have any particular attachment to the thing. They may even hate the thing. I’m not talking about those people. There is no victory to be claimed in planning anything around an audience that isn’t at some level already bought-in or capable of buying in to what you’re building. A VB enthusiast should feel like a kid of Christmas morning about this endeavor: they may not know what’s in the giftwrapped boxes but they’re excited. I want to justify and grow that excitement. If you don’t care, that’s fine, you just probably shouldn’t be reading my blog and I definitely shouldn’t be trying to court your approval. Personally, I believe it would be healthiest for the VB community to as much as practical consist of enthusiasts rather than users. This doesn’t mean they aren’t “real developers” or that their projects aren’t “real apps” and it doesn’t mean (primarily) people who aren’t getting paid to build software, but that the ones who are getting paid are enthusiastic to come into work and work in VB every day.

For a long time due to its popularity with businesses VB has accrued all sorts of users, including an unfortunate lot who were forced to use by circumstance. The problem is that group produces an awful lot of noise when planning and prioritizing feature work because ultimately they want to use something else entirely and they want VB to just become that other thing. I want VB to be a language of choice, not force, and to cultivate a community of folks who choose VB because the “Spirit of VB” resonates with them and empowers them. I can wax philosophical about the “Spirit of VB” for pages (and I know I must do so soon) but for the sake of some brevity I’m going to try to keep this post a little more concrete.

Breaking down the VB experience

Back in my Microsoft days, once a year during MVP Summit I’d get a chance to meet with the brightest minds in the VB community (diehard enthusiasts and absolute professional badasses) and present an early preview of what we were working on and get their feedback and hear their pain points from real world projects that they wanted solved in the next version (hopefully). In the later years, as lead language designer, it fell on me to not only participate in the summit (which I loved doing) but to present Microsoft’s side of the conversation for VB. I took to a format that I styled the “State of the Language Address” whereby I broke the VB experience down into four areas of concern for the VB audience:

  • Platforms & Technologies
  • Language & Runtime
  • IDE & Tooling
  • Community & Content

For each area we’d summarize the feedback we’d heard from the community over time, how we’d reacted to their feedback, and what our next plans were for the area. Having found this a useful structure in the past, I will frame the rest of this post around these four areas. Typically platforms & technologies would be the top-of-mind area for MVPs and I still think it’s the most impactful but for this post I’m going to lead with the language because a key priority for my language modding efforts is investing in features which enable broader platform reach, particularly in Wave 2.

Language & Runtime

As I mentioned earlier, I have a 10-page outline of just the language modifications I intend to produce over the next… 18-months(?) or so. Some are bigger, most are small, all are doable and I believe genuinely in the spirit of the VB language; they fit with the values VB has articulated thus far and will be designed and added in a manner which is natural and self-consistent.

Originally, I broke this outline into 4 major categories:

  • Modernization—New functionality added to service modern scenarios or modern platforms. E.g. XAML Literals, JSON literals/pattern matching.
  • Evolution—Enhancements that extend an existing construct to its logical “next step”, e.g. query operators in `For Each` loops.
  • Streamlining—Reduction and in some cases elimination of boilerplate/ceremony in common scenarios or polishing “rough edges” in a scenario, usually through some addition to make the language more self-consistent, e.g. type-inference improvements.
  • Bugfixes—Changes made purely for self-consistency, correctness, interoperability, or code-optimization, e.g. new implicit line continuation, VB runtime performance improvements.

In addition to language changes I intend to release a modded VB runtime with new support types and other enhancements as the language evolves. For the time-being I’ve baked these changes into projects compiled using ModVB as embedded code a la VBCore.

Those categories still hold true but I’ve since found it more practical for communicating with my readers to describe them in terms of time, specifically 6 or more “waves” each with a particular theme. Each wave is not exhaustive—every conceivable feature under a given theme won’t be added in that wave. Rather, I have prioritized certain features in that them around the need for feedback, the risk of adding them, size (time), and impact (the usual suspects). I’ve summarized the first 5 waves and beyond below.

Wave 1 – JSON

The first wave released this summer and was centered around JSON Literals and Pattern Matching.

JSON Literals & Pattern Matching

This wave was intended to be short and to prove out the possibility of shipping language features via VSIX and NuGet at all. It is but a prelude—an amuse-bouche—to the full scope of language enhancements that are planned. In fact, it’s not even the complete story I imagine for JSON or pattern matching, both of which have deeper integrations on the backlog. Having said that, it was successful at its purpose and an update based on feedback should be released before Wave 2.

Wave 2 – XML

It is not my intent to be hyperbolic when I say this. Wave 2 contains the most important features that could ever be added to VB.NET. Seriously, if just these features were added and nothing else it would constitute a full step forward for the language over where we are now, though fortunately we won’t have to make that trade. The core feature of the wave are enhanced XML literals (every day I wrestle with calling it XAML literals though it just as well could be called HTML literals too), a feature I prototyped back in 2019 but am now rebuilding The Right Way™ along with the support feature of top-level code.

As demonstrated in these videos, these two features combined create a story for VB.NET building modern apps for web and mobile using ASP.NET Core and Xamarin.Forms .NET MAUI and untold other DSLs for other technologies.

ASP.NET Core Web Programming with VB.NET Pattern-Based XML Literals
Client-Side VB.NET in Browser with VB.NET Pattern-Based XML Literals
Cross-Platform Mobile Development with VB.NET Pattern-Based XML Literals

I do have work to do to account for the changes from Xamarin.Forms to .NET MAUI though from preliminary investigation MAUI is an evolution of Xamarin.Forms and adheres to the same design patterns. Libraries implementing the XML patterns for web and MAUI are being developed in lockstep with the patterns themselves. I have not had a chance to deep dive into the story for Blazor but the F# technology Bolero which enables F# developers to make use of Blazor tech to bring F# to the client-side gives me confidence that the system is open enough for VB to also benefit from Blazor, provided the right experience in the language. Alternately (or additionally), the OpenSilver project (which provides a plugin-free open implementation of Silverlight based on modern web tech) may also be a great opportunity to bring rich modern web application development to VB.NET!

This is the approach F# uses to reach new platforms and it works. I’m determined to use it to bring a high quality web and mobile story to VB.NET as part of the ModVB project.

I believe this work, especially when combined with the Roslyn scripting API (which will one day be completed for VB if only in this mod) will unlock the ability to create UI previewers/designers and other tools.

A detailed deep-dive into the design changes I’ve made since I wrote those initial prototypes as well as the libraries I’m writing alongside the features as part of Wave 2 to support web development and .NET MAUI will be the topic of a future post.

Wave 2 is planned for release around December 2022.

Wave 3 – Productivity

A grab bag of 4-7 productivity features I’ve been itching to add and demonstrating remarkable self-control by not adding. They’re all small to medium in size compared to waves 1 and 2 and will likely be released one at a time. I simply need to get these things out of my head.

Wave 3 doesn’t have a forecasted delivery window though waves are intended to be 2-3 months in duration.

Wave 4 – Async

I vacillate back and forth on whether this and wave 5 should be swapped. Today it comes first because the LINQ related features are all pretty straightforward and making any changes with Async deserves plenty of bake time to ferret out bugs and react to feedback. There are several missing pieces to VB’s Async story, e.g. an `Await Each` construct, `Async Iterator` methods, `Await` in `Catch` and `Finally` blocks, and more. Not every feature will be included in Wave 4 for time reasons but by the v1.0 release of ModVB I plan to address them all.

Wave 4 is after Wave 3.

Wave 5 – LINQ

There are no surprises in Wave 5. It’ll include deeper integration of LINQ into the language and revisiting several features added as part of the initial LINQ release way back in 2008. For example:

  • Integrating query operators into the For Each loop.
Extending VB.NET For Each loop with LINQ comprehension operators
  • More expressivity in object member and collection initializers
  • Range expressions
  • Possibly more query comprehensions

Wave 5 is after Wave 4.

Wave 6 and Beyond

OK, so technically there isn’t a “Wave 6” so much as everything else is post Wave 5. This is where the less glamourous or most difficult work goes. Bug fixes. Optimizations. And one or two … high-cost/risk features like Nullable Reference Types live. This is the long tail of quality-of-life features, advanced features, and edge cases (many of them long requested by the VB community) that will easily eat up 60-80% of the time on the project. I may try to plan several formal waves or I might just loop through the list picking off low-hanging fruit until it’s all done.

So, for example, for ever VB users have been asking for Modules that don’t automatically promote their members to their parent namespace. I absolutely will address this request after over a decade of fielding complaints about it. It’s not super impactful or risky, so it’s just “on the list”.

A Note on Language Stability: My plan isn’t to release a million features every year. Many of the features alluded to above are long overdue and if I’d had my way would have been completed years ago. For this reason, I’ve built up something of a backlog. But after the v1.0 release of ModVB, I expect language iteration to comparatively slow because it’s simply not necessary to shatter the earth with every release of a language. I intend to code in VB.NET until they pry it from my cold dead hands. But I don’t intend to try to re-invent the fundamentals of programming every release in order to keep up with any other language; every major release won’t be Generics, LINQ, Async, or even dynamic. I truly believe that with the set of features I’ve collected VB.NET will be in an incredible spot as a language and will need relatively few new general purpose features. Instead, I intend to pivot to more evolution on the end-to-end tooling of VB.NET to recapture as much of that magic that’s always kept the language ahead of the pack. I say all of this not to dampen expectations; this isn’t a policy thing and I’m happy to be proven wrong. Rather, I say this to give the more cautious developers assurances that I’m not going nuts and throwing everything and the kitchen sink into the language. It’s not going to look unrecognizable when ModVB v1.0 releases and I intend to keep the pace of change sustainable after that.

Platforms & Technologies

As mentioned above, my immediate concern for Wave 2 is unblocking ASP.NET Core and .NET MAUI. In addition to the language features described in Wave 2 I will be shipping runtime extension libraries for .NET MAUI as well as a brand new VB-targeted web programming library and several templates for using both. Blazor will hopefully come at a future date. I might do a proof of concept for a web UI designer sometime after Wave 2 is complete but I don’t expect any serious pursuit of such a tool until after v1.0 particularly as this may require completion of the long in limbo Roslyn-based VB.NET Scripting engine.

Other technologies such as Unity are interesting to me because they’re interesting to VB enthusiasts though it isn’t an urgent concern; happy to collaborate with anyone who’s more fluent in that space about their needs or opportunities to light something up here.

IDE & Tooling

In addition to the core support for new language features in the VS editor I also plan a slate of VS extensions to supplement the VB experience. In fact, I’ve already started development on one that’s near readiness for a release but I’m pausing work on it while I finish Wave 2.

I’m very interested in partnering with anyone who has the know how of how to plug ModVB into a .NET Interactive/Polyglot Notebook.

Long term (as we approach ModVB v1.0 release—late 2023/early 2024) I expect to pivot toward looking for a cross-platform IDE for the modified VB language. The open-source F# Ionide and C# OmniSharp plugins for VSCode should provide excellent instruction for where to begin here and again I’d love to collaborate with anyone with expertise and interest in this area.

Community & Content

As I mentioned above, my endeavor is not merely to create a modified version of the VB.NET language but a revitalized VB.NET community around it. To that end, in early 2023 I will be launching a new community web site to promote and educate VB enthusiasts about a variety of topics, including this and other mods, news, and other tools in the broader VB.NET ecosystem. I would like to make this new site a gathering place for VB content from other community members with an aggregate blog, forums, and technical articles. The big question now is whether to roll something out based on an existing web hosting provider like WordPress or roll my own using the ASP.NET Core support I’m building in Wave 2.

Wrap up

I’m excited to be working to bring new capabilities to the community, not least of all because I love VB.NET and absolutely want to use it on all of my future projects, personal and professional. I’d love to talk more often and in greater detail about the design process I’m following but it’s hard. My (unsustainable) process has thus far been to quite literally seclude myself from the world and focus (and re-focus) myself relentlessly toward the next goal neglecting everything else in my life.

When I wrote “An Exhausting List of Differences Between VB.NET & C#” I literally hermited myself in my apartment for 5 or 6 straight weeks where I did little but eat and write. I didn’t even do laundry or clean my home. The journey to Wave 1 this summer was very similar; I was literally buried in trash recycling by the end of it. I have no work-life balance, I think, because of my mental health struggles. I also have mental health struggles because I have no work-life balance. To do this project I have to wear every hat in the development process from PM/dev/test to marketing and docs. I have the experience to wear those hats but not all at once and switching modes is especially challenging. It can just be so hard to focus and stay on task and I build up so much guilt and self-loathing about not being faster or more consistent or more communicative. So often I just want to crawl into a hole and stay there. But when I muster the strength to crawl out of that hole all I can think about is all the wonderful stuff I want to build with and for VB.NET. Even in my depressed moments that’s what I’m thinking about. I realize, this is my life’s work.

I bring that all up to say, this week after dodging it for 30+ years I was diagnosed with ADHD and have been prescribed some meds to help me focus. I’m really really hoping this new chapter in my mental health journey will let me find more balance and that will be good for me and for you all. Thanks for your support and continued patience.

Look out for…

  • Wave 1 Update 1 soon
  • Wave 2 circa December 2022
  • More design discussions from me sprinkled throughout

TL;DR

Everything we need is doable. Much of it has been done before. There will come a time for almost everything. We may be taking steps back but when we leap forward I believe we’ll be in a much stronger position than ever before.

The answer to “Where’s this all going?” is… Everywhere.

Regards,

-ADG

Introducing ModVB—A free, independently-created “mod” for VB.NET that gives you new features just by installing a VSIX…

Featured

…and referencing a NuGet package. That’s it. That’s the whole thing.

As mentioned in my previous post, I think modding represents one path forward for my favorite programming language, VB.NET. So, I’ve created my own mod to help me explore the years of ideas I’ve had for VB and to share them with others!

ModVB

ModVB is more than just an unimaginative name. Yes, it means “modified”. But it also means modern. And modulo (with respect to).

And the mathematically inclined will note that: (VB + ModVB) Mod VB = ModVB

(Assuming ModVB is less than 2x VB, which it is). You see, I really did put a lot of thought into this!

The first wave of new functionality is introduced in the very next post on this blog: ModVB “Wave 1”—JSON Literals & Pattern Matching. Please check out the post and the accompanying video for more details on what you can do with this first pre-release version. I’ve labeled it an alpha because it’s very early in the development of the mod and semantics will likely change. That said, I’ve built it for stability so crashes or hangs should be very few (and please report them if you find them so I can fix them ASAP).

Look forward to future posts from me about how you can make and distribute mods of your own!

About the Developer (Me)

A man in a suit (auto-generated)

My name is Anthony D. Green (he/his/him). I’m a 37-year-old VB developer from Chicago, IL, USA. I previously worked at Microsoft for 8 years on the Managed Languages team. For most of that time I was the program manager for the Roslyn compilers; communicating with customers and working with Microsoft architects, developers, and testers on designing and implementing the VB.NET and C# compilers in VB and C# respectively as well as a large part of the Roslyn APIs that power the VB and C# IDE experience inside Visual Studio.

For my entire Microsoft career, I served as a language designer on the VB language design team, which I led for 2 years, during which I also served ex officio on the C# language design team. Both before and after my time at Microsoft I’ve worked as a software architect consultant in Chicago for a total of 5 years, mostly working with .NET and other Microsoft technologies.

VB.NET is my passion and I’ve enjoyed working with it both as my profession and as a hobby, starting with VB4 when I was just 13-years-old. I’m building this mod mainly for myself because I want to see the language and experience evolve to their limits but I hope others will get benefit from it as well because I really want to see the VB.NET community grow and flourish. To everyone who chooses to join me on this journey, thank you for trying my mod!

Installation Instructions

This process is easy solely thanks to the heroic efforts of the Roslyn infrastructure teams throughout the years. I just implement the features; they made it possible for me to share them with you.

Prerequisites

  • Visual Studio 2022 (any edition, including Community) and VS2022 Update 17.3.1

Steps

  1. Launch Visual Studio 2022 in a new hive

This creates an isolated configuration of Visual Studio 2022 that you can install the mod into. ModVB is currently in pre-release and installing into a separate hive means you can always launch Visual Studio 2022 normally if you encounter bugs or blocking issues or simply want to return to the behavior of “Vanilla” VB.NET. Even when the mod eventually releases you may wish to use separate hives to maintain this flexibility.

  • Open the Visual Studio 2022 Developer Command prompt (Start > “Dev…”)
    • Enter the command `devenv /rootSuffix ModVB`
  • Click “Continue without code” (bottom right)

You can always confirm when you’re using the ModVB hive because “MODVB” will appear in the upper right corner of Visual Studio 2022 when you’re using that hive.

  1. Add the ModVB MyGet VSIX feed to Visual Studio as an additional VSIX gallery
    • On the menu, open “Extensions > Manage Extensions”
    • Click “Change your settings for Extensions” (bottom left)
  • Click “Apply” and then “OK”
  1. Download and install the ModVB Language Services extension
    • On the menu, open “Extensions > Manage Extensions”
    • On the left, select Online > ModVB
    • Download the “ModVB Language Services (Built with Roslyn)” extension
  • Close Visual Studio 2022
    • The VSIX Installer will open. Follow the instructions in the installer to complete the installation
    • Re-launch the Visual Studio 2022 ModVB hive by entering `devenv /rootSuffix ModVB` at the Developer Command Prompt. You may wish to create a separate Windows shortcut with this option for convenience.

Installing the language services VSIX enables any new or modded IDE experiences inside of Visual Studio 2022, including the IDE experiences for new language features such as IntelliSense, syntax highlighting, and code actions. However, for building projects using new or modded language features and semantics you’ll need to reference the ModVB compiler from those projects. This also enables those projects to build as part of your CI/CD process on build servers which won’t have Visual Studio 2022 or the ModVB Language Services VSIX installed.

  1. Add the ModVB MyGet NuGet feed to Visual Studio as an additional NuGet package source
    • On the menu, open “Tools > NuGet Package Manager > Package Manager Settings”
    • On the left, select “NuGet Package Manager > Package Sources”
  • Click “Update” and then “OK”
  1. Install the ModVB.Compilers.Toolset package to any projects you want to compile using new language features
    • On the menu, open “Project > Manage NuGet Packages…”
    • At the top, select Browse
    • Select the ModVB package source in the drop down at the top right
      • Make sure “Include prerelease” is checked (next to search box)
    • Select “ModVB.Compilers.Toolset” on the left
  • Click “Install” on the right

Congrats! Your project will now build with the ModVB compiler and have access to new features for VB.NET. Of course, if you’re not using these features the behavior of building will be exactly the same as before. Remember to add this package to any additional projects where you wish to use new functionality. Happy coding!

Next steps

  • Read this post detailing the new features in “Wave 1” of the ModVB alpha
  • Try out the mod—do anything you’re interested in doing with the new features
  • File bugs at https://github.com/modvb/backlog/issues
  • Feedback and discussion on my blog, your blog, Twitter, YouTube, or wherever you hang out with other VB.NET enthusiasts!
  • Don’t forget to SHARE!

An Alternate Path

Featured

It’s funny how a tiny bit of feedback can change the nature of a post. I had originally planned to write about modding in the gaming community and the impressive accomplishments independent developers and enthusiasts have had in expanding and extending the experience of games, sometimes vastly, beyond the intent of the original creators (e.g. Breath of the Wild: Second Wind). In some cases, games have come to be defined by their modding culture (e.g. Skyrim) and in others mods have taken on a life of their own to become products that arguably overshadow their base games (e.g. Counter-Strike) or even spawn new genres (e.g. Dota). I had some anecdotes about the first software I ever “modded” (Mavis Beacon Teaches Typing) or twiddling with QBasic games’ source code as a young teen.

That said, I got a bit of feedback after my last post that something maybe a little broader (yet also more direct and specific) would probably be better received so I’m going to try to fumble through distilling at least some of two years of thought and research and—yes, planning—into an elevator pitch. Well, maybe not that short, but still a challenge of brevity.

For starters, let me set a few expectations. Firstly, what I’m proposing is without a doubt going to be a pain in the ass, at first at least. It’s a lot of work. When I say a community-driven­­ path forward, I don’t mean a community-accentuated path forward. I don’t mean community-supplemented or community-enhanced. We need to shift our entire paradigm from being a Microsoft-centric developer experience to one where almost (*) everything we enjoy comes from us. That means features, tools, designers, docs, samples, templates, and even the respect that I mentioned in my last post. These are all things we’re used to getting from Microsoft and lamenting when we don’t get them from Microsoft and complaining to Microsoft about it. That entire inclination has to die and we have to rebuild a community that holds itself primarily accountable for its own capabilities.

This is not going to happen overnight and it’s not going to start just because I say “go”. I’ve seriously thought through the process, and I can’t see it taking less than a year or two to build out an experience with the kind of polish that can sustainably grow our community. But virtually all the technical pieces we need to build… well, anything, are available somewhere. The extensibility points are there; we just have to do the work both functionally (in code) and culturally (in ourselves). If you were looking for “ThatVBGuy” to post on his blog with some tablets in hand explaining how with a few reg key fixes I can make this all go away, sorry, this ain’t that post. We’re talking 3 steps back, 5 steps forward.

Though technically we’re not sliding backward, we’re just not moving forward right now, so I guess it’s just 5 steps forward while the world is moving forward like 2-3 steps maybe but we’re going to accelerate so that by the end of it we’re far ahead of where we are and even ahead of where we would have been if we just floated along with the current. Yes, we’re in a boat now even though when I started this parenthetical I was thinking about a land-vehicle like a car or a train.

I also want to be clear that not everybody is going to make it to the other side (we’re back on land now, the metaphor is some kind of chasm, I think). It’s completely foreseeable that a lot of us are going to take a faster, easier path and move to a developer experience that’s already propped up elsewhere. There are always going to be those who need a Microsoft-sanctioned/supported solution and either can’t or won’t give up the Microsoft-centric mindset. And that’s okay! I just can’t be one of those people and I’ll explain more about my personal investment in this (other than quitting my job to focus on kickstarting this effort for the next few months) in my next post.

Now, I know I’m not the first to say this is what needs to happen. Everything I’ve said so far will need to be said again, and expanded upon, and most importantly discussed and debated amongst ourselves—we few. So I have to imagine when people say, “Well, what’s the alternate community-driven path?” they already get that the path is to be community-driven. What they really are asking is “How?”, or “Where do we start?”, or “What can I do?”. Maybe they want a roadmap to give them assurances or confidence that this can really be done or how likely it is to be done in a way and timeframe that is applicable to their projects and needs.

For starters, what can you do as a VB.NET developer who wants to help?

Get comfortable with discomfort. We can build some great experiences but first we’re going to build some pretty raw ones. If the minority of the larger VB community that cares to push it forward can’t suspend our discomfort with doing things in a way that may seem initially hacky, we’re not going to get to the place where the majority can follow. Remember that most other languages outside the Microsoft ecosystem are built up like this. VB has enjoyed a long period of being the exception. Now we must rise to meet the rule.

Understand the systems that exist today. You’ve used Visual Studio and .NET but do you understand the extensibility points? Have you ever built a VS editor extension? How well do you understand the dotnet command driver? How’s your NuGet package authoring skill? Do you truly understand how new UI platforms work under the hood? Have you ever written a .NET Analyzer? A source generator? This is the time to pick an area of interest and study up. We’re all going to have our areas of strength and no one person can know or do it all.

Be ready to engage. It’s easy to be attracted to the allure of being the one who comes up with or builds exciting stuff. But what gets projects done and done right is feedback. I’m going to post a lot on my blog as things progress. Be ready to share your opinions and discuss with others. Be ready to test things that are built by others. Roslyn is arguably the largest VB project ever built and it crossed the finish line not just because of smart devs churning through features but equally smart testers constantly testing the bits. Giving feedback on the design. Filing bugs. Fixing bugs. Building projects with what was ready. I hesitate to quantify it but an amazing tester filing bugs is worth maybe 5+ good devs checking in features.

What’s Anthony doing?

First up, I’m building a mod as a proof of concept. This is a different animal than the prototypes and stuff I’ve built so far (think more Resharper than io.js). I’m targeting this month (June) so I plan to have something more concrete to talk about in the next two weeks. Aside from the at-times agonizing process of authoring these posts, this is now my full-time (unpaid) job. And as I learn more, I hope to educate others about how to do the same and begin to coalesce our identity and efforts into everything I can see in my mind’s eye that the VB experience is capable of being.

Need more info?

Sound off in the comments below!

-ADG

Autumn 2023 Update #2

Announcements

OpenSilver 2.0 adds VB.NET supportthis is AWESOME! Please go show some love to those who support our community ❤️

@DualBrain has set up a VB Discord server for all dialects of VB, Classic and .NETJoin Today!

Hey all,

In my last post I said I might be finished in November. That didn’t happen and the publication I’ve been working toward is now coming up on 200-ish pages. I’m aiming through editing to keep it in that range (for comparison, the entire VB language specification is < 700 pages). Editing has been a mix of expanding some chapters and reducing/combining others. I’m really liking the improvements to flow now with each section having 1-3 deep chapters (like XAML literals) and the rest be shallower chapters. I know this document is huge so I’m really trying to keep it from being boring or monotonous. I’ve added a section (technically I moved some chapters from other sections to a new section). I am considering (no promises) a stagger publication of one section at a time, but I haven’t yet committed to that approach.

A friend has convinced me to come up for air this month to get some other things in my life ready for the new year so I won’t promise this monster of a publication will be done this month. I’m still working on it but I’m trying to be more realistic about other end-of-year competition for my time, including holiday time with family.

Happy Holidays and Warmest Regards,

-ADG

Special Thanks to ANYONE who is donating or has ever donated to me via Patreon! I could not pursue my dream of producing modern VB features and content without their support.

Dr. Donna M., Alex T., Kirill O., Neal G., Govert D., Dr. Anne J., Taiwo A., Jonathon M., Peter B., Kevin R., and Paul C.

Autumn 2023 Update

Announcements

OpenSilver 2.0 adds VB.NET supportthis is AWESOME! Please go show some love to those who support our community ❤️

@DualBrain has set up a VB Discord server for all dialects of VB, Classic and .NETJoin Today!

Hey all,

It’s November already. I’m still plugging away at the ModVB Language/Runtime/Compiler backlog reveal. I’m currently working on the section on “Null and Nothing”. This was the last section to be added explicitly and is the least or second least developed. After this I just need to pump out a few pages on performance and interoperability and I think I’m on track to drop the massive collection of posts this month.

In the meantime, a community member over on the VB Discord server asked for a quick reminder of the highlights of what’s planned for ModVB. I paired down my documents to something … highlight-y and I thought others would benefit from the summary. The exact order of sections is subject to change but the spirit and highlights of each are below.

Remember, ModVB is not one feature or a few features but a focal point for the continued evolution of the VB.NET language, based on the Roslyn codebase. With regard to actual capabilities/features/fixes, there are currently over 100 distinct work items on the backlog in the language, compiler, and runtime library, broadly organized into a dozen or so buckets (subject to change). Please understand these highlights are nothing approaching exhaustive.

UI, XML, and XAML

Enhancing VB’s existing XML capabilities for modern front-end development targeting the web and mobile.

Highlight:

  • XAML literals
  • + at least 5 more!

JSON

Replicating VB’s amazing XML story with the JSON data format.

Highlight:

  • JSON literals and JSON pattern matching.
  • + at least 4 more!

Smart Casting and Pattern Matching

Features for testing types and shapes of data and inspecting complex data structures.

Highlights:

  • Checking a local variable/parameter with TypeOf will cause its type to change where the compiler can infer it is safe.
  • The top-requested VB feature: Select Case TypeOf
  • General pattern matching against types, tuples, wildcards, strings, and other user-defined named patterns.

Streamlining and Boilerplate Reduction

Does what it says on the tin.

Highlights:

  • Key Fields & Properties, and Auto-Constructors
  • Top-Level Code
  • Type-Inference improvements
  • + at least 10 more!

General Modernization and Evolution

Revisiting almost every statement in the language and a few other constructs to improve them in some way, long requested features, consistency fixes, etc.

Highlights:

  • Tuple deconstruction in several places in the language such as local variables, For Each loops, queries, etc.
  • Select Case on object identity.
  • “Closed”, Generic, and Nestable Module declarations.
  • + at least 15 more!

Declarative Programming and Code Generation

Features which make it easier to reuse functionality across repetitive tasks both with and without source generators.

Highlight:

  • Source Generator support features
  • + at least 2 more

Asynchronous Programming Enhancements

Deeper integration of asynchrony with the language.

Highlights:

  • Await Each blocks
  • Async Iterator Functions
  • + at least 4 more!

LINQ Enhancements

Revisiting LINQ after 15 years with new features and fixes to make working with data even more productive.

Highlights:

  • Queries in For Each (and Await Each) blocks.
  • New operators
  • + at least 7 more!

Dynamic Programming  Enhancements

Revisiting VB late-binding to make it work better and work in more places.

  • At least 11 topics!

Interfaces, Inheritance, Extensions, & Delegates

Making OO fundamentals more productive.

Highlight:

  • Implicit Interface Implementations
  • + at least 6 more!

Null and Nothing

Making working with the reality of nulls in code safer, more elegant, and more consistent.

Highlight:

  • Nullable reference types
  • + at least 3 more!

Performance and Interoperability

Improvements to let VB developers write faster code for performance-critical components as well as interoperating with modern features of the .NET platform.

Highlight:

  • ByRef Structure story
  • + at least 4 more!

Versioning

Better features for dealing with or initiating change between library versions.

Runtime Library Enhancements

Revisiting the VB runtime library so that it’s more portable, more performant, and more useful.

Highlights:

  • Runtime helper optimization
  • My namespace modernization
  • + more.

Summary

If you were keeping count that’s over 100 separate topics! After I finish writing I’ll move things around some to make things flow better and I’m trying to combine certain very similar topics where it makes sense and each topic may be anywhere from half a page to 10 pages for the largest topics so it’s not an exact page count but as promised, this “post” will be massive. My next update should be that the artifact is “text complete”. See you back here soon-ish.

Regards,

-ADG

Special Thanks to ANYONE who is donating or has ever donated to me via Patreon! I could not pursue my dream of producing modern VB features and content without their support.

Dr. Donna M., Alex T., Kirill O., Neal G., Govert D., Dr. Anne J., Taiwo A., Jonathon M., Peter B., Kevin R., and Paul C.

Update on the Colossal Backlog Publishing Endeavor

I need every drop of writing power in me to go toward writing this monument so I’ll keep this short. Per my last post I’m taking my 12-page outline for the ModVB language enhancements and expanding it out into a high-ish level conceptual overview of each enhancement with a short-ish motivating scenario and/or example of each. I hoped to be done by end of July. That was pure foolishness. As it stands this overview will definitely be over 100 pages printed!

Unlike my original Exhausting List of Differences Between VB.NET and C# post, this won’t be a single monolithic document. To make it easier to publish, revise, read, comment on, link, etc. instead I’ll be publishing each enhancement as a separate post. They’ll all be published at the same time and linked from a master table of contents (and maybe an index) which will be the “post”. Right now that means something over 110 (and counting) separate posts each averaging about a printed page, organized into 14 sections (and counting).

I’m trying to keep them all at a similar level of detail which is more than “I got an idea”/suggestion on GitHub/UserVoice level but not as detailed as a feature speclet that a inexperienced compiler dev could immediately start work off. I have several reasons for not wanting to publish each post one at a time over the next 4 months, one of which is that it would mean publishing each post one at a time over the next 4 months. But also, doing it this way will empower us to have a much better conversation about the entire scope of the backlog, prioritization, and how and where it could make sense for future contributors to pitch in.

Sorry it’s taking so long, but this is the culmination of 20 years of professional experience working in and on VB.NET, we’re all going to have to be patient with it. Thanks!

Regards,

-ADG

Scaling out

Announcements

  • Long-time VB MVP Cory Smith has set up a Visual Basic Discord server for all dialects of VB including VB.NET, Classic VB(6), VBA, and VB Script. All are welcome! Please consider joining today!
  • Several supporters have mentioned that Patreon is blocked in some regions of the world and that would be supporters would appreciate an alternative. I thank them for bringing this to my attention and am investigating additional platforms for crowdfunding, though more importantly I’m planning for more ways to contribute.
  • VB support for consuming C# required members has been implemented and is expected in VS 17.7!

Intro

Hey all,

You’re loooooooong overdue for some updates so let’s get to it. So far this year has been an adventure and I’m going to give you a whole rundown of events in the second half of this post but first I want to talk about some changes (good I think) I’m going to be making to be more transparent as well as help capture and accelerate momentum for ModVB and related projects in the near future. The project isn’t vaporware, I just suck at scaling (I apologize) and I’m pivoting to address that.

What’s going to happen moving forward

I’ve been giving tremendous thought to how I can better scale. This ModVB project is important to me and a (slowly) growing number of followers and it deserves more than it’s been getting. Also a few of community members, including past community contributors to the Roslyn project, have reached out offering to pitch in. Taking on more contributors has always been in the cards and we’re rounding the corner to the right time to do so. One consideration to doing so has always been my own bandwidth for working with contributors and testing and reviewing code coming into the project. I’m not going to loosen up on my quality bar for the project, but I am readying myself mentally and practically to function in the role of project manager, tester, and code-reviewer. I also respect that simply having more hands involved in the work will lead to more steady progress and build confidence for would-be adopters that this project is a safer bet.

Here are some more concrete steps that I’m working on:

(Finally) publishing most of compiler/language/runtime backlog

I’ve frequently mentioned the 11 (now 12) page outline of language changes I have for tracking planned and potential modifications to VB. I know I’ve been pretty secretive about this list so far mostly just to avoid feedback flooding but also showmanship and other reasons. Those days are fast coming to a close. And I’m not going to just publish the outline as is but a true to form Anthony-style “Wall of Text” laying out each feature, key design points, motivating scenarios, and in even my thoughts on what the “demo script” for the feature would look like. This will go waaay faster than my current approach of waiting to do everything myself and revealing things only with a < 5 minute YouTube video and will empower others to go after the low-hanging fruit in parallel with my own work. In fact, one VB community member already has implemented a feature that I’m very interested in pulling into ModVB.

I’m putting this post together to inspire both contributors and supporters with my vision for the future of the VB experience that I truly believe is exciting and achievable by the community, completely independently. This document will be the basis for both project planning and myriad strategic and design discussions I’m literally aching to have so … I’m really excited to finally be preparing it to share! I sincerely hope that finally seeing most of the big picture will put new winds in this project’s sails.

Setting up the repos for more contributors (than me)

  • Setting up repos and testing infrastructure for
    • Compiler/IDE (Roslyn)
    • Runtime
    • Compiler extensions (e.g. JSON literal/XAML literal bindings)
    • Project templates
    • More…
  • Setting up docs for contributor success
    • Licensing
    • Contribution guidelines
    • Compiler tutorials
    • Feature speclets and full specifications (over time)

Timeframe

The timeframe I’m targeting for most of this is early-mid summer, so, either late June or sometime in July and just in time for the 1 year anniversary of my announcing ModVB.

And the XAML literals + top-level code work is still coming. It’s not vaporware, it’s just approaching release asymptotically while I’ve been juggling other projects and life events. Which brings me to…

What’s happened so far

When I decided, a year ago, to take some time off to kick start this ModVB thing, I imagined (and planned, and saved for) 5 months or so. Basically a nice self-funded sabbatical to dedicate myself wholly to a creative endeavor that truly matters to me. I want to re-emphasize that number: 5 months. I figured that was enough to get through the first couple of waves and parachute into a new job somewhere. SPOILER ALERT: The mass layoffs, various hiring freezes, and general contraction of the tech sector starting right about when I was supposed to deploy this metaphorical parachute quickly showed me the error of my presumption. So, when I write to you a year from the start line you can understand that at least 7 of those months of unemployment were completely unplanned for. I’ve basically been operating in survival mode since last fall and the great news is that I survived. I survived with a lot of help, from family, friends, clients, and those of my readers who became Patreon supporters.

Highlights

I was (briefly) writing a VB.NET book for a popular publisher

That’s right! In the beginning of this year I was approached by a popular publisher about authoring a book on VB.NET, adding to my list of 3 or 4 projects I was juggling at the time. I was excited for the opportunity but ultimately after going through the initial steps of authoring, I realized that in just explaining the feature work in ModVB in the near future I was already committed to writing like a book’s worth of content and that it was insane to try to write TWO books at once while also juggling project work. Maybe next year.

I changed health insurers… and lost my entire healthcare team and all my appointments

I switched to a more reasonably priced health insurance plan which has been a major relief. However, because healthcare is never simple in this country, doing so moved my entire healthcare team “out of network” and basically meant restarting my entire healthcare regimen from scratch with a new primary care person, a new team, etc. I still haven’t really recovered from the switch though fortunately my medications have been maintained.

Stunlocked

Having said that, it turns out that managing a workload of software projects from multiple contracts, managing my (mental) health, and managing ModVB are all full-time jobs on their own. Being one person, I’ve naturally been sucking at all of them.

“Enough about your personal problems, is this XAML stuff ever going to happen?”

Yes. Here’s what goes into a normal feature for me.

  • Designing/implementing/testing the feature in the compiler
  • Designing/implementing/testing basic IDE functionality around the feature in editor
  • Designing a compelling motivating scenario to demonstrate the feature in under 5 minutes and implementing any demoware sample projects that go with it

This is normally like ~2ish months of work. But XAML literals have extra steps because they implement a base pattern in the compiler but to really exercise the feature you need a mature implementation of the pattern. By comparison, the design of LINQ is a very meaty set of features in the compiler but imagine if LINQ couldn’t be coherently released or demonstrated without first implementing several LINQ providers—not just LINQ to SQL or Entity Framework but two or three other implementations. This is where the tail end of development has just been dragging on. I can’t just say voila XAML is implemented in VB now but I need to release it with:

  • XAML bindings for .NET MAUI
  • XAML bindings for Avalonia

These are important because while they both use the same XAML syntax the way that XAML is translated is wildly different, which is good because it exercises the generality of the feature but I also have to prototype enough of XAML bindings for a hypothetical/prototypical VB Web library which doesn’t yet exist but which we’ll want to start building pretty much immediately after launch. And each of those bindings has to be tested as well. And consider that the feature has to both be tested in a unit test in the compiler which does not import any of these UI frameworks into the compiler test dependencies, i.e. I’ve had to mock compiler extensions representative of all the real scenarios that need to be supported in in the initial release but ALSO have separate automated test suites for each of the real extensions. Right now I’m trying to distill a kind of “VB XAML standard” test profile that can be reused across multiple implementations to hit a set of key scenarios (e.g. object graph deserialization, data binding, styles, templating). It might seem like I’m boiling the ocean and maybe I am but what I don’t want to do is ship an incomplete pattern which then requires me to ship new revisions of the ModVB compiler every week or two as the implementations of compiler extensions uncover and fix more bugs. Now there’s a dependency hell where you need to upgrade the extension libraries AND the compiler constantly to get anything to work. I would like the compiler pattern to be mostly stable and the extensions themselves to do more of the churning.

All of that would be a lot if I weren’t, as always, struggling with mental health issues, and if I weren’t juggling multiple projects with multiple clients that I’m trying to meet obligations with. To put it into perspective, out of the entire backlog this is both the most important and the single largest feature I ever plan to implement in ModVB. That’s why my original (flawed) plan was to do this kind of work during a period where I could focus almost entirely on ModVB. But it’s OK, because soon talented folks are going to be able to help me scale better.

Summary/TL;DR

Juggling lots of smaller contract gigs and personal issues has really slowed progress but there’s more bandwidth on the horizon. Regardless, it’s always been the expectation that ModVB would welcome some additional contributors and that time is fast approaching this summer; preparations are being made, most importantly of which is publishing a wall of text detailing the full backlog for the language so that both contributors and supporters can gauge their interest to chip in. XAML work still incoming, gated by a high quality bar.

XAML Literals Design Updates and Implications

Wave 2 is progressing steadily though I think it’s approaching an inflection point. January was very challenging balancing out working on XAML literals alongside my contract work, but I still feel like I’m converging on a release. I want to share more insights into what’s going on behind the scenes, so I wrote this post. It’s really a chance for me to just nerd out about design so if it doesn’t excite you that’s ok. I find it cool because I’m a language design nerd. To appreciate some of the recent design changes and their implications you need some background on compilation that I’m going to try to sum up. A few years back I wrote a primer on Roslyn and how compilers work. Folks have been asking me to pick that series back up (and I know I’ll need to to involve more folks in mod development) so think of the first part of this post as a draft of doing so.

A quick review of compilation

At a high level, compiling a VB.NET program is broken into 4 phases:

  • Syntax—what you type
  • Semantics—what it means
  • Lowering—reducing high-level VB constructs like loops and Await into low-level CLR constructs like Goto (it’s almost all gotos at the CLR level)
  • Emit—writing the actual bytes that the CLR will load and run to an EXE or DLL file

The scope and separation between the first two are what’s relevant for this post.

Syntactic analysis means taking a sequence of characters literally one character at a time and assigning those characters to structure based on the set of rules that most people intuitively think of as “the language”. Syntactical analysis itself happens in two “phases”. I put that in quotes because in a textbook college-level computer science class you might think of them as “phases” that happen one after the other but technically in Roslyn they happen at the same time and are very interconnected. Those phases are scanning (also called lexing, or lexical analysis, or tokenizing) and parsing.

Anyway, if you’re reading this blog you’re almost certainly a VB.NET developer so I can just describe these concepts in terms of VB without straining analogies to natural languages like English.

Scanning

Given the string of characters: Await A.B(C) ‘ Comment, scanning chops the string up into a sequence of 7 tokens:

  • Await
  • A
  • .
  • B
  • (
  • C
  • )

Scanning of course uses whitespace to separate tokens but whitespace (in VB) isn’t itself a token (other than newlines). Comments are, like whitespace, not tokens, they’re just something the scanner knows to skip over.

Parsing

The parser takes that flat sequence of 7 tokens and builds a tree structure

  • ExpressionStatement
    • AwaitExpression
      • AwaitKeyword: Await
      • InvocationExpression
        • MemberAccessExpression
          • Identifier: `A`
          • Dot: `.`
          • Identifer: `B`
        • ArgumentList
          • OpenParenthesis: `(`
          • Argument
            • Identifier: `C`
          • CloseParenthesis: `)`

So, in this example the scanner decides that A, B, and C are identifiers (as opposed to keywords, punctuation, operators, or literals) but not what they refer to. The parser is what uses those tokens to build structure of expressions and statements (and declarations): C is an argument to an invocation, B is (presumably) a named member of A, etc. But again, parsing doesn’t decide what any of those identifiers refer to. Maybe none of those identifiers refer to anything and the program is an error. Maybe A is a namespace and B is a function in a module in that namespace. Maybe A is a local variable, B is a property that returns a collection, and C is being passed to the default property of that collection. The parser doesn’t determine that, that’s what Semantic Analysis does.

Interdependence

Again, in an academic context lexical analysis and parsing are very neatly separated but in practice in VB.NET the parser directs the scanner with context that only it knows about. For example, the parser knows whether it’s parsing inside of an Async method and that’s what determines whether Await is a keyword or an identifier (and thus whether Await (x) is a parenthesized expression being awaited or something called Await being invoked with argument x).

This familiarity between the parser and the scanner are what make it almost trivial to embed languages like XML, JSON, and the String.Format syntax for interpolated strings into VB; the parser is what knows whether it’s “in” an XML literal, JSON literal, or interpolated string and thus instructs the scanner to tokenize characters appropriate to that context.

Token values

Part of scanning also involves assigning what in Roslyn we call the value of a token. So identifier A, escaped identifier [A], suffixed identifier A% all have the value “A”—that’s the name that identifier declares or refers to. Decimal integer literal 10, hexadecimal literal &H0A, and binary literal &B1010 all have the value of integer 10. The value is part of the token, is what’s used by the compiler during semantic analysis, and is separate from text of a token. Another more relevant example is the string literal ”5’8””” which contains an escaped quotation mark, the value of that string is 5’8” what would be written to the console if you printed that string token. Likewise the value for XML text tokens in an XML literal replace XML character and entity references like &amp;, &#38;, &#x26; with the character &.

Immutability

Now, the last thing to keep in mind about the phases of compilation is that the output of parsing is an immutable syntax tree—a hierarchical data structure that cannot change once created and is fed into semantic analysis (along with project references) to determine the meaning of a program. There’s limited examples of parsing re-interpreting a scanned token in the moment (e.g. around contextual keywords) but that’s still in the midst of parsing. Once parsed there’s no going back during semantic analysis to decide that the developer typed something else.

OK. I think that’s enough of a review for the rest of this post.

A XAML Literal is a semantic distinction, not a syntactic one

With that understanding, note that whether or not an XML literal in VB represents a XAML literal is a semantic distinction. XAML is XML and syntactically (to the parser) a XAML literal will look like any other XML literal. It’s during semantic analysis that the XML namespaces in scope (which, remember, may be imported at the file or project level) are examined and checked against the available runtime extensions to determine what that XML translates to. Whether that’s an XElement instance, a System.Windows.Controls.StackPanel instance, or a instance of a class from an entirely different library is all determined semantically.

Markup extensions

This is all mostly good and well until you come to implement markup extensions because markup extensions have their own syntax.

In my original prototypes, I didn’t implement any special handling of markup extensions in the compiler. There were just hooks for extensions to process attribute values before assigning final values to members and those extensions could inspect XAML values for things they knew about like StartsWith(”{Binding “) and do whatever they wanted without the compiler having to know anything about it.

There are a several problems with this approach.

Markup extensions can be nested

OK, so one problem that comes up is that markup extensions aren’t just a pattern of simple values, they can be nested:

Text=”{Binding Name, Source={StaticResource Company}}”

But this isn’t the end of the world. Yes, it means the any implemented extension has to be able to recursively parse and process all of the supported markup extensions recursively. Annoying, but doable.

Markup extensions can be user-defined

Supporting all of the familiar markup extensions like Binding, StaticResource, DynamicResource, etc, would be very tedious on its own. But then you have to consider that XAML allows anyone to define their own markup extensions. And it’s not unreasonable to expect that if a user has defined such an extension that they’d be able to use them in a XAML literal just as they would in WPF, Silverlight, or WinUI.

So, ok, the extensions can try to parse and resolve custom binding extensions. Since these extensions will likely be mapped to arbitrary XML/XAML namespaces rather than in the default namespace it does mean the compiler must have a way to supply the XAML namespace mappings in scope for the extensions to do this but for other scenarios (type converters) the compiler would have to do that anyway so it’s not the end of the world. It does mean a lot of parsing, converting, and reflection at runtime which is a perf hit but again, it’s doable.

Users should expect a rich IDE experience typing markup extensions

If you were typing a XAML literal what kind of editor support would you expect? Completion on XAML namespace prefixes? Markup extension type names? Maybe data-binding property names? What about colorization within the attribute?

Technically the Visual Studio and Roslyn APIs do supply enough hooks that someone could, with a lot of work, implement these things over an opaque string (e.g. RegEx syntax colorization). But…

Markup extensions can have errors

Since all of this handling of markup extensions is deferred until runtime through extension helpers, when something goes wrong the way it would naturally be surfaced to a developer is at runtime—which stinks. What if you mistype a markup extension name—wouldn’t you want a compile-time squiggle for that? Of course you would! And then what about syntax errors—if you fail to close a markup extension you’d expect to hear about that early too.

But that’s where things get tricky! Because as I mentioned a while back, a XAML literal is a semantic distinction. In other words, …

<TextBox Text=”{Binding Name” />

…is an error if and only if that XML element expression is also a XAML literal. It would be too grievous a breaking change if otherwise valid XML literals all of a sudden started reporting syntax or semantic errors based on the fact that they would be erroneous XAML.

At first glance this made me think that compile-time error reporting should be left to an analyzer rather than the compiler because the compiler has a higher bar for correctness than an analyzer, which can only report warnings which can be ignored. Now you have a situation where the XAML provider extension is reparsing attributes values at runtime, and the IDE and analyzers are each reparsing attribute values as you type which really erodes the promise of Roslyn. So, eventually I decided that I wanted to do some analysis of markup extensions in the compiler.

Now that the compiler is processing markup extensions, it has to be decided how this information is represented. Naively, you’d add to the Roslyn Syntax API several nodes modeling markup extension syntax like you would when adding any new syntax to the language. But once again, remember that we only know that an XML literal represents a XAML expression at binding time—during semantic analysis. It would be … problematic for the syntax API to report that something “looks like” a special kind of XAML attribute. But, maybe not too crazy. By analogy, the compiler already has the concept of “structured trivia”. This technique allows the compiler to attach structure to a comment in a way that sits beneath its normal representation and this is what is used for XML documentation comments—interested parties can inquire on whether a comment has structure and other parties can treat it as just a comment. Maybe I could do something like that with the XML text tokens of an XML literal but I’m extremely hesitant as while structured trivia (comments) are a concept in the Syntax API, there’s no such analog for a potentially structured token.

Circling back to the question of how errors are represented, …

<TextBox Text=”{Binding Name” />

In our example we only want to report a syntax error here for a XAML literal, not a regular XML literal. But as I mentioned, once created a syntax tree is immutable. We can’t decide later that it has syntax errors or that it does not have syntax errors. So this is the cleverness that I propose:

We parse the markup extension but throw away any errors we find. If there were no errors we produce a valid WellformedMarkupExtensionSyntax node and if there are any errors we produce a valid MalformedMarkupExtensionSyntax node. In either event we don’t report ANY errors during parsing. When it comes time to bind a MarkupExtensionSyntax, if it’s a WellformedMarkupExtensionSyntax AND contained in a semantic XAML literal we bind it using XAML rules. If it’s a MalformedMarkupExtensionSyntax (inside a XAML literal) we reparse the entire expression but reporting this time reporting “syntax” errors alongside semantic errors. If it’s not a XAML literal we ignore the structure and just use XML value of the text in a regular XML literal.

Voila! Now we can have our cake and eat it too. We can report “syntax” errors for markup extensions but only inside XAML literals, and we can provide rich structure to the IDE and analyzers that they don’t have to recreate for themselves, and XAML provider extensions don’t have to do almost anything because the compiler is handling the semantic analysis of the markup extension (which is a XAML concept by the way, so it’s entirely reasonable that the compiler would do this). This is great because the actual rules for parsing and binding “text” inside a markup extension are arduous enough that to force anyone else to re-implement them would simply be mean spirited.

Spoiler: This isn’t what I ended up doing.

Markup extensions II

As I indicated in my last post, this whole thing has been part of the process of embracing the scope of the core feature of “Wave 2” as a true to name XAML literal, not just XAML-like enhancements to XML literals. This shift in mindset both frees me and binds me to lean on the formal XAML specification. For the most part this has been liberating and the XAML spec is actually a bit of a fun read. It describes a type system for XAML without actually depending on that type system being implemented by the CLR or any specific XAML vocabulary such as WPF, with notions such as “assignable to the x:MarkupExtension XamlType” rather than the CLR notion of inheritance from the WPF MarkupExtension class. It describes relationships between “types” and “members” and lookup without depending on the concept of a CLR property or classes or even static typing per se. If this doesn’t sound like a fun read to you, that’s normal. I am not. Not anymore.

So, there I am reading the XAML specification and for the most part everything is making sense and on page 72, in Section 8.6.3 “Member Node Creation from an XML:attribute”, I come across this one line:

Let attributeText be the string xmlAttribute[normalized value].

…and my head explodes.

The normalized value. Not the text as written. That’s when it clicks that the processing of attributes isn’t an alternate interpretation of XML but a subsequent step. In other words, in XAML (and you can verify this in the Visual Studio XAML editor right now), all of these (and more) are equivalent and perfectly valid markup extension usages:

  • {Binding Name}
  • &#x7B;&#x42;&#x69;&#x6E;&#x64;&#x69;&#x6E;&#x67;&#x20;&#x4E;&#x61;&#x6D;&#x65;&#x7D
  • &#x7B;Binding Name&#x7D;

So it’s not enough to simply look at the text as we’re scanning an XML text token and check for a { character, one has to first tokenize the entire attribute value as XML—then retokenize the normalized value (the value after all XML character references and entities have been resolved). And there’s a further layer to this because markup extension syntax itself has an escape syntax where the value of {}{Binding} (or e.g. {}{Binding}) is not a markup extension but whatever follows the {} (however they’re encoded). And I hear you right now thinking “Who cares, though? You’re never going to run into this in the wild” and you’re right but we’re doing this The Right Way™️ so… in for a penny…

While at first this realization may seem like a complication, it actually simplified the design in that it left me firmly resolved that all computation of XAML “values” should happen entirely in semantic analysis. That is to say that once again parsing will be completely pure of any involvement of the rules of XAML or markup extensions or markup extension escapes and will purely handle the interpretation of text as valid XML. This is great for all existing code that handles XML literals as no special handling would be needed to undo pre-emptive XAMLfication of any speculative WellformedMarkupExtensionSyntax, MalformedMarkupExtensionSyntax, EscapedXamlAttributeTextSyntax, or anything else like that.

Designing an API for efficient tooling

As I mentioned above, the XAML specification provides a whole type system describing XamlType and XamlMember instances so it’s very natural to represent these concepts as first class in the compiler internals and to expose them through an API so that the IDE and analyzers can ask questions like “What type does this element construct?” and “What member does this attribute initialize?”; reasonable questions for providing a rich editing experience with completion and quick info, etc.

However, the Roslyn APIs are designed such that semantic questions are normally asked in relation to syntax nodes that very efficiently convey which span of text is being asked about and with XAML literal translation happening entirely in semantic analysis there is now a need for some way to ask these questions about offsets in an attribute’s text (and that’s a simplification). Long story short, I’m creating a pseudo-syntax API that captures the markup extension syntax so that tools can still be written efficiently. It’s very consistent with the true syntax API as I feel that’s ideal for API consumers. Having said that, I do think that XAML code editing tooling will have to wait for a future minor ModVB release because I need to get this feature out into your hands for testing/feedback ASAP. Maybe I’ll enlist help in building the code editing tooling from a community member.

Separations from and Interactions with top-level code

A few more design points…

InitializeComponent

In my earlier prototypes I planned to have the binding of top-level expressions special-case XML literals to support the common case of a XAML window or root object being initialized by a single literal. However, with this revision I’ve decided to move this behavior from being a special case of top-level code to being inherent behavior of a XAML literal expression statement inside a Sub (a method which does not return a value) whose name matches some configurable identifier such as InitializeComponent. This means that whether as a top-level expression statement, an expression statement in a top-level method, or an expression statement in a normal method in a normal class XAML-based initialization will work as expected. Aside from being a little cleaner this design is critical to enable testing of the XAML initialization scenario in isolation without top-level code, which is not implemented yet.

Name and x:Name/Code-behind

Another use case which is essential to reproducing the experience we’re all familiar with from WPF is backing field generation and wire-up for named controls. As it stands, the design of XAML literals is that using the x:Name directive (defined by the XAML spec) or setting the equivalent member (also a specified concept) on a XAML object will lookup a field or property on the specified x:Class and initialize it with that object. It does not cause that field or property to be created.

Part of the reason for this is because of a separation of subphases within Semantic Analysis—XAML literals are just normal expressions and are bound in the context of a method. Method body binding happens after the phase of semantic analysis that processes declarations. Of course, it would place undue burden on users if they were required to defined all their backing fields for named controls manually so the tension right now is between having these declarations created as a part of building declarations from top-level code (in contrast to the previous heading regarding InitializeComponent methods), or generating the fields explicitly through a source generator. The generator approach is closer to how WPF works today but the problem is to correctly determine which fields are generated and even their types requires semantic information. For example, if I assign a name to a control inside of a ControlTemplate or DataTemplate, Style, ResourceDictionary, or other deferred content that name technically exists within a separate namescope (A XAML spec concept). Only named objects within the root namescope should have fields generated. Which classes create their own namescopes is semantic knowledge which incurs a heavier performance penalty when using a source generator because the compilation must first be built up to the point where that information is available in order for a generator to ask about it.

Of course, a source generator could be hard coded with this knowledge (unlike the compiler) on a per-XAML vocabulary basis. One could even squint if the generator created too many fields in this case. But the alternative is to allow the compiler itself to do just enough semantic analysis of XAML types while building the declaration tree to create these fields. Both solutions can be elegant if you throw enough code at them and I’m currently trying to resolve this tension, though it won’t hold up testing/stabilization of the XAML literal feature because of my original design point—the XAML literal need only find the fields and initialize them, it takes no dependency on how those fields or properties are defined.

Another scenario that benefits from this separation, sort of, is the use of a XAML literal as an expression within a method. Having the literal take no responsibility for generating any new symbols means that a) I don’t need to design what to generate in such a use case, and b) it’s straight forward to design backing such objects with local variables.

The long tail of XAML scenarios

I hinted at the special handling of deferred content such as ControlTemplate and DataTemplate instances in the previous heading. There’s more testing and implementation to be done there beyond just namescopes. For example, .NET MAUI handles these very differently from WPF or Avalonia UI so I have to implement and test both approaches.

Another trailing work item I need to implement is ambient properties, which are a WPF concept rather than a XAML concept but an essential one for styling. I’m tempted to defer supporting this until later but styling is a pretty central WPF use case. Either way I think this scenario defines the cut line for wrapping the initial implementation of XAML literals. There is bound to be a long tail of minute technology specific scenarios the community will find when trying to port examples of different technologies to use ModVB and I’ll simply have to address those bugs as they come up.

Once and only once I feel confident in the stability of this subset of XAML literal capabilities will I move on to the implementation of top-level code. I expect a lot more stability bugs to come out of that feature than XAML literals and I want to make sure I’m not fighting two fires at once by mixing the two. I do think my original design for top-level code is pretty elegant and still the correct way to go and should be fairly quick to implement so fingers crossed Wave 2 should release this month. I already know supplemental features and tooling will come in a future wave.

Wrap-up

Thanks for listening to a rambling nerd to the bitter end. My next post will be a short personal update talking about what it’s been like juggling 4 projects in the month of January and other stuff.

Regards,

-ADG

This blog post was brought to you in part with the support of my wonderful Patreon community (which you can join). Special thanks to my Lunch tier patrons and higher: Dr. Donna M., Alex T., Kirill O., Neal G., Govert D., Dr. Anne J., Taiwo A., Jonathon M., Peter B., Kevin R., and Paul C.

Checking in (Wave 2)

Season’s Greetings, All,

I hope this holiday season finds each of you and your families in good health and good spirits. I tried to get this update out before Christmas, but it was not to be. I have several announcements (all positive) to make across several topics both near and far. I know everyone’s wondering what’s the deal with “Wave 2” given that it was supposed to have a release or some demos out this month and I will get to that in this post, I promise, but first I want to get a few quick but important asides out of the way.

Patreon Launch Update

First, let me say a gigantic THANK YOU! to everyone who responded to my call to action regarding my new Patreon. Thanks to your support I’m already well on the way to my first fundraising goal! I’m very pleased with the initial uptake and will be working on getting those tier rewards out ASAP. Every drop of support means the world to me in both a very practical way and in a psychological way. It really keeps the fight in me to know other folks still give a darn.

Also, got some part-time freelance VB contract work, one project of which is thanks to a Patreon supporter answering my call for opportunities. Working on VB projects is always a dream come true so thanks to that special supporter for this opportunity and for the rest of my supporters, please keep ’em coming!

“Wave 6” Theme Update

I wouldn’t normally concern you all with an update on something so far out but I wanted to give my readers some assurances that I have been paying attention to their pain points and am making plans to help. Originally, I didn’t have a clear vision on any specific themes after “Wave 5” (which is themed after LINQ). However, based on observation, a theme for “Wave 6” has emerged around C# interop. More specifically, I’ve seen a number of VB users being blocked by new APIs both in the .NET 6/7 profile and in 3rd party libraries that use features such as the `unmanaged` constraint, stack-only “ref” structure types, and similar scenarios. Some cases where VB is being blocked, such as `required` members, are already being worked on by Microsoft and I sincerely thank them for it, but others they have announced no plans to address. I recognize the pain this causes VB users who are blocked from using these APIs and will be dedicating an entire milestone to designing and implementing some degree of support to unblock you, particularly with a focus on consuming these APIs and I wanted to share that intention with those impacted sooner rather than later. The wave plan is pretty fluid after “Wave 3” balancing several factors including the need for earlier design feedback but another factor is urgency so I may pull this new “Wave 6” in earlier. Stay tuned.

“Wave 2” Scope Update

“Wave 2” is all about unblocking new platforms through the repurposing of VB XML Literals syntax to enable declarative UI development on the web and cross-platform mobile development. This has always been the plan. What’s changed is that for a while now (years, in fact), I have been very cautious in describing the centerpiece feature of this wave as “Enhanced XML Literals”, “Target-Typed XML Literals”, “Pattern-based XML Literals”, or “a XAML-like syntax”, very intentionally avoiding the term “XAML Literals” because that term makes certain promises that I wasn’t willing to casually commit to. My intention was always to target a strategic subset of XAML syntax that would coincidentally produce the same result in common scenarios as what is used in Xamarin.Forms, but wouldn’t be bound to the full breadth of scenarios of a technology like WPF or the formal XAML specification. And for what I prototyped back in 2019 that was good enough.

What’s changed is the scope of technologies being targeted. WPF (and UWP) already support XAML design/code generation for VB.NET projects. The missing stories were ASP.NET Core for Web and Xamarin for Mobile. Xamarin.Forms (and by extension .NET MAUI) was designed to mimic the idioms of WPF but is implemented very differently in how those declarative idioms are translated to code. In places where WPF (and Silverlight) re-invented certain programming constructs in a distinct way from the .NET languages, Xamarin often embraces the “C#”-native way of implementing those idioms. For example, for control- and data-templating, WPF uses an arcane infrastructure behind the scenes using factories to instantiate controls, repeatedly in the case of `ItemsControls`, at runtime (deferred) given a declarative template written in XML. Xamarin.Forms on the other hand embraces lambda expressions. A lambda expression is the idiomatic way in C# (or VB or F#) for passing instructions to an object for how to construct something in a differed way that can then be called into over and over again. If you look at the constructor for the Xamarin.Forms DataTemplate class, it takes a Func(Of Object). WPF by contrast is not nearly so straightforward to use in an imperative code-only way.

Because I only needed to go after Xamarin.Forms, my design for the factory patterns an XML Literal would translate into were pretty narrowly targeted at what would unblock declarative Xamarin.Forms capabilities in VB with only loose consideration of what kinds of patterns would be necessary to target non-trivial WPF expressions.

With all that said, the scope of technologies I’m now pursuing has changed. I’m now targeting:

To be clear, though this might smell like “scope creep” as a language designer this is a really good thing! Designing a feature around a broader set of scenarios generally yields better, more generalized features with greater applicability, which is generally preferable to the accumulation of one-offs.

All of these potential targets are being actively and holistically considered in the design and implementation of this feature. Avalonia UI is the critical addition to this list because unlike Xamarin.Forms it embraces a very WPF-like translation of declarative XAML into .NET objects. Additionally, in designing the patterns for translation I am acutely aware that 1) someone will have to write the “builder” libraries that enable support of a particular technology and their life shouldn’t be overly tedious (e.g. they shouldn’t need to explicitly code support for every dependency property or conversion in an entire framework), and 2) VB users will want to consume 3rd party components written targeting these technologies that have no knowledge of this feature, or themselves may implement controls or user controls targeting those technologies and that I do not want to place a burden on those VB users to mediate integration of those components with the feature.

All of that has led me to the conclusion that I must embrace fully* the XAML-osity of the feature to provide the best experience for VB users. I’m effectively designing and implementing a XAML “meta-processor” inside the VB compiler—a feature that allows one to efficiently describe and implement multiple technology-specific XAML processors in a prescribed way. This design shift raises the bar significantly for the feature but it’s also very liberating in that I don’t need to employ as many workarounds to support certain scenarios in indirect ways. For example, markup extensions (e.g. {Binding}) until now were a concept I wanted to be completely opaque to the compiler with it passing enough information to the builder library that it could then parse and interpret those markup expressions at runtime. Or certain elements of the XAML language like x:Name, x:Type, and x:TypeArguments that are defined by XAML which before were meaningless to the compiler itself and very complicated to reproduce at runtime. Those are restrictions I placed on myself, but now these are things described by the formal XAML specification that the compiler can fairly know about.

I know some folks might find such deep integration of yet another language into the VB language/compiler distasteful and it’s absolutely not a coupling I take on lightly but over the last 14 years, time and time again new technologies have taking bets on the XAML language and I believe it is now stable enough, mature enough, and enduring enough to take a bet on too, not unlike the bet VB (and C#) both took on the SQL-like syntax of LINQ.

That said, even though I’ve fully* embraced “XAML Literals” as the north star of the feature I still put an asterisk on that word “fully” because I want to be clear that I’m still very consciously constraining myself on the minimal integration of concepts absolutely mandated by the XAML spec (with which I’m now becoming intimately familiar) and required to support these targeted platforms. I fully believe that “limitations foster creativity”. Accordingly:

Examples of things I’m ok with the language having some awareness of now:

  • The basic conventions of XAML with regard to object creation and member setting (including “attached” properties and events).
  • Things in the x: (xaml) namespace.
  • That an attribute value represents a markup extension.

Examples of things I’m NOT ok with the language having any awareness of:

  • Any particular technology or assembly names.
  • Any particular technology namespaces.
  • Any particular type specific to any particular technology (e.g. FrameworkElement, MarkupExtension, ContentPropertyAttribute, DependencyObject/DepedencyProperty).
  • Type converters

I’m holding the line on the latter list, which means building the necessary layers of indirection for those scenarios. Every concept the language understands must be very carefully considered, even with this bold new shift. For example, I don’t want VB to “know” about type converters. It’s OK for a builder library to know about them but not for the language to know about them. They come with a lot of design baggage and raise questions I really don’t want to answer right now such as “Why do XAML literals understand and use type converters, but the basic casting syntax does not?” and that would be a fair question if the language or compiler did actually know about them. But as I am implementing them, these dependencies are properly hidden behind the abstraction of a XAML conversion without a direct requirement on how such a conversion is implemented. Further, such knowledge isn’t even hinted at by the XAML specificationthe term type converter does not appear in the spec even once—this is an implementation detail coincidentally associated with WPF but not strictly mandated by the XAML language per se.

Oh, and FYI, here’s a picture of an Avalonia UI app running on Android built using VB!

So, yeah, in summary, they’re called XAML Literals now. I gave up (for the better).

XAML Literal Entanglements with “Wave 7”

OK, technically this isn’t an update because I never told anyone this to begin with but it’s impacting the schedule so I’m sharing to be a little more transparent. There’s a critical piece of the pattern of XAML Literals that was designed in a certain way that’s a little… special. But the plan was to have this specialness be a subset of some specialness that I’d expand on more in “Wave 3” with an otherwise unrelated feature. This matters cause if you do a thing once in a language it’s a one-off, but if you do it two or three times it’s a design pattern and that’s legit so that plan was to do both things the same way and introduce a new pattern.

The problem is that the feature in “Wave 3” also needed to be reconciled design-wise with the existence of source generators—a feature of both the VB (and C#) compilers that Microsoft introduced “recently”. UI frameworks are squarely in the set of core scenarios for source generation and the democratization of source generation should make any language designer consider very careful what goes in the language and what goes in a generator. Until recently I’d written a “TODO” to reconcile new features with source generation with a note that some of the stuff in waves “2” and “3” could change drastically or disappear once I took a serious look at source generation “in the future”. The idea of presenting to you all a design in “Wave 3” which could be obviated entirely by source generators just kept nagging at me so this month I took the time to seriously think through how the mysterious feature in “Wave 3” and source generation would co-exist in one sensible experience and the short of it is that the entire design of the feature in “Wave 3” was obliterated. The design made a lot of sense before source generators were shipped (I’d started designing it at a time when source generators seemed very complicated and unlikely to release) and with them having released, putting some serious design thought into it caused me to throw away … everything other than the end-developer experience I’m holding to, though the implementation will be completely different.

This design sidebar crystalized to me that the theme behind “Wave 7” would be exploring the impact of source generation on language design further. The feature tentatively in “Wave 3” might just move back to “Wave 7”. “Wave 7” might move up. And the piece of design shared between XAML Literals and the feature in “Wave 3” either becomes a strange one-off in the language or needs to be changed. So, I’m changing it to align with the design pattern I now anticipate using elsewhere in the language. It’s coming along well. It simplifies the design, actually. It’s not nearly as “clever” as it was before which is a surprisingly hard to let go of for some reason, but ultimately better for everyone else but my ego. As part of doing things The Right Way™ for ModVB, I strive to never ship an implementation which I can’t say in good faith is expected to be a durable design or implementation. Prototype code is meant to be throw-away, what ships in ModVB is not.

Conclusion

“Wave 2” did not and will not release in December 2022 as planned due to valuable design changes, namely: targeting Avalonia UI to enable cross-platform UI development in VB for Windows, MacOS, Linux, Android, iOS, and the web; embracing the XAML specification as a first-class goal; and the integration of forward-looking design intentions from “Wave 3” and what is now coalescing as “Wave 7”. Implementation is otherwise going well and the deliverables for “Wave 2” are now expected in January 2023. I promise that I believe these changes were the right and necessary decisions to make given the information at hand and look forward to showcasing the fruits of all this labor early in 2023.

I thank everyone for their continue patience and support and wish everyone continued Happy Holidays and a very Happy New Year!

Regards,

-ADG

Path to Sustainability

In my last post, I discussed the high-level “what” of my current endeavor—to fully actualize the design potential of the VB.NET experience. This post is a little more practical about one necessary aspect of how we get there and sustain everything that’s built along the way.

To be blunt, this is a post about (crowd)funding and there is a Patreon call-to-action at the end but ultimately, I want to create a more robust solution that is community-centric rather than company-centric or me-centric and that’s what I’m going to talk about first. I’m going to lay out my long-term vision for how the VB.NET community can collectively sustain itself in an independent way and then in the second part I’m going to talk in a very pragmatic way about how I can maintain my own level of contribution through the short- (read: starting today) to medium-term. Both the long- and short-term views, I believe, are critical to ultimate success so I encourage you to read through both and understand if, what, when, and how you see yourself contributing.

Long-Term Vision

I remember a few years back catching a Tweet from Xamarin founder and then Microsoft Distinguished Engineer Miguel de Icaza advocating for the unionization of tech workers. I had been questioning (in my own head) for a while the assumption that as software developers we already have it “so good” that we don’t need unions and that beyond wages and benefits packages there’s not really value to be added for the tech worker. I don’t believe this assumption and I was so glad someone else in the industry and someone so successful therein had said it out loud.

Continuing this as a thought exercise I considered that employees aren’t the only other participants in the business of technology who could benefit from the ability to collectively organize, bargain, and act. The customers are often also at a huge disadvantage in the relationship with big tech. If you build on any technology produced and controlled by a singular corporation, you’re at the mercy of that corporation and whatever its transient agendas are. And if one day that agenda requires or benefits from, bluntly, abandoning that technology, as an individual developer or even a small-medium sized company with a large investment in software built on that technology your options are often few. This power imbalance becomes more drastic when doing business with multi-trillion dollar massively diversified companies like the ones that lead the tech industry in the US and abroad.

My thinking in the moment was, wouldn’t it be an interesting dynamic if, for example, a large portion of Microsoft Azure subscribers could “go on strike” to prevent the sudden and rather graceless abandonment of, e.g., Silverlight, etc.

That idea is ridiculous. Or at least, improbable, I think, but the point of the thought exercise remains that a more equitable structure between the producers and consumers of technologies would benefit consumers. And philosophically, this isn’t about VB.NET alone—we all could rattle off a long list of successful productive technologies that have gone out of favor with their original creators. If the business is looking out for the shareholders, and a labor union is looking out for the employees, who’s looking out for the customer?

Fast forward some thinking and research, and you come to the idea of a consumer cooperative. An organization collectively owned and managed by its members—the consumers—which optimizes not for profit but for consumer value. I would like to see a collective, technology adopter-owned model gain traction in the industry. The membership would be made up of individual developers and companies that adopt one or more related technologies. Some members would contribute to the development of those technologies and the membership would democratically make decisions on how to manage the lifecycle of those technologies, presumably without disproportionate regard to fads or any one member.

And again, I’m not saying this just as a VB.NET enthusiast but honestly as someone who’s followed the full Microsoft technology stack for like 20 years now. I’m just fatigued with the cycle. I’m tired of feeling anxiety about Thing K because other Thing M came out. I’m not saying new technologies shouldn’t come out, or that as consumers we should never migrate to a new platform or library but that I personally want that process to be entirely driven by a community of pragmatic individual choices about the real benefits and tradeoffs of new technologies over established ones rather than any sense of fear about the motivations or limitations or ambitions or caprice of any single company or the coupling of that company’s business interests in one space to another.

Using Silverlight again as an example, I am more than happy to adopt modern web-assembly based technology to provide rich desktop-like experiences on the web over Silverlight, in my own time and for my own reasons. I just don’t want Apple announcing that iPhone won’t support Flash and how that relates to Microsoft’s phone business (or any phone business) to be one of those reasons. (Shout out to OpenSilver for their modern, plugin-free, open source re-implementation of Silverlight!) I don’t want to go without a reasonable incremental update to the technology I’m using to not happen because a newer technology exists and “they” are pushing folks to use it instead so what I’m using today is frozen to provide some perverse incentive to migrate. I don’t want fear and frustration to be my forcing functions. I want an inversion of control.

Now, I’m not saying Microsoft or any other company should be forced to invest heavily in any particular technology indefinitely or even one more day than is in their business interest. What I don’t want is the outlook and lifecycle of, as much as practical, my tools to be dangerously and at times violently coupled to those interests. It’s just stress I don’t need for the next 30 years of my career. I want a setup where they don’t have to and the world can keep spinning productively.

So, with all that said, I believe that what the VB.NET community needs is an independent, community-owned (adopter-owned) non-profit organization unequivocally devoted to advancing the VB.NET experience and to growing and supporting the ecosystem of related tools and technologies to maximize community-value.

In this last year many VB enthusiasts have come to me privately and said, “if only we could pool our resources … all the developers and companies with important codebases in VB and fund more work on VB…”. Well, we can do exactly that, not unlike the Python community with their foundation, and the F# community, and the Haskell community, and the Rust community, and so many others. There are going to be tools and websites and samples and templates and extensions and packages and videos and more and ultimately, I would like the assets I build around ModVB or anyone else builds to further the VB.NET experience to be owned and maintained by such a VB.NET “foundation”. But that’s not an overnight process and I’m only very early in researching the practicalities of such an organization and how it would relate to Microsoft and the .NET Foundation and regardless building the membership required to fund VB.NET’s bright future will take considerable time. Which brings me to…

Short-Term Realities

Okay, so that’s the long term and the broad view for the community. How do I personally keep a roof over my head and food on my table while building up all these goodies for myself and fellow VB enthusiasts? Several potential approaches present themselves:

  • Get another part-time or full-time job, or recurring “gig”
  • Independent consulting/contracting
  • Support contracts
  • Crowdfunding

Until this summer I worked a full-time consulting job in Chicago and while lucrative it left virtually no time or energy (mental and otherwise) to pursue the various designs I had in mind for ModVB. In fact, having worked completely full-time for two straight months on the JSON literals and pattern matching features in Wave 1 of ModVB, I cannot imagine in hindsight what I was thinking before. The idea that I could put in that kind of effort on the side or on the weekends is laughable and to achieve just what I achieved in Wave 1 could easily have taken me a year putting in a few hours on Saturdays and that’s assuming my every consulting project was zero-stress, on time, on budget, just one project at a time, going smoothly every week with late nights or weekends. In my experience this just isn’t realistic.

As for part-time work… it’s a strange thing in our industry that you can make a very good living working 50+ hours a week but if you just want to work 20-25 hours it doesn’t just cut your earning potential in half. Time is the most valuable resource we have in life and it’s very difficult to negotiate for more of it while also not struggling and that’s been the challenge for me with part-time work. This paradox is exacerbated with the “gig economy”. I’ve tried to work Uber part-time to allow me the freedom to pursue my vision but the earnings compared to working in my field just aren’t there. Why stress out 30+ hours a week to barely make ends meet when I could stress out full-time and earn many multiples more.

So that’s the options that offer security. That leaves me with a blended strategy of the other three. Independent contracting gives me more flexibility on time insofar as I can decide after a project when to take on more work (if available). I’m currently looking at some VB freelancing opportunities and welcome any offers or connections from any of my readers or supporters for more.

Support contracts are the way I’ve heard of other freeware projects getting funded, like various Linux distributions. I make ModVB great and then companies that want to use ModVB can pay me a support contract to help them use it or prioritize their bugs or train them. While I’m committed to producing a quality product and fixing bugs regardless it’s certainly possible to prioritize some things to contract supporters. I don’t have much experience with this kind of dual-licensing strategy but I do believe you need a critical mass of functionality before you can offer support contracts and ModVB isn’t there yet. As the project matures this option will hopefully become more realistic.

Finally, I’m left with crowdfunding. Doing great work and hoping that the community of VB.NET enthusiasts and other interested parties will support me doing more of it. As scary as a leap of faith can be I feel this is the right time to take one. And so, today, I’m officially launching my very own Patreon. I don’t have much to offer by way of swag or private discord servers, but I am open to suggestions for more ways to connect with my supporters. I expect this will be a long journey. I’m not expecting everyone to dig deep in their wallets overnight but with each successive wave of features and tools I’m optimistic more and more of the community will come on board and lift me and the project. If you look at my page you’ll see I’m not trying to be greedy or make a fortune; it’s very pragmatically structured around my making ends meet to empower me to deliver the most value for the VB community. Every day I’m chasing alternate opportunities to earn money is a day I’m not spending advancing VB.NET to be all it can be, so, please check it out, support if you can, and share it with your colleagues and fellow community members. Thanks!

Next post

My next post will be an update on the state of Wave 2, which is currently targeted for one or more potentially incremental releases starting around the middle of this month (December). It should be fun times so look forward to it.

Regards,

-ADG

ModVB “Wave 1”—JSON Literals & Pattern Matching

Thanks everyone for your patience these past two months. I had to learn and do a lot to bring you this first release so I hope you find the wait was worth it. Today I’m launching an alpha of my mod, imaginatively titled “ModVB”. Installation instructions here for how you can safely install it to use JSON Literals and Pattern Matching right now! (Spoiler: You install a VSIX and reference a NuGet; that’s it)

This post is about the new features included in “Wave 1”. These are not prototypes like in my previous videos. Prototypes are quickly implemented to demonstrate the utility of one or few narrow scenarios taking whatever shortcuts possible to get to something that “works” (enough to solicit feedback) as quickly as possible. But the functionality in my mod is done “The Right Way”™ and should be extremely stable (crashes or freezes should be rare; please report immediately) and pretty close to what could be the final design. Read more about them below.

I’ve talked about JSON literals in previous posts, so I’ll start off with the newest feature on the block, pattern matching.

Pattern Matching

With respect to JSON, pattern matching is essentially the opposite of a JSON literal. Whereas a literal makes it easy to construct a JSON object in a way that preserves and reflects the structure of the JSON you’re trying to create, pattern matching allows you to deconstruct JSON in a way that likewise preserves and reflects the JSON’s structure.

Patterns

As of the time of this writing, here are the patterns that are supported by the Select Case ShapeOf syntax:

In all cases, null-ness may be “runtime extension defined” (see “Runtime Extensions & API Compatibility” below). This just means the compiler calls a helper method to determine whether something is true or not rather than purely CLR semantics. This is necessary because, for example, a JValue object is not null but represents the concept of null in the JSON space. Likewise, a JsonElement structure isn’t nullable but may still represent a null value.

The JSON Object Pattern

{
  "<name 1>": <pattern>,
  "<name 2>": <pattern>,
  ...
}

Matches a value if it: is not null, represents a JSON object (as defined by runtime extension), has the non-optional members named, and each member found matches its corresponding pattern, recursively.

This is evaluated top to bottom (depth first) so the entire first member and its pattern are evaluated before the second member is searched for.

This is not exhaustive; the JSON object may include more members than the ones listed and still match.

May be marked optional by placing a question mark ? after the closing brace. In which case, the pattern will also match a null value, i.e. the value is null OR matches the above criteria.

An individual name-value sub-pattern can also be marked optional:

Case {
  "firstName": required,
  "middleName"?: optional
}

In this example pattern requires that a firstName be included in the JSON payload but the middleName may not be present at all. This is subtly distinct from being present but null. Some JSON producers may not include null values by default. This syntax provides an easy way of consuming JSON from these producers in a way that treats an omitted value the same as one that is explicitly null.

The Empty JSON Object Pattern

{}

Matches a value if it: is not null, represents a JSON object, and that object has no members.

Unlike the non-empty JSON Object Pattern, this pattern is exhaustive.

May be marked optional by placing a question mark ? after the closing brace. In which case, the pattern will match a value that is null or is an empty JSON object.

The JSON Array Pattern

[
  <pattern>,
  <pattern>,
  ...  
]

Matches a value if it: is not null, represents a JSON array (as defined by runtime extension), that array has at least as many elements as listed, and each of the first n elements matches its corresponding pattern, recursively.

This is evaluated top to bottom (depth first) so the entire first element and its pattern are evaluated before the second element is searched for.

This is not exhaustive; the JSON array may include more elements than the ones listed and still match.

May be marked optional by placing a question mark ? after the closing bracket. In which case, the pattern will also match a null value, i.e. the value is null OR matches the above criteria.

The Empty JSON Array Pattern

[]

Matches a value if it: is not null, represents a JSON array, and that array has no elements.

Unlike the non-empty JSON Array Pattern, this pattern is exhaustive.

May be marked optional by placing a question mark ? after the closing bracket. In which case, the pattern will match a value that is null or is an empty JSON array.

The Constant Value Pattern

<constant literal>

This can be any constant literal in VB (including Date and Decimal values and Nothing) or the JSON constants true, false, or null. These JSON constants behave exactly as VB True, False, and Nothing respectively.

Matches a value if it (for constants other than Nothing or null): is not null, is convertible to the type of the constant, and after conversion has a value equal to that of the constant.

If constant is the literal Nothing or null, will match if the value is null.

May be marked optional by placing a question mark ? after the literal. In which case, the pattern will match a value that is null or equal to the constant.

The Variable Declaration Pattern

<identifier> As <Type>

Captures a value in a variable. The As <Type> clause may be omitted.

Matches a value if it: is not null and is convertible to the specified type. If the type is omitted the pattern will match any non-null value and the type of the variable is inferred from the type of the value.

Convertibility considers reference conversions and intrinsic language conversions as well as runtime extension defined conversions. In the absence of one of these a conversion may also be attempted if the value represents a String and the specified type defines a TryParse method or the specified type is an Enum type, in which case the pattern will succeed if a call to TryParse succeeds.

May be marked optional by placing a question mark ? after the type. If the type is omitted, the question mark ? may be placed after the identifier. In either case the pattern will match all values null or otherwise.

A note on optional patterns

When a JSON object pattern, one of its name-value subpatterns, or a JSON array pattern is marked optional, any variable declaration patterns within are necessarily nullable; if the parent pattern matches a null value these variables will also be initialized to null.

More to come

More patterns will be added later, as well as an expression version of the ShapeOf operator.

Runtime Extensions & API Compatibility

Both pattern matching and JSON literals are pattern-based, in the same way as LINQ and the For Each loop; they aren’t tied to any particular library or set of types. Instead, the implementations of these features look for helper types in a special Extensions namespace, decorated with the correct attributes, and having methods with certain names and shapes. The indirectness of that sentence reflects the levels of indirection built into the compiler to achieve this flexibility. You don’t need to worry about any of that other than that you should reference particular NuGet packages from the ModVB NuGet feed when you want to use these features with the corresponding APIs.

If you want to use JSON literals and/or pattern matching with:

  • Newtonsoft.Json.Linq types (JObject/JArray/JContainer/JValue/JToken/JProperty)

Reference package: ModVB.Runtime.Extensions.NewtonsoftJson

  • System.Text.Json types (JsonElement/JsonProperty) or System.Text.Json.Nodes types (JsonObject/JsonArray/JsonValue/JsonNode)

Reference package: ModVB.Runtime.Extensions.SystemTextJson

More such libraries will be released for other JSON libraries in the coming weeks.

It’s worth noting that these are not mutually exclusive. You can reference as many extensions as are applicable to your project without conflict. Pattern matching only pays attention to the extensions required for the type being matched against and JSON literals are target-typed (more on that below).

JSON Literals

I talked about JSON literals in this blog post and this blog post. Please check them out. The scenarios still apply, and the behavior is still the same except the JSON literals in this mod are far beyond prototype quality. There are two new behaviors as well: flattening and target typing.

Flattening

The properties of objA are “flattened” into objB (objA isn’t changed, of course). If instead of adding another object directly you add e.g. an IEnumerable(Of JObject), each JObject is flattened into the result in sequence:

The query yields a bunch of objects, but those objects are all flattened into the one outer object. This is a convenience feature particularly in LINQ as it allows you to yield a list of objects as a collection of properties, without requiring either an explicit syntax for a name-value pair or instantiating JProperty explicitly.

Target-typing

JSON Object expressions and JSON Array expressions are target-typed. Meaning, if you for example, assign a JSON literal expression to a variable or pass one as an argument to a parameter of type JObject/JArray (or one of their base types), the expression will construct that type. You can also use an explicit cast to determine the type of the resultant object.

Default JSON Object and Array types

In contexts where no target-type is available such as when using local or method type inference, or contests where the expression is being converted to Object, the expression will be evaluated as if it has the default JSON Object or Array type.

This lets you have a project that references multiple JSON APIs but doesn’t require explicit casts everywhere if one API is preferred. So, if for example, you have a project that predominantly uses the Newtonsoft JSON.NET types but you need to interoperate with APIs that work with the new System.Text.Json.Nodes types you would probably set your default object and array types to JObject and JArray and use target-typing in those specific times when you need to construct a JsonObject or JsonArray.

The default JSON Object type can be configured by setting the DEFAULT_JSON_OBJECT_TYPE preprocessor constant in your project settings to the fully qualified name of the type you wish to use. Likewise to set the default JSON Array type set the DEFAULT_JSON_ARRAY_TYPE constant.

If you didn’t know that VB pre-processing constants can have actual values—they can. A fact that I’m going to exploit heavily in future releases to avoid more cumbersome changes to MSBuild or the project system. Besides, after decades of being in the language, it’s time that capability start pulling its weight!

A few more notes: If this setting isn’t in your project settings the compiler defaults to the JSON.NET types because 2.17B downloads can’t be wrong and System.Text.Json.Nodes only exists in .NET 6 and later. I am thinking about making this default change based on whether you’re targeting .NET 6 and up or .NET Framework though.

The default JSON Object and Array types do not impact pattern matching except when trying to use an JSON object or array pattern against a value of certain types like Object or interfaces. In that situation you’re assumed to be testing the reference against the default object or array type and so the pattern will fail if the value is an object-typed reference to a different JSON library’s object or array type. To solve this you have to cast the value to at least the root of the correct type hierarchy (e.g. JToken/JsonNode) to let the compiler pick up the correct extensions.

Why Not Just Use Serialization?

On a previous post one or more commenters asked (paraphrased), “Why bother with this when everybody uses serialization?”

Well, for starters everyone isn’t using serialization. Only people who are using serialization NOW are using serialization. There are developers who aren’t exploring the power of the cloud or NoSQL database techniques at all and maybe this way of doing things makes it more appealing or more productive for them to do so, myself included.

I love types, I do. But honestly I’m a little fatigued on superfluous data transfer types that only exist to be serialized/deserialized. I had a project recently that was inundated with these anemic domain “models” for the sake of “doing it right” even in situations where no class was really needed.

For example, we had this dashboard API, part of which returned data to be rendered in the browser by a client-side JavaScript charting library in a SPA. There was absolutely NO value in trying to recreate the API of the JavaScript library. Ultimately, to the chagrin of my peers, I just constructed a returned a bunch of JObjects and JArrays matching what the charting library expected rather that “pretending” to build some rich type safe object model that was just going to get stringified immediately.

I worked on the Roslyn APIs for over half a decade; I believe in the benefits of types. But I also spent a good chunk of my career chasing after the ideal of “persistence ignorance”. The idea that domain objects are pure and free floating abstractions that are the true model of all software and the fact that objects happen to get saved to databases, or happen to get saved to the file system, or happen to get sent back and forth over the network via services is just an implementation detail. And in plenty of cases that’s absolutely true.

But there are also cases where the goal is to talk to a REST service you don’t own, or to manipulate some JSON in a database. If the currency of truth in an operation is JSON, it’s not the implementation detail, your POCOs are. The JSON is the whole point. Serialization classes are just something I would have done to make the SPA’s JavaScript charting library fit into a .NET world; I didn’t own that model.

And besides, if it’s true that serialization is the be all and end all of dealing with JSON no one would make types like those in the System.Text.Json.Nodes or Newtonsoft.Json.Linq namespaces. They exist both for convenience and because sometimes you’re working with highly heterogenous, schema-less data where the impedience mismatch between your data and the OOP world imposes a cost you deem not worth paying.

Another example that comes to mind is DataTables.

Wait, hear me out!

We’ve had typed DataSets in .NET for almost 2 decades now. Strongly typed. IntelliSense. All that. We’ve also had popular ORMs on .NET for almost as long. Yet often when I look on Facebook or a forum, there I see people using untyped datasets. Why?

My hypothesis is that when thinking about databases DataSets and DataTables are this remarkably minimal abstraction over the natural domain of a relational result set/table/row/column. They are to databases what HTML and CSS are to Web UI (as contrasted with the abstractions of Web Forms). Some people just like that thin abstraction, or need it, or benefit from it. My hypothesis is that untyped JSON objects are the analogous abstraction for the Cloud and NoSQL and that providing them will bring more developers to modern paradigms beyond the ones who are doing it now.

And then there are interactive and exploratory scenarios… if I’m in an interactive or exploratory context (and as part of this modding endeavor I plan to pursue more interactive and scripting contexts for VB.NET) I don’t want to jump through all the hoops of serialization to explore an API or a database. This convenience empowers me at different phases of my project or different activities on a project where speed and ease (for me) are paramount.

Sometimes when I’m mocking out a UI I don’t make any classes, I just whip out some XML literals and mock up an entire data model to bind to in XAML because that’s the fastest way to get a prototype going. And don’t even get me started on all of these reporting-only small classes that only exist for one UI or another, no domain purpose: OrderSlim, OrderSearchResult, OrderSummary, OrderInfo, OrderViewOnPage4. I’m very interested in exploring CQRS and I think raw JSON is a perfect fit for a lot of the Query responsibilities instead of all those fake domain objects.

In short, after all that rambling, there are tradeoffs and these features exist on a spectrum, and I’m making them to give the language more versatility and appeal to a broader set of users and scenarios. Serialization still cool.

What’s Next?

Why, “Wave 2” of course! It won’t be a big secret like “Wave 1”, I’m going to be upfront with what’s going to be in it but I need a little recovery time before getting started on that and to tend to other matters in my life which have gone neglected these last two months. I also plan to blog a little more about the contents of “Wave 1” and release a couple more NuGet packages with JSON literal and pattern matching support for a few more JSON APIs such as MongoDB and Oracle NoSQL.

  • Read this post for instructions on how to install ModVB for yourself
  • Try out the mod—do anything you’re interested in doing with the new features
  • File bugs at https://github.com/modvb/backlog/issues
  • Feedback and discussion on my blog, your blog, Twitter, YouTube, or wherever you hang out with other VB.NET enthusiasts!
  • Don’t forget to SHARE!