Checking in (Wave 2)

Season’s Greetings, All,

I hope this holiday season finds each of you and your families in good health and good spirits. I tried to get this update out before Christmas, but it was not to be. I have several announcements (all positive) to make across several topics both near and far. I know everyone’s wondering what’s the deal with “Wave 2” given that it was supposed to have a release or some demos out this month and I will get to that in this post, I promise, but first I want to get a few quick but important asides out of the way.

Patreon Launch Update

First, let me say a gigantic THANK YOU! to everyone who responded to my call to action regarding my new Patreon. Thanks to your support I’m already well on the way to my first fundraising goal! I’m very pleased with the initial uptake and will be working on getting those tier rewards out ASAP. Every drop of support means the world to me in both a very practical way and in a psychological way. It really keeps the fight in me to know other folks still give a darn.

Also, got some part-time freelance VB contract work, one project of which is thanks to a Patreon supporter answering my call for opportunities. Working on VB projects is always a dream come true so thanks to that special supporter for this opportunity and for the rest of my supporters, please keep ’em coming!

“Wave 6” Theme Update

I wouldn’t normally concern you all with an update on something so far out but I wanted to give my readers some assurances that I have been paying attention to their pain points and am making plans to help. Originally, I didn’t have a clear vision on any specific themes after “Wave 5” (which is themed after LINQ). However, based on observation, a theme for “Wave 6” has emerged around C# interop. More specifically, I’ve seen a number of VB users being blocked by new APIs both in the .NET 6/7 profile and in 3rd party libraries that use features such as the `unmanaged` constraint, stack-only “ref” structure types, and similar scenarios. Some cases where VB is being blocked, such as `required` members, are already being worked on by Microsoft and I sincerely thank them for it, but others they have announced no plans to address. I recognize the pain this causes VB users who are blocked from using these APIs and will be dedicating an entire milestone to designing and implementing some degree of support to unblock you, particularly with a focus on consuming these APIs and I wanted to share that intention with those impacted sooner rather than later. The wave plan is pretty fluid after “Wave 3” balancing several factors including the need for earlier design feedback but another factor is urgency so I may pull this new “Wave 6” in earlier. Stay tuned.

“Wave 2” Scope Update

“Wave 2” is all about unblocking new platforms through the repurposing of VB XML Literals syntax to enable declarative UI development on the web and cross-platform mobile development. This has always been the plan. What’s changed is that for a while now (years, in fact), I have been very cautious in describing the centerpiece feature of this wave as “Enhanced XML Literals”, “Target-Typed XML Literals”, “Pattern-based XML Literals”, or “a XAML-like syntax”, very intentionally avoiding the term “XAML Literals” because that term makes certain promises that I wasn’t willing to casually commit to. My intention was always to target a strategic subset of XAML syntax that would coincidentally produce the same result in common scenarios as what is used in Xamarin.Forms, but wouldn’t be bound to the full breadth of scenarios of a technology like WPF or the formal XAML specification. And for what I prototyped back in 2019 that was good enough.

What’s changed is the scope of technologies being targeted. WPF (and UWP) already support XAML design/code generation for VB.NET projects. The missing stories were ASP.NET Core for Web and Xamarin for Mobile. Xamarin.Forms (and by extension .NET MAUI) was designed to mimic the idioms of WPF but is implemented very differently in how those declarative idioms are translated to code. In places where WPF (and Silverlight) re-invented certain programming constructs in a distinct way from the .NET languages, Xamarin often embraces the “C#”-native way of implementing those idioms. For example, for control- and data-templating, WPF uses an arcane infrastructure behind the scenes using factories to instantiate controls, repeatedly in the case of `ItemsControls`, at runtime (deferred) given a declarative template written in XML. Xamarin.Forms on the other hand embraces lambda expressions. A lambda expression is the idiomatic way in C# (or VB or F#) for passing instructions to an object for how to construct something in a differed way that can then be called into over and over again. If you look at the constructor for the Xamarin.Forms DataTemplate class, it takes a Func(Of Object). WPF by contrast is not nearly so straightforward to use in an imperative code-only way.

Because I only needed to go after Xamarin.Forms, my design for the factory patterns an XML Literal would translate into were pretty narrowly targeted at what would unblock declarative Xamarin.Forms capabilities in VB with only loose consideration of what kinds of patterns would be necessary to target non-trivial WPF expressions.

With all that said, the scope of technologies I’m now pursuing has changed. I’m now targeting:

To be clear, though this might smell like “scope creep” as a language designer this is a really good thing! Designing a feature around a broader set of scenarios generally yields better, more generalized features with greater applicability, which is generally preferable to the accumulation of one-offs.

All of these potential targets are being actively and holistically considered in the design and implementation of this feature. Avalonia UI is the critical addition to this list because unlike Xamarin.Forms it embraces a very WPF-like translation of declarative XAML into .NET objects. Additionally, in designing the patterns for translation I am acutely aware that 1) someone will have to write the “builder” libraries that enable support of a particular technology and their life shouldn’t be overly tedious (e.g. they shouldn’t need to explicitly code support for every dependency property or conversion in an entire framework), and 2) VB users will want to consume 3rd party components written targeting these technologies that have no knowledge of this feature, or themselves may implement controls or user controls targeting those technologies and that I do not want to place a burden on those VB users to mediate integration of those components with the feature.

All of that has led me to the conclusion that I must embrace fully* the XAML-osity of the feature to provide the best experience for VB users. I’m effectively designing and implementing a XAML “meta-processor” inside the VB compiler—a feature that allows one to efficiently describe and implement multiple technology-specific XAML processors in a prescribed way. This design shift raises the bar significantly for the feature but it’s also very liberating in that I don’t need to employ as many workarounds to support certain scenarios in indirect ways. For example, markup extensions (e.g. {Binding}) until now were a concept I wanted to be completely opaque to the compiler with it passing enough information to the builder library that it could then parse and interpret those markup expressions at runtime. Or certain elements of the XAML language like x:Name, x:Type, and x:TypeArguments that are defined by XAML which before were meaningless to the compiler itself and very complicated to reproduce at runtime. Those are restrictions I placed on myself, but now these are things described by the formal XAML specification that the compiler can fairly know about.

I know some folks might find such deep integration of yet another language into the VB language/compiler distasteful and it’s absolutely not a coupling I take on lightly but over the last 14 years, time and time again new technologies have taking bets on the XAML language and I believe it is now stable enough, mature enough, and enduring enough to take a bet on too, not unlike the bet VB (and C#) both took on the SQL-like syntax of LINQ.

That said, even though I’ve fully* embraced “XAML Literals” as the north star of the feature I still put an asterisk on that word “fully” because I want to be clear that I’m still very consciously constraining myself on the minimal integration of concepts absolutely mandated by the XAML spec (with which I’m now becoming intimately familiar) and required to support these targeted platforms. I fully believe that “limitations foster creativity”. Accordingly:

Examples of things I’m ok with the language having some awareness of now:

  • The basic conventions of XAML with regard to object creation and member setting (including “attached” properties and events).
  • Things in the x: (xaml) namespace.
  • That an attribute value represents a markup extension.

Examples of things I’m NOT ok with the language having any awareness of:

  • Any particular technology or assembly names.
  • Any particular technology namespaces.
  • Any particular type specific to any particular technology (e.g. FrameworkElement, MarkupExtension, ContentPropertyAttribute, DependencyObject/DepedencyProperty).
  • Type converters

I’m holding the line on the latter list, which means building the necessary layers of indirection for those scenarios. Every concept the language understands must be very carefully considered, even with this bold new shift. For example, I don’t want VB to “know” about type converters. It’s OK for a builder library to know about them but not for the language to know about them. They come with a lot of design baggage and raise questions I really don’t want to answer right now such as “Why do XAML literals understand and use type converters, but the basic casting syntax does not?” and that would be a fair question if the language or compiler did actually know about them. But as I am implementing them, these dependencies are properly hidden behind the abstraction of a XAML conversion without a direct requirement on how such a conversion is implemented. Further, such knowledge isn’t even hinted at by the XAML specificationthe term type converter does not appear in the spec even once—this is an implementation detail coincidentally associated with WPF but not strictly mandated by the XAML language per se.

Oh, and FYI, here’s a picture of an Avalonia UI app running on Android built using VB!

So, yeah, in summary, they’re called XAML Literals now. I gave up (for the better).

XAML Literal Entanglements with “Wave 7”

OK, technically this isn’t an update because I never told anyone this to begin with but it’s impacting the schedule so I’m sharing to be a little more transparent. There’s a critical piece of the pattern of XAML Literals that was designed in a certain way that’s a little… special. But the plan was to have this specialness be a subset of some specialness that I’d expand on more in “Wave 3” with an otherwise unrelated feature. This matters cause if you do a thing once in a language it’s a one-off, but if you do it two or three times it’s a design pattern and that’s legit so that plan was to do both things the same way and introduce a new pattern.

The problem is that the feature in “Wave 3” also needed to be reconciled design-wise with the existence of source generators—a feature of both the VB (and C#) compilers that Microsoft introduced “recently”. UI frameworks are squarely in the set of core scenarios for source generation and the democratization of source generation should make any language designer consider very careful what goes in the language and what goes in a generator. Until recently I’d written a “TODO” to reconcile new features with source generation with a note that some of the stuff in waves “2” and “3” could change drastically or disappear once I took a serious look at source generation “in the future”. The idea of presenting to you all a design in “Wave 3” which could be obviated entirely by source generators just kept nagging at me so this month I took the time to seriously think through how the mysterious feature in “Wave 3” and source generation would co-exist in one sensible experience and the short of it is that the entire design of the feature in “Wave 3” was obliterated. The design made a lot of sense before source generators were shipped (I’d started designing it at a time when source generators seemed very complicated and unlikely to release) and with them having released, putting some serious design thought into it caused me to throw away … everything other than the end-developer experience I’m holding to, though the implementation will be completely different.

This design sidebar crystalized to me that the theme behind “Wave 7” would be exploring the impact of source generation on language design further. The feature tentatively in “Wave 3” might just move back to “Wave 7”. “Wave 7” might move up. And the piece of design shared between XAML Literals and the feature in “Wave 3” either becomes a strange one-off in the language or needs to be changed. So, I’m changing it to align with the design pattern I now anticipate using elsewhere in the language. It’s coming along well. It simplifies the design, actually. It’s not nearly as “clever” as it was before which is a surprisingly hard to let go of for some reason, but ultimately better for everyone else but my ego. As part of doing things The Right Way™ for ModVB, I strive to never ship an implementation which I can’t say in good faith is expected to be a durable design or implementation. Prototype code is meant to be throw-away, what ships in ModVB is not.

Conclusion

“Wave 2” did not and will not release in December 2022 as planned due to valuable design changes, namely: targeting Avalonia UI to enable cross-platform UI development in VB for Windows, MacOS, Linux, Android, iOS, and the web; embracing the XAML specification as a first-class goal; and the integration of forward-looking design intentions from “Wave 3” and what is now coalescing as “Wave 7”. Implementation is otherwise going well and the deliverables for “Wave 2” are now expected in January 2023. I promise that I believe these changes were the right and necessary decisions to make given the information at hand and look forward to showcasing the fruits of all this labor early in 2023.

I thank everyone for their continue patience and support and wish everyone continued Happy Holidays and a very Happy New Year!

Regards,

-ADG

13 thoughts on “Checking in (Wave 2)

  1. Thanks Anthony for all these details, and good luck.
    I just don’t quit understand the XAML translation thing. Wpf provide System.Windows.Markup.XamlReader that can create wpf objects from a XAML string, and I expect that xamarin and all other techs has similar APIs to do the same, so, why do you need to build such functionality in the compiler? I think the compiler just need to support the xaml syntax, and use the API supported with each tech to process it. Do I get it right, or there is something I am missing here?

    Like

    • I’m absolutely not building that functionality into the compiler but as you say supporting the abstractions to let a support library call them. That’s the kind of templating support I didn’t originally provide flexibility for because Xamarin.Forms could be supported through passing delegates and what I plan for ASP.NET also works with delegates and potentially expression trees. I only intended limited scenario support for WPF but Avalonia is designed with those same patterns that I previously scoped out. The markup extensions are more than just strings though, because they can be nested and depend on compile time context… xml namespace definitions I’ve decided to do more semantic analysis of them. I believe it’ll also yield better startup performance than just delegating to runtime parsing and instancing through reflection. You’re getting it right.

      Like

      • Regarding binding and converters. Using xml literals should make this syntax easier, since xaml here is embedded in the vb code and has direct acess to its context, so no need for converters as vb conversions are applied and bindinding syntax should be simplified using lambdas which will allow to define more complex bidings logic easily (while keeping old syntax available). I also suggested to use shorter syntax for style setters (I will search for the link)

        Like

      • So I was investigating lambda-like syntax for binding and that’s a great topic for another day. For now I’ve pulled back the scope to the existing binding syntax but will revisit cooler bindings in the future. VB conversions aren’t enough, many of the types used in UI don’t define any conversion operators or methods on the destination types at all.

        Like

  2. Any interest in others working on implementation or additional features. I added _ ‘ to parser before they stopped accepting language changes. I also implemented unsigned functions during fluid period and unsigned blocks after they completely stopped allowing those kinds of changes, the last being very complex without modifying the C# code and that was a showstopper for Microsoft. Also a number of analyzers break with ModVB and I have done work on many analyzers in VB for VB. Personally this would for me be a project of love not money as I am very happily retired but bored. All my spare time recently has been on a medical application in VB and ModVB would help me improve it, if others who are helping me could use it easily. One thing it needs to easily allow is conditional compilation with whatever MS is shipping with less functionality.

    Like

    • I think opportunities for collaboration will open up soon. I am particularly concerned about analyzer compatibility. Right now I’m assuming a big issue with shipped analyzers is that I haven’t signed any of the binaries, which I should fix this release. Are you still working on your translator?

      Like

      • Only when I run into a bug that blocks me. If someone submits I PR I will review and accept but 1 person can’t keep up with all the changes happening in C#. I did add enough support for records to handle common cases but they evolved and there were things I couldn’t translate and ran in issues with the #if where VB didn’t support them in places C# does so translation tended to break, best I could do make lots of use if _ ‘ comment, like I do for /* comment */. I really need VB to support GRPC but right now while I translate it it’s a bit of a hack. The difference in what I do vs commercial translators is I handle a lot of stuff where they just give up and for free I do entire solution translation.

        Like

  3. Thank you ADG for a comprehensive update! I can readily appreciate how much work goes into publishing each Wave and just reading your updates gives me insight into what goes on behind the scenes. All the best and Happy New Year to you!

    Like

  4. Congratulations Grandmaster Anthony D. Green, I really like the scope of technologies you are focusing on, keeping VB.NET competitive and with more mature technologies without an excessive number of tools is very important.

    In this new journey of VB, which begins with your leadership, not having confusion and countless tools to do the same thing is very important, being simple and direct has enormous advantages.

    VB for desktop is already wonderful.

    VB for Web needs improvement, but being in ASP.NET and being competitive in various scenarios is what matters.

    VB for mobile needs to be born, but we don’t need to have 50 different options to do the same thing, we just need a simple, objective and well-documented tool.

    VB for MS office needs to be more of an option to VBA and not just point to Python as a possible replacement.

    VB in the browser may be interesting in the future, as it is a trend of C# and Python.

    With all these precautions not to create too many tools that confuse rather than simplify the user, we can be another simple, productive, economical and very viable language option for new projects.

    We can’t make the same mistake as C# which is getting too big, unfocused, ineligible for many developers when the team is large, in addition to having totally confusing documentation due to too many tools. We don’t need to follow this path or even follow the path of C#, C# aims to commercially please new developers, and as new languages appear like Python, C# tries to approach, this in the future may be a success, but it may be make a big mess.

    We need to be practical, readable, fast and well documented, we need to go back to having the simple, easy and RAD language classification and be competitive with Python and if possible even with Golang, which today is an option for C#.

    The Community should soon help with the integration of the main C++, Python, C# and JavaScript libraries making them consumable by VB.

    Like

  5. As for what’s coming up, one major new feature will be support for VB.NET in the browser, expected to come in OpenSilver 2.0, which is planned for release early next year.

    “This will make OpenSilver the only technology out there that supports VB.NET and XAML to build web apps,” Albani said. “It will throw a lifeline to the huge number of Visual Basic developers that are out there, and that are sad to see their favorite programming language being progressively abandoned. As we are flooded with requests for assistance to migrate VB.NET Silverlight applications, we believe that this feature will find a lot of interest in the Microsoft developer community.”

    https://visualstudiomagazine.com/articles/2022/10/17/opensilver-1-1.aspx

    Like

    • Yes, exactly. I likely won’t need to do anything special for OpenSilver because they’re doing it themselves but just in case any plans change I’m keeping myself open. Plus there will eventually be some benefits to my approach around the mixing of declarative XAML and VB code that interests folks. In short, I’m keeping them in mind but if they get to the finish in Q1 like their roadmap says I won’t have wasted any effort. It’s great either way!

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s