How I Survived My Fortieth Birthday

OR “On psychological flexibility”

In this post I’ll talk about my mental health. VB/Tech talk will be back after.

Last month I turned 41 but what I want to talk about is last year. You’ve probably noticed that I haven’t blogged in over a year and that’s a health thing not an interest thing.

I’ve broken down a couple times and am trying to build myself back up. Most forms of communication were very difficult for me for the last year or two. Written, verbal, in person, friends, strangers, authority figures. I lost the ability to make almost all decisions.

I have a not of those neuro-divergences. I’m working with my healthcare time to identify all of them but it’s a lot more than the “clinical depression” that I’ve focused on for the last 15 years. Currently my top of mind is “anxiety”. Not like the general emotion everyone feels before an interview but clinical anxiety. I think I was diagnosed for anxiety back in 2010 but I thought of it as an accent pillow on my depression. I’ve had breakthroughs with the depression but I’m still not ok and I have to find a new definition of normal that doesn’t presuppose a potentially unobtainable goal.

In group therapy we learned about a lot of terms that didn’t hit with me right away but they’re coming in handy now. I have a lot of … “perfectionism”, “all-or-nothing thinking”, “catastrophizing”, “cognitive distortion”. In a lot of ways I didn’t recognize until THIS spring I live in constant fear. Not like, of physical violence but innumerable undesirable outcomes and I’ve become so psychologically inflexible that I’m basically paralyzed. My home until recently was basically a trash dump for recycling, old meds, shipping containers, batteries, and really anything that would require decision making. I had to pay a decluttering service to help me throw it all way. It took 2 people plus myself 2 days and it was exhausting. So many decisions. But I’m a little better.

“Anything worth doing, is worth doing poorly”

There are so many things I don’t want people to think about me and I’m always trying to manage that so the experiment I want to try is to put out more content in what I consider poor condition. I need to write badly. Rushed. Not revised enough. Be ok with sounding erratic. Impulsive. Sloppy. Maybe I can go back, maybe not. But it’s more important that I say anything than that I say it in the ideal way every time.

Maybe if I keep being reckless I’ll stop being afraid and maybe even get some good stuff out. I have to trust that my audience trusts that even if I sound … unprepared that there’s a lot of thought beneath the surface.

I think the most important thing to know is that I am now thinking of myself in my second half of life. Which is a LOT more forward looking than I was at 39. I’m not running out of time; I’m just getting started on the the sequel.

2024 Recap

  • February – I looked into the abyss and the abyss looked back. I reached what I hope is “rock bottom” in terms of apathy. Apathy for everything in the world but terror for myself. I was trying to write to a colleague about my goals and I stared by contrast listing things that I didn’t care about. That list got very very long. I (thought) that I didn’t care about anything really and that’s not a place I wanted to be 8 months before my 40th birthday.
  • I abandoned most of my life. I stopped checking email. My phone was on do not disturb from February to October. I turned off all notifications. I left people and projects hanging without any communication. I was just trying to survive. I owe folks more than just apologies. I gotta do a whole 12-Step amends-type thing.
  • June – Reconnecting with my creative goals on the language design side with ModVB wasn’t exactly a “cry for help” but it was desperate. I told my family and friends that I would be completely self-isolating for 30 days and to not try to contact me until July unless someone was dead or dying. In hindsight I should not have tempted fate with that kind of specificity. No one died but there was a whole thing with mutual acquaintance and the FBI on one side and an urgent double organ transplant emergency within like 2 weeks of my attempted isolation. They’re fine now. I had another terrifying setback in the beginning of July. I couldn’t actualize anything–it seemed.
  • August–I had a bit of a breakthrough about things I’d been carrying since early childhood that were killing me and opened myself up to alternate paths of success.
  • September–the first month I’d had in years where the sky didn’t feel heavy when I looked at it. It was just blue and pretty.
  • October–I wanted to try to do something grand to look forward to my birthday rather than dread it so I’d aspired to go to Miami or something but then like 3 hurricanes hit Florida and that didn’t happen so I just drove around Detroit after midnight blasting the Robocop theme song.

November 2024-February 2025 was cold, mistakes were made, I tried to get back on my feet and got knocked down hard again but fortunately I was able to get new healthcare coverage and begin the long process of healing. My physical health and mental health were tag-teaming me and I was pretty sure I was just destined to decay for the rest of my life but then I went to physical therapy and learned that the pain I’d been living with wasn’t normal aging or one of a host of inevitable inherited ailments I get from either side of my family.

For the first time in like a decade I have:

  • A primary care doctor that’s a doctor and at a medical center that appears staffed and equipped above the minimum.
  • A psychiatrist helping me manage my meds. I’m now on 3 different meds and I expect there will be more and I’m optimistic.
  • A therapist I talk to every week thanks to BetterHelp.com
  • A dentist office within a 20 minute drive.

Then my healthcare premium quadrupled in July but me and the feds are working it out now. I’m not okay. I’m not well. I’m not anywhere near “ready”. I’m still under constant mental assault but I’m learning to recognize when I get stuck and building muscle memory on how to dislodge myself.

A Beautiful Mind

2001’s Academy Award winning “A Beautiful Mind” has been morbidly inspirational in my life now in two ways. It’s biographical film about a Nobel Prize winning economist John Forbes Nash Jr. (played by Russel Crowe). It’s supposed to be an inspiration film about (tortured) genius, in the same vein as 2023’s “Oppenheimer” (about J. Robert Oppenheimer) or 2014’s “The Imitation Game” (about Alan Turing) and it does succeed in that but the brilliant economic work of Dr. Nash wasn’t what impacted me the most.

In 2015 Dr. Nash and his wife died in a car accident. They were riding in a taxi that had a collision and neither of them were wearing seatbelts and were killed as a result. For my whole life I’d been in the (in hindsight weird) habit of not wearing seatbelts in the backs of taxis/Ubers/Lyfts. Not sure why, it was just a kind of common “convention” when I grew up like being back there was magically safer than in a normal vehicle. And then a man lauded as a genius who had this whole big movie made about his life and his work died from not wearing a seatbelt. It was sobering to realize that such a bright light could be extinguished from the world in such a unremarkable way and since then I buckle up every time whether I’m a passenger in a cab or a rideshare or I think I’ll be moving slowly through downtown streets. I share that not to use his death as a rhetorical prop but to beg any readers who might also be in the habit of not wearing seatbelts in back seats or at low speeds or in cabs or limos to drop that habit immediately. Don’t risk that being the end of your story, PLEASE!

That PSA aside there’s now another influence from that film that I had in the Spring of 2025. There’s this critical moment toward the end of the film with Dr. Nash finally accepts that he’s mentally unwell and has been imagining countless interactions with delusions throughout his life. The hallucinations had been so real and what finally convinces him that he has schizophrenia in the 11th hour as his marriage is falling apart and his wife is about to leave him for good is he notices something that feels real but *can’t be true*. “She never gets old!” he tells his wife. The little girl he’d seen in these episodes for decades can’t be real because she never gets old. His disease overplayed its hand so to speak. That one little detail changed his life.

Earlier this year I was trying to write a small sample of a VB program or potential extension of string interpolation for someone online to illustrate a point. And I sat at my desk for like an hour or two frozen and unable to implement it. My mind was racing through all of these considerations but I was paralyzed at my keyboard. And the lies and self-deprecation start to pop up. “Am I a fraud?” “Have a really just been skating by all this time?” “Did I just fool everybody until now and I can’t even implement this?”

And in the midst of that death spiral of doubt I saw the never aging little girl in my delusions. Not only had I implemented this proof of concept sample at least once or twice already in the past (I could remember doing it) but I also implemented the much more robust implementation of actual string interpolation in VB that shipped 10 years ago and has been used by many developers worldwide since (including myself). It simply could not be possible that I was not capable of implementing a thing I’d already implemented like 3 times and shipped once in a wildly available commercial product made by one of the largest tech companies in the history of the world. Those thoughts I was having… couldn’t be real. And so that inability or resistance or obstruction to progress wasn’t from a deficiency of knowledge or skill or qualification but something else. I had definitely been able to do something and wasn’t doing it now–solve for x. And that’s when I recognized it as more than hyperventilating panic attacks–“Oh! THIS IS ANXIETY!!”. You can get to 40 with a shocking lack of self-awareness (I had actually told one of my teammates at my previous consulting company that I was experiencing “cognitive decline”).

I’d been so focused for the last 15 years on the idea that I couldn’t function because I was sad that I hadn’t properly considered how much I was sad because I couldn’t function for a reason other than sadness.

That was not a cure

But it was an invaluable clue and now I’m using this shift in perspective to re-evaluate many blockages in my life and as I am able to try to push myself through. It’s like everything I do throws up big red emergency lights and I have to manually hit “Override” and it’s very tiring but slowly I’m making progress.

Learning from my past

Ok, one last anecdote. In September of 2018 I’d just flew back to Chicago from my first visit to Seattle post-Microsoft. I’d met with my friends and old teammates and ate old food favorites and I was ready to try to turn my attention back to tech after months of being in transition. I came home, put my luggage down, hit the power button on my PC and hopped in the shower. I was so inspired I didn’t want to have to wait for it to boot after the shower. I wanted it ready for me when I got out.

I get out of what I assume was a wonderful hot shower and get dressed and sit at my desk and try to wake the computer and the screen is lit but not showing anything. I didn’t even stop to put on socks so my feet were still a little wet on the wood laminate flooring under my desk, I was making a little puddle–or so I thought. Eventually I found out that the puddle wasn’t from my wet feet but from my liquid cooling system which had ruptured while I was in the shower. One of the tubes had popped off a processor and saturated the video cards and power supply.

My baby was ruined!

Who knows how bad it was. Seemingly everything had been hit by neon green coolant that was supposedly non-conductive but was riddled with impurities from years of my neglecting to change the coolant. I’m lucky I didn’t electrocute myself stepping in the puddle while the box still “on”. This was the worst possible time. I wasn’t yet working again, I was budgeting my savings, that computer was so expensive to build–I couldn’t possibly replace it now. I don’t think they even make that dual-socket motherboard anymore.

And it took a while but slowly I was able to gather myself and take the whole thing apart and assess. To see what could be salvaged. To see if parts could be cleaned. And even though I had built this entire PC from parts myself that was waaaaay back in 2011 and I’d forgotten everything about PC hardware since then. What did all the jumpers do? Where did the cables go? What did you call any of these “things”. Computers had changed so much since when I first learned back in 1996 with IDE cables and Soundblaster sound cards. I’d forgotten about SATA II and all that stuff I had to learn back in November 2011 just to put together a (then) modern PC.

It took a couple weeks but I did ultimately disassemble, clean, and put humpty dumpty back together again (only having to replace the video cards, power supply, and cooling system). I downloaded the manual PDF and re-read the stuff and switched to a fan-based air-cooled system and didn’t break the bank while I was on sabbatical.

Somehow I was able to do again that thing I had already done that for some reason I feared I wasn’t able to do anymore cause… I don’t know. And I did it the same way I did it the first time–one small step at a time.

And aside from one mysterious 2-day hiccup earlier this year that stymied my premature efforts at returning to work, that same machine is still running fine. 14 straight years. I’m going for 15 before I build a new one next year (because I am still psychologically rigid about multiples of 5 but I can only tackle so much of my own “crazy” at one time, give me this).

One step at a time

All that to say, I have to remember that very tall mountains that seem impossible can be climbed one step at a time. And that’s how I feel and have felt for the last 6 months or so. Like I’m at the early hours of a very long climb but that eventually I’ll get to the top. And folks will just have to bear with me if I’m a little (or a lot) slower at climbing than I used to be. I finally have a firm belief that I can get there, even if it’s going to take a while. I needed to share that so folks will understand my fumbling forward in the near future and my long and unexpected silence until now. If I owe anyone an apology for my behavior (and I know I do to many), know that it’s coming. If my writing seems a bit out of character or unpolished for a while, try not to hold it against me.

“I’m all tied up in inside. And if I don’t untie myself inside… the emotional knots… I’m gonna explode!”

I’m learning to give myself the grace I’ve learned to give others and to embrace a greater degree of self-acceptance of my inescapable human-ness. I’ve let myself become the incarnation of not just “Making The Perfect the Enemy of The Good” but making it the enemy of the “At All” and I really want to try something different now.

Faithfully,

-Anthony D. Green

Streamlining and Boilerplate Reduction

The theme of streamlining code and reducing boilerplate extends through the entire ModVB agenda. In the previous section I talked about how improvements in type inference can do this and in future sections Iโ€™ll show more examples that also align strongly with broader themes. This section highlights a few investments that donโ€™t align with other themes but rather are purely planned to delete characters from code while still preserving VBs intuitive and self-descriptive nature.

Immersive Files (incl. top-level code)

The feature in this section with the farthest reaching impact potential is what I have recently dubbed Immersive Files. This includes โ€œtop-level codeโ€ which appears in many other languages, but as is often the case, ModVB extends this concept much further, so donโ€™t skip the details on this one.

Of course, it all began with top-level code and trying to get back to the simplicity I started with in QBasicโ€”that a program could be one or more executable lines of code without all the boilerplate of a `Main` method or a module to contain it

FirstProgram.vb

Console.WriteLine("Hello, World!")

SecondProgram.vb

' This is an entire program.
Imports System.Console

Function Prompt(message As String) As String
    WriteLine(message)
    Return ReadLine()
End Function

Let name = Prompt("What is your name?")

WriteLine($"Hello, {name}โ€‹.")

These are great demos for educational purposes and there was much discussion of these โ€œsimple programsโ€ and wanting to bring scripting and interactive experiences to VB and C# like what F# has had forever. And creating as minimal an on-ramp for new developers is of course critically important but the immersive files feature is designed such that they can have value for any developer or project regardless of skill level.

If youโ€™re an experienced VB developer, when you read the above simple programs you probably came to some quick conclusions about what was going on like that the executable statements run in a Sub Main or that the function Prompt was being declared in an implicit module, which are certainly true of how I imagined those files would be compiled but the great thing about immersive files is that they can be useful for an open-ended number of immersive experiences with completely different compilation (maybe) and execution models. More explicitly,

  • The statements will end up in a method but it doesnโ€™t have to be a Main method nor does it have to be Shared and it could be an Async or even Iterator method.
  • Any non-type declarations will end up bundled in a type, whose name is inferred from the filename, but that type doesnโ€™t have to be a module or even a class. This type can be Partial.

Instead of me as a language designer making these decisions, I have designed the feature so that these decisions can be made by VB developers and communicated to the compiler through attribution from a base class that a file may derive from explicitly through an Inherits statement or implicitly through configuration.

Let’s look at some examples…


Starfield.vb

buffer.Clear(Color.Black)

' Draw 10 yellow 5x5 squares randomly across a 600x400 screen.
For i = 1 To 10
    buffer.FillRectangle(Brushes.Yellow, Rnd() * 600, Rnd() * 400, 5, 5)
Next

In this video I recorded, I have defined a base class called GameLoop that is attributed so that when an immersive file inherits it, any of its top-level statements go into an Execute method thatโ€™s run on every frame and can draw to a graphics buffer. This allows the developer to be โ€œimmersedโ€ in the animation loop of this starfield.


Index.vbxhtml

<?xml version="1.0" encoding="UTF-8"?>
<html>
  <head>
    <title>Welcome to my site!</title>
  </head>
  <body>
    <h1><{ ViewData!Message }></h1>
    <h2>This is a header.</h2>
    <p>Today is <{ Date.Today.ToString("MMM d, yyyy") }>.</p>
    
    <a href="/">Home</a>
    <a href="/home/about">About</a>
    <a href="/home/contact">Contact</a>
  </body>
</html>

And in this video, I use a base class that puts top-level statements inside a Render method that sends HTML from an ASP.NET Core MVC controller to a browser. This example takes advantage of a top-level expression thatโ€™s a single large XML literal which is the return value of the top-level method that the base class then calls when rendering the page to the response stream. This is an extreme though common example where the primary logic of an entire type, in this case a View class, is entirely or almost entirely restricted to a single functionโ€”computing the result markup. This feature allows the developer to be โ€œimmersedโ€ in that all-consuming purpose.


I will use this same technique in the future to enable VB to easily utilize several of .NET’s many popular XAML-based UI frameworks. But in these case the top-level expression will be a XAML literal (coming in Section 4) forming the body of the implicit InitializeComponent of a Window, UserControl, or ContentPage from these frameworks. This approach will bring cross-platform and mobile UI development to VB, without requiring any support from UI framework authors.


Supporting top-level LINQ query expressions (which could be compiled or simply interpreted in the IDE) will allow the creation of immersive interactive live data analysis experiences like what you get in SQL Server Management Studio, LINQPad, or other tools for data science (expect a demo one day).


Or, a VB.NET Script file for automation and management of Windows or other operating systemsโ€ฆ


But even beyond all that fancy stuff, what if Iโ€™m just tired of Class statements?

Hear me out.

In VB6, .frm files were forms I didnโ€™t need an explicit Class statement. The name of the form always matched the name of the file. And in VB.NET we lost that โ€œwhole fileโ€ immersive experience in exchange for the ability to put multiple types in a single file, which is great, but what if we could have it all?

Form1.vb

Sub Button1_Click() Handles Button1.Click
    MsgBox("Hi")
End Sub

Sub Button2_Click() Handles Button2.Click
    MsgBox("Bye")
End Sub

Combined with VBโ€™s already great experiences of project root namespaces, and project-level imports, that file is looking pretty clean. And by making it immersive I save 4 spaces of indentation on every line! One step closer to my beloved single-procedure view.


Which brings us to a conversation about conventions and configuration. I love VBโ€™s project-level imports and root namespaces, but Iโ€™ve long desired the ability to configure them for a group of files in my project with more granularity, e.g. the ability to assign a root (implicit) namespace to all files in a particular folder (usually representing an architectural layer), and/or to add one or more Imports to all source files in a particular layer/folder. And I can definitely imagine wanting to give a particular immersive configuration to groups of source files based on folder, extension, or just naming convention.

So, ModVB will add the ability to configure these settings and apply them based on a pattern which could describe folders, conventions, and extensions (maybe regex, maybe not):

  • Extensions for type kinds? .vbclass, .vbstructure, .vbinterface, .vbmodule
  • Extensions for base types? .vbxhtml, .vbxaml, .vbsql, .vbform
  • Or conventions for base types *Controller.vb, *Form.vb, *Service.vb

Beyond top-level for learning how to code (which, again, is very important), Iโ€™d use immersive files extensively on every project and Iโ€™ve been coding in VB.NET for 20 yearsโ€”Iโ€™ve proven I know how to declare a class by now, Iโ€™ve earn this!

Key Fields & Properties, and Auto-Constructors

Another thing Iโ€™m a little tired of typing after 20 years is very simple constructors for setting fields and properties. I love parameterized constructors! They really were one of the things I was very excited for in VB.NET coming from VB6โ€”the ability to ensure that an objectโ€™s state was initialized on creation really appealed to me and it still does. Thatโ€™s why I continue to initialize my objects through constructors, even for non-public types, instead of just making fields public or other violations of encapsulation. That said, it’s kind of tedious after thousands of times doing it.

To that end, ModVB will allow you to apply a modifier to any class or structure instance fields or properties of your choosing, designating them as Key fields/properties. If no other constructor is provided by the developer, the compiler will generate one with parameters matching the keys and initialize them.

Class AppointmentBookingService
    
    Private Key CalendarService As ICalendarService,
                PaymentService As IPaymentService,
                EmailService As IEmailService

    ' Look at this code I'm never going to have to type again!
    ''Public Sub New(calendarService As ICalendarService,
    ''               paymentService As IPaymentService,
    ''               emailService As IEmailService)
    ''               
    ''    Me.CalendarService = calendarService
    ''    Me.PaymentService = paymentService
    ''    Me.EmailService = emailService
    ''End Sub

End Class

And yes, naming conventions are important to me as they are to many .NET developers, so the compiler will camelCase those parameter names appropriately, though this may be configured. This design has great synergy with the Immersive Files feature because it doesnโ€™t require a class statement. So much boilerplate gone!

Some of you may have noticed that the Key keyword is actually borrowed from VB anonymous types. In vanilla VB, anonymous type properties marked Key contribute to that anonymous typeโ€™s implementation of Equals and GetHashCode and are implicitly ReadOnly. Key fields/properties in regular classes or structures retain this capability, but, to ensure the broadest applicability of the feature, they are not required. Rather, if a class or structure implements IEquatable(Of T), a default override of Equals and GetHashCode will be provided by the compiler using these fields/properties, if the developer does not provide explicit overrides of these members.

Structure Money
    Implements IEquatable(Of Money)
    
    Public Key ReadOnly Property Value As Decimal
    
    Public Key ReadOnly Property Currency As Currency
    
    ' Sub New(value, currency), Equals, and GetHashCode provided by compiler.
    
End Structure

Explicitly marking the fields/properties of a type that are โ€ฆ key to its identity and function also allows the IDE to be more guided in suggestions and code generation for other members, such as operators. This leaves open a question of which members should be generated by the compiler by default and which should be suggested by the IDE if the developer begins defining them. For example:

  • A default implementation of the = and <> operators
  • A default implementation of the Deconstruct method for pattern matching

Iโ€™m leaning toward the compiler generating the = and <> operators and Iโ€™m leaning toward not making an object implicitly deconstructable because that could break encapsulation. One compromise Iโ€™m considering though is a simple rule: Unlike the generated constructor, which takes all keys, the Deconstruct method(s) will only expose those keys which are Public, with the rationale that if a Key field/property is already part of a typeโ€™s public API, itโ€™s not a violation of encapsulation to also make that value accessible via pattern matching.

Pattern matching isnโ€™t in VB yet so I donโ€™t know how popular these methods are going to be, so ultimately the decision on whether to adopt this convention-based approach will heavily depend on usage patterns and feedback, so sound off in the comments!

Wildcard Lambda Expressions

Idiomatic .NET development has changed massively since v1.0. One of the most impactful shifts came after v3.5 in 2008 with the introduction of lambda expressions. Beyond their necessity to support LINQ, lambda expressions have become fundamental to how all manner of APIs work. One pattern in particular thatโ€™s been extremely popular is passing a lambda expression as a kind of strongly-typed โ€œproperty referenceโ€. Indeed, the origin of this feature was my desire to create a syntactic distinction for values in XAML literals that described data-binding paths:

<TextBox Text={*.PrimaryContact.FirstName}/>

Whether that potential use case ever materializes, the feature is generally useful for the many .NET APIs which take Expression(Of Func(Of T1, T2)) lambdas as a declarative property mapping. I felt the need for this feature especially on a recent-ish project using Entity Framework Core, where even C#โ€™s _ => _.Foo syntax very quickly grew tiresome (so many <SHIFT> key presses, itโ€™s like playing piano)

' Vanilla VB
modelBuilder.Entity(Of Blog) _
            .Property(Function(b) b.Url) _
            .IsRequired()
            
modelBuilder.Entity(Of Blog) _
            .HasMany(Function(e) e.Posts) _
            .WithOne(Function(e) e.Blog) _
            .HasForeignKey(Function(e) e.ContainingBlogId) _
            .IsRequired()
            
' ModVB
modelBuilder.Entity(Of Blog) _
            .Property(*.Url) _
            .IsRequired()
            
modelBuilder.Entity(Of Blog) _
            .HasMany(*.Posts) _
            .WithOne(*.Blog) _
            .HasForeignKey(*.ContainingBlogId) _
            .IsRequired()

I feel the wildcard syntax really captures the spirit of how much I donโ€™t care about the variable.

Of course, these expressions arenโ€™t limited to just property access or just one level so expressions like this will also work

Let users = context.Users.Include(*.Profile.Avatar).ToList()

This raises the question about nesting and what the second `*` below should mean?

Let blogs = context.Blogs.Include(*.Posts.Select(*.Comments))

Maybe the syntax should it be **.Comments, or maybe *.Posts..Comments?

Recently I noticed that F# has already added this kind of feature so maybe Iโ€™ll peek at what they do in this case. Sound off in the comments!

SPOILER: Fortunately, in the LINQ section youโ€™ll discover that Include is being added as a new query operator so maybe this was just a bad example.

Markdown Documentation Comment Syntax

While studying the topic of user interaction design (one of my favorite topics), I read somewhere (that I can no longer find) either the statement โ€œThat which is easy will be done often (even if it shouldnโ€™t)โ€ or the corollary โ€œThat which is hard will not be done often (even if it should)โ€. The second one is most important to this section. If you want people to โ€œdo the right thingโ€ you have to make โ€œthe right thingโ€ easy, and certainly not a pain.

I believe in documenting my declarations and I know many other developers who also believe thisโ€ฆ in their hearts. But in our fingers, itโ€™s just a little too much to do for non-public APIs. I want to encourage both new programmers and veterans to document their declarations. Itโ€™s good for maintainers, itโ€™s good for tooling because documentation is surfaced in IntelliSense. To that end, I intend to allow a dramatically simplified syntax for doc comments. After playing around with a few minimalist designs Iโ€™ve decided that rather than demand anyone remember yet another format and despite my general preference for WYSIWYG editing, it would be best to accept a markdown-based format.

'''
''' Returns a function from a description of a line in slope-intercept form.
'''
''' # Parameters
''' - @m: The slope of the line. Must not be @Double.PositiveInfinity,
'''       @Double.NegativeInfinity, or @Double.NaN.
''' - @b: The y-intercept of the line. Must not be @Double.PositiveInfinity,
'''       @Double.NegativeInfinity, or @Double.NaN.
'''
''' # Returns
''' An instance of `Func(Of Double, Double)` delegate type which
''' returns the y-coordinate given an x-coordinate.
'''
''' # Exceptions
''' - @ArgumentOutOfRangeException: Either @m or @b is
'''     @Double.PositiveInfinity, @Double.NegativeInfinity, or @Double.NaN.
'''
''' # Remarks
''' A line can be of one of 3 forms:
''' 1. Horizontal,
''' 2. Vertical, or
''' 3. Diagonal.
'''
''' This API only supports forms 1 and 3.
'''
''' # Examples
''' ## Normal usage
''' ``` vb.net
''' Let y = F(3 / 2, -5)
''' Graph(y, 0 To 100)
''' ```
'''
''' ## Horizontal line
''' ``` vb.net
''' Let y = F(0, -5)
''' Graph(y, 0 To 100)
''' ```
Function F(m As Double, b As Double) As Func(Of Double, Double)
    If Double.IsInfinity(m) OrElse Double.IsNaN(m) Then
        Throw New ArgumentOutOfRangeException(NameOf m)
    ElseIf Double.IsInfinity(b) OrElse Double.IsNaN(b) Then
        Throw New ArgumentOutOfRangeException(NameOf b)
    End If
    
    Return Function(x) (m * x) + b
End Function

The three '‘s are still required (instead of one) to keep devs from accidentally publishing private comments with their public docs.

This is about as minimal as itโ€™s going to get which is great for getting over the laziness threshold to type it but also great for reading online, e.g. viewing PRs on GitHub. No worries, the output format will still be the same XML and XML in the comments will still be supported. In fact, youโ€™ll be able to mix them. There are some great scenarios out there for the open-ended XML syntax such as the <response code=โ€###โ€> tag used by ASP.NET for Swagger API documentation generation and I donโ€™t want to break any of them.

#Ignore Warning Directive

For a syntax that supposedly represents ignoring a warning, it sure takes up a lot of spaceโ€ฆ

' Before.
#Disable Warning BC42356 ' No `Awaits`; don't care.
Async Function SayHiAsync() As Task
#Enable Warning BC42356

    Console.WriteLine("Hello, World!")

End Function
    
' After.
#Ignore Warning BC42356 ' No `Awaits`; don't care.
Async Function SayHiAsync() As Task

    Console.WriteLine("Hello, World!")

End Function

This new directive will ignore the specified warning for the next statement only. Not the next block, the next statement. Itโ€™s actually pretty rare that I interact with warnings in VB or would choose to ignore them, but with the rise of code analyzers this is sure to become more common, so Iโ€™d like to be able to ignore them in a way that I can mostlyโ€ฆ ignore.

New Implicit Line Continuations

VBโ€™s implicit line continuation has been an incredibly impactful addition to the language and an elegant solution to allowing more flexible formatting of code without requiring any explicit line termination. I havenโ€™t heard many requests for new places to allow them over the years but hereโ€™s what I have heard so far

  • Before Then
If someComplexConditionExpression1 AndAlso
   (someComplexConditionExpression2 OrElse
    someComplexConditionExpression3)
Then
    ' Do stuff.

End If
  • Before Handles
Sub Button1_Click(sender As Object, e As MouseMoveEventArgs)
    Handles Button1.MouseMove

    ' Do stuff.
End If
  • Before Implements
Function GetEnumerator() As IEnumerator(Of T)
    Implements IEnumerable(Of T).GetEnumerator
    
    ' Get thing.
End Function
  • Between ) and As in Function statements
Function BuyCar(
           make As String,
           model As String,
           year As Integer,
           Optional trim As String
         )
         As Car

Of course, any changes in implicit line continuation have to be very carefully considered with regard to the IDE experienceโ€”the IDE has to be pretty smart about whether the user is done typing or not and I canโ€™t regress any muscle memory in common scenarios. With that said, also Iโ€™m considering

  • Maybe allowing comments between query clauses?
' Getting customers
From customer In db.Customers
' in Illinois
Where customer.Address.State = "IL"
' with orders
Join order In db.Orders On order.CustomerId = customer.Id
' made last year
Where order.OrderDate.Year = 2023
' from most expensive to least
Order By order.Total Descending
' getting the top 10
Take 10

If this would help you, please send me a better example (instead of my contrived one where the comments are completely redundant)!

  • VERY UNLIKELY: Before commas (,) and various binary operators like AndAlso and OrElse?

I know it’s pretty common in SQL and other languages to put the conjunction on the next line so you can delete or comment out trailing expressions in a list easily but that could be pretty invasive to the typing experience in many scenarios. It would take a lot of analysis to understand the impact of this request, which I’ll happily do at some point in the future. As it stands this is very unlikely.


Please let me know if you have more scenarios. I know I want to do something about leading .‘s and calling fluent-APIs but right now Iโ€™m leaning more toward an IDE-focused solution for a few reasons, so Iโ€™ll discuss that design in a future IDE-focused post.

Bug Fixes & Other Minutiae

Now the lightning-round!

  • Async Main methods will be legal
Async Function Main(args As String()) As Task(Of Integer)

    Await Console.Out.WriteLineAsync("This is an example.")

    Return 0
End Sub

Thereโ€™s no principled reason it wasnโ€™t before, just never got around to it.

  • Optional parameter default value inference

Reusing an earlier example

Function BuyCar(
           make As String,
           model As String,
           year As Integer,
           Optional trim As String ' <-- No default here.
         )
         As Car

Typing = Null and = Nothing over and over again doesnโ€™t actually help me. Itโ€™s just noise and it obscures the important cases where the default value isnโ€™t just the parameter typeโ€™s default value.

  • Optional keyword โ€ฆ optional after first optional parameter?
Function MakeDateTime(
           year As Integer,
           month As Integer,
           day As Integer,
           Optional
             hour As Integer,
             minute As Integer,
             second As Integer,
             millisecond As Integer,
             microsecond As Integer
         )
         As Date

Because in VB optional parameters must be at the end of a parameter list, the keyword neednโ€™t be repeated for every subsequent parameter after the first.

  • Type inference for accessor value parameters
' Before
Property Name As String
    Get

    End Get
    Set(value As String)

    End Set
End Property

' We shall never know the delegate type of this event.
Custom Event NameChanged As EventHandler(Of EventArgs)
    AddHandler(value As EventHandler(Of EventArgs))

    End AddHandler
    RemoveHandler(value As EventHandler(Of EventArgs))

    End RemoveHandler
    RaiseEvent(sender As Object, e As EventArgs)

    End RaiseEvent
End Event

' After
Property Name As String
    Get

    End Get
    Set(value)

    End Set
End Property

Event NameChanged As EventHandler(Of EventArgs)
    AddHandler(value)

    End AddHandler
    RemoveHandler(value)

    End RemoveHandler
    RaiseEvent(sender, e)

    End RaiseEvent
End Event

I donโ€™t know that these type specifiers help folks, but they do make changing a property or event type more error prone.

A little-known secret, the value parameter list in Set accessors has always been optional. If you delete it (including the parentheses) the compiler declares an implicit parameter named Value but the IDE always generates it for clarity. AddHandler and RemoveHandler should also do this to be consistent though personally I prefer having an explicit parameter.

  • Custom keyword in for custom Event declarations not required?

Also did you notice that the second event declaration in the previous example didnโ€™t have a Custom modifier?

When Custom events were introduced to VB I suspect that the Custom keyword made parsing expanded event declarations easier, but when auto-implemented properties were introduced it became apparent that this keyword wasnโ€™t necessary to distinguish the one-line declaration from the multi-line one. While I do find the modifier useful for establishing the vocabulary of how we VB devs refer to such events, I feel compelled to ensure properties and events are consistently flexible here.

  • Optional parentheses for NameOf expressions
Public Shared ReadOnly HeaderTextProperty As DependencyProperty =
                         DependencyProperty.Register(
                           NameOf HeaderText, ' <-- No parentheses.
                           GetType(String),
                           GetType(MyControl)
                         )
                                            
Public Property HeaderText As String
    Get
        ...

Because of the way VB NameOf is designed (itโ€™s a reserved keyword) I donโ€™t think these parentheses were ever required. Theyโ€™re an artifact of thinking of it like GetType() when I should have been thinking of it like AddressOf. What do you thinkโ€”parens or no parens?

  • Open generic types/methods(?) allowed in NameOf expressions
' Vanilla VB
' This is legal:
Shared ReadOnly DictionaryType As Type = GetType(Dictionary(Of,))

' But you're required to write this:
Shared ReadOnly DictionaryName As String = NameOf(Dictionary(Of Object, Object))

' ModVB
' Now this is also legal:
Shared ReadOnly DictionaryName As String = NameOf Dictionary(Of,)

VB allows โ€œopen generic typesโ€ in GetType operands but not NameOf operands; in NameOf we require developers to supply meaningless dummy type arguments. Iโ€™m pretty sure this inconsistency isnโ€™t necessary in VB and is just a historical artifact.

Wrap up

Remember, Folks: The more you say it, the less special it becomesโ€ฆ

I love VBโ€™s declarative and descriptive syntax and will always prefer it over more terse languages with heavy uses of symbols with special meanings. That said, Iโ€™ve been doing this VB.NET thing for a couple decades now and there are a few common patterns of coding that Iโ€™d like to clean up just a little bit so I can focus more on what matters. Hopefully, you agree with these changes and if not or if Iโ€™ve overlooked something that matters to you, donโ€™t hesitate to sound off in the comments!

Thanks for reading! Please share and discuss everywhere!

โ† Type-Inference Enhancements | Overview โ†‘ | Next Section Coming Soon! โ†’

Type-Inference Enhancements

Type inference is great! You get the clean productivity of a dynamic programming language with the benefits of static typing. VB came out of the gate swinging back in 2008 with its type-inference but thereโ€™s yet more that can be done!

Flow-Sensitive Typing

The marquee feature is flow-sensitive typing, which Iโ€™ve been chasing (off and on) for 7 years now. Itโ€™s gone through several different names in that time: Option Infer Turbo, Smart Variable Typing, The Crazy Feature, Smart Casting (what Kotlin calls it), โ€œThat thing TypeScript does whereโ€ฆโ€, Smart Type Testing, and finally Flow-Sensitive Typing. Iโ€™m going to keep using that name because for 7 years I casually searched the internet for a consistent term to describe it across languages and didnโ€™t find one until about a month ago so now Iโ€™m going to abuse it so that others may find it.

I was initially very hesitant to propose this feature to the design team. In fact, initially whenever a VB user would (always cautiously and in private for some reason, like they dare not hope) say to me โ€œYou know what I really want? I wish that if I test the type of a variable and it succeeds, I wish the variable acted as if it had that type inside the Ifโ€. And when this happened Iโ€™d say, โ€œOh yeah, but there are problems with thatโ€ฆ and what we could do is declare a new variable with that typeโ€ฆ pattern matchingโ€ฆ easierโ€. And one day I got to writing out a long reply (presumably to someone online) explaining all the technical challenges and on one side of my brain I was enumerating the problems, and on the other side I was hacking out โ€œwell, actuallyโ€ฆโ€ solutions until finally I convinced myself that it was at least possible to do it.

So, I went to the VB LDM with my โ€œcrazyโ€ idea. I let them know up front that I knew it was โ€œcrazyโ€ but that I just wanted them to suspend their disbelief long enough to mull over some possibilities. And I didnโ€™t exit the meeting with a resounding โ€œletโ€™s do itโ€ but I did get some thoughtful head nods that it wasnโ€™t quite as un-implementable as they thought it would be. And after the meeting, I was chatting to Mads Torgersen said to me in our 1:1, โ€œYou know, I think youโ€™re underselling the feature and yourself. Itโ€™s not crazy at all, this is just one of those problems in programming and some languages solve it using pattern matching and some solve it this wayโ€ so I stopped calling it crazy and kept iterating on the design. Thanks, Mads!

Fortunately for you, you get the skip over the considerable time Iโ€™ve put into how it can actually be implemented in the Roslyn VB compiler without violating certain invariants and architectural truths Iโ€™d rather not disturb. You just get to just focus on the developer-facing implications, which are considerable. Let me walk you through the history

As I said, it all started from a very simple (to make) request: If I test the type of a variable and it succeeds, I want the variable to behave as if it had that type from then on.

As the saying goes, easier said than done.

If TypeOf animal Is Duck Then
    animal.Quack()
End If

Oh, I see now that I’m editing this that I should mention this isn’t Duck-typing, this is a compile-time Duck, I just picked that species by chance.

And, of course, the all time top requested VB feature, to be able to Select Case on the type of a value

Select Case TypeOf animal
    Case Mammal
        animal.SecreteMilk()
        
    Case Bird
        animal.LayEggs()
        
    Case Fish
        animal.Swim()
        
    Case Insect
        animal.Creep()
        
    Case Else
        Throw New NotImplementedException()
        
End Select

Which of course, works with short-circuiting operators like AndAlso and If

If TypeOf animal Is Bird AndAlso animal.WingspanInMeters > 2 Then
    ' Big Bird
End If

Let holders = If(TypeOf quadruped Is Bird,
                 quadruped.Hindlimbs, 
                 quadruped.Forelimbs)

And this progressive refinement of types works across subsequent checks too

If TypeOf animal Is Vertebrate Then
    animal.FlexBackbone()
    
    If TypeOf animal Is Reptile Then
        animal.ShedScales()
        
        If TypeOf animal Is Snake Then
            animal.Coil()
        End If
    End If
End If

This is particularly useful when using old COM style versioned (IVsProject, IVsProject2, IVsProject3) without having to maintain separate variables for each version of the interface.

But itโ€™s not limited to automatic downcasting along the inheritance chain. With interfaces you can have variables of intersection (โ€œAndโ€) types

If TypeOf animal Is Mammal AndAlso TypeOf animal Is ICanFly Then
    ' Members of both types are available.
    animal.SecreteMilk()
    animal.Fly()
End If

Intersection types are not new to VB. Previously, such types were only possible in generic types and methods which used complex constraints

Sub M(Of TAnimal As {Mammal, ICanFly})(animal As TAnimal)
    animal.SecreteMilk()
    animal.Fly()
End Sub

 So, itโ€™s a very natural extension to support them here.

And that brings us to a huge spoiler for a later section. The current design for ModVB is that โ€œNullable Reference Typesโ€ are implemented as a special case of this flow-sensitive typing (as if T were a subtype of T?, such that t IsNot Null is synonymous with TypeOf t Is T)

Let animal As Animal? = SomeFunction()

' (Actual error text subject to change)
' Error: `Move` is not a member of `Animal?`.
' -OR-
' Error: Cannot call member `Animal.Move` from nullable value.
animal.Move()

If animal IsNot Null Then
    ' No error.
    animal.Move()
End If

The further specifics of nullable reference types will be discussed in that section but keep in mind when reviewing this section that much of the infrastructure is shared between them.

Now, 20 years ago, I was 19 and excited about the upcoming VB.NET 2005. One of the new additions to the language was the now infamous IsNot operator. And I thought it would make so much sense if this new operator worked with TypeOf so I filed a feature request onโ€ฆ Connect? Whatever it was called then, and the then VB Language PM Amanda Silver politely replied back โ€œGood idea, but itโ€™s too late in the product cycle, weโ€™ll consider it for next releaseโ€. It did not appear in the next release. Or the one after that. Or, technically, the one after that. But, fortunately in 2010 I had gotten a job at Microsoft working on the VB compiler and I was soon assigned to the Roslyn rewrite and we were redoing all the features again in managed code and I begged the then dev lead to let me implement one of the smaller features, just for fun, ya know, nothing up my sleeve. “Um, how about VB TypeOf?” and he said, โ€œOK, sureโ€, and assigned me VB GetType. And then I clarified that I meant VB TypeOf not the VB equivalent of C# typeof, ya know, like C# is but definitely not VB Is and he fixed it and finally almost a decade of waiting was about the pay off.

Iโ€™d like to take the credit for being sneaky here, but we had a whole code review process, everybody could see what I was doing: I had to change the parse tree and update the IDE. The real credit goes to the reviewers for letting me get away with adding a new feature in the middle of the rewrite despite our strict policy against adding new features at that time. So thanks to them for not raining on my very old parade. Fast forward to Visual Studio 2015 and after 11 years, TypeOf ... IsNot ... finally shipped. The moral of the story is not “Never give up!” but instead that since I added it to the language, Iโ€™m now responsible for it until the end of time (but Paul Vick is also responsible because he added IsNot to begin with when I was young and impressionable).

Which brings us to negative uses

' Works with guard statements (Return, Exit, Continue, Throw).
If TypeOf animal IsNot Mammal Then Return
    
animal.ShedHair()

OrElse

' Monotremes are egg-laying mammals, including the Platypus and the Echidna.
If TypeOf animal IsNot Mammal OrElse animal.IsMonotreme Then
    mayLayEggs = True
End If

Loops too

Let animal As Animal = GetDolphin()

Do Until TypeOf animal Is ILandDwelling
    ' Or maybe I should have said:
    ' Do While TypeOf animal IsNot ILandDwelling?
    animal = animal.GetImmediateAncestor()
Loop

animal.Walk()

Select Case

Select Case TypeOf dinosaur
    Case IsNot Bird
        
        dinosaur.GoExtinct()
        
    Case Else
        ' This metaphor is straining.
        If dinosaur.IsDodo Then
            dinosaur.GoExtinct()
        Else
            ' Expensive O(n) lookup on Wikipedia.
            
        End If
        
End Select

And in queries

Let usStateBearsByHibernationPeriod =
    From
        state In usStates
    Let
        mammal = state.StateMammal
    Where
        TypeOf mammal Is Bear AndAlso Not mammal.IsExtinct
    Order By
        mammal.HibernationPeriod Descending

So far, so good! And this is, with a few caveats, is where the design of this feature stabilized up until about 3 months ago, when consideration of another seemingly unrelated feature much, much farther down my backlog triggered a stray thought.

' In vanilla VB, type inference requires the 2nd & 3rd operands
' to convert to the type of one or the other or type inference
' fails, resulting in type 'Object'.
Dim animal = If(someCondition, New Cat, New Dog)

Under Option Strict On this is an error and under the Option Strict (Custom) settings I use this is also an error but even if it werenโ€™t, this is still suboptimal to me.

In ModVB if there is no target-type (new in ModVB) for the If operator, and no common type between the 2nd & 3rd operands, rather than skipping straight to Object at the bottom of the hierarchy, the compiler will instead search for the most derived common base class or interface

' In ModVB, 'animal' would have type 'Carnivora',  which is the taxonomical
' order common to cat-like and dog-like placental mammals, but that's not
' actually useful because not all carnivorans are carnivores, e.g. bears
' are often omnivores except for the Giant Panda which mostly eats plants.
' So, even though it's inferred that type, the only members I can use are
' defined on 'Mammal' in this example, which is still more than I could
' use if the type were 'Object', so... 
Let animal = If(someCondition, New Cat, New Dog)
animal.SecreteMilk()

But then I got to thinking, if we later added union types to the language, wouldnโ€™t the expectation be that the type of animal were Cat OR Dog? Surely, I should give some thought to that now just in case. Besides, the code โ€œknowsโ€ that itโ€™s a Cat or a Dog, isnโ€™t it wasteful to throw that information away?

And this then became a design principle for the feature, to not throw away type information the compiler has without a good reason. Now letโ€™s take a quick break to talk notation for this brand new world.

  • Letโ€™s start with a type: A
  • Next, as mentioned above, VB already has syntax for intersection types in generic constraints: {A, B}
  • Now we need a syntax for union types: A Or B or {A Or B}
  • {A} is the same as A, so the curly braces can be used for either conjunction or grouping depending on your needs. I tend to always wrap these special types in curly braces because thatโ€™s most likely how they would appear in a program due to operator precedence.
  • Other mathematical properties of association and simplification also apply like {A Or {B Or C}} is the same as {A Or B Or C}
  • Now add another more explicit syntax for intersection types for completion sake: A And B or {A And B}. This means exactly the same as {A, B} and vice versa

Now we need some conventions. Letโ€™s go back to our very first example

1: Let animal As Animal = GetAnimal()
2: 
3: If TypeOf animal Is Duck Then
4:     animal.Quack()
5:
6: End If
7:

On line 2, the IDE will tell me that the type of animal is Animal, as declared. What should it say on line 4? You might expect Duck but there are some implications to that decision. Certainly it can Quack like a Duck and be passed to Duck taking methods (still not duck-typing) but the interesting question is, what happens if I attempt to assign to animal on line 5? The variable was declared to accept any Animal, however, in the range of this If it is temporarily promoted to Duck so is re-assignment allowed? If so, must it be assigned only instances of Duck? if so and I attempt to assign an instance of Animal, is that reference automatically downcast to Duck?

Skipping ahead, the answers are yes, no, and no. Itโ€™s been a common enough experience in my career that Iโ€™m walking up or down tree-like data structures (controls on a form, nodes in a parse tree, entries in a file system, etc.) where I have code like this

Let item As BaseClass = firstItem
Do While SomeTest(item)
    item = GetParentOrChild(item)
Loop

You know, โ€œSet focus to the first control, but if the first control is a GroupBox or Panel, set focus on its first controlโ€, or โ€œWalk up my containers until you hit a thingโ€, trust me, it happens, so itโ€™s important that these variables retain the ability to be assigned and to be assigned any object of their declared type. And beyond that VB doesn’t yet have read-only local variables and this isn’t the time or place to add them.

So, how to denote this in a way that informs rather than confuses the developer. Hereโ€™s the convention Iโ€™ve come up with: When a variableโ€™s type is temporary changed (promoted?), tooling and error messages will show its type as {DeclaredType, TemporaryType}. You can think of this as primary and secondary type, or write type and read type. Applying this to the example

1: Let animal As Animal = GetAnimal()
2: 
3: If TypeOf animal Is Duck Then
4:     animal.Quack()
5:
6: End If
7:

On line 4, the IDE will tell me that the type of `animal` is {Animal, Duck}. Which is true, itโ€™s an Animal And itโ€™s a Duck. This might seem a little redundant but the convention is that the first type in the list always tells you whatโ€™s legal to assign to the variable and any types that follow are what will be read out at this point in execution. So, animal can be assigned any Animal but is currently a Duck.

And naturally this all applies as you might expect to AndAlso

If TypeOf animal Is Mammal AndAlso TypeOf animal Is ICanFly Then
    ' Type of 'animal' is '{Animal, {Mammal And ICanFly}}'
End If

And OrElse

If TypeOf animal Is Bat OrElse TypeOf animal Is Bird Then
    ' Type of 'animal' is '{Animal, {Chordate And ICanFly}, {Bat Or Bird}}'
End If

But the cascade of design implications doesnโ€™t stop here because now it becomes necessary to revisit the whole nullable types thing in light of these changes. Originally, I said that the way it worked was that If t IsNot Null Then was the same as If TypeOf t Is T Then and the type of the variable would automatically change from T? to T and there would be much rejoicing. But now the question is does this logic hold up if instead treating T? like a base type that has a derived type of T, one reasons about T? as a synonym for {T Or Null}?

The full implications of that question will be covered in future sections on โ€œNull and Nothingโ€ and โ€œNew Kinds of Typesโ€ but the short answer is yes for this feature but there are some additional decisions that need to be made if the desire is enforce that consistency, with regard to performance, etc. The end result is that conceptually and if only internally, thereโ€™s a new extremely limited type, the Null type. We have And types, Or types, the Null type, and T? is the same as {T Or Null}

Up until this point Iโ€™d only considered assignment as a thing that could restore a variable to its declared type: If you re-assigned an animal in a way that would break its temporary type you’d be limited to only using it as an Animal again. Part of that was thinking about implementation simplicity but then I get to thinking about this kind of scenario

Let animal As Animal
If choice = 1 Then
    animal = New Cat
ElseIf choice = 2 Then
    animal = New Dog
Else
    animal = New Fish
End If

Whatโ€™s the type of animal after the End If? Just Animal? Surely, weโ€™re not just going to throw away that information? Cโ€™mon! Look at that information. Itโ€™s so adorableโ€ฆ ๐Ÿฅบ

And now that we have these ad-hoc union types we can say with confidence that the type of animal after the End If is: {Animal, {Cat Or Dog Or Fish}}

Or maybe even {Animal, Chordate, {Cat Or Dog Or Fish}} because you may need to access the DorsalNerveCord property for some reason.

The exact canonical display format for such types will be the subject of much rigorous discussion at a future date! Whatโ€™s important to remember is that the first type (Animal) is what can be written to that variable. All other information is supplemental.

But, I hear you say, โ€œWhat about this case?โ€

If TypeOf animal IsNot Fish Then
    fishtank.Add(animal)
End If

โ€œShouldnโ€™t it be a compile-time error if you try to Add an animal thatโ€™s not a Fish to fishtank?โ€ (Note: the alternative is to implicitly-convert animal to Fish and fail at runtime). โ€œSure, itโ€™s not a common case butโ€ฆ THINK ABOUT THE INFORMATION! Doesnโ€™t the compiler look kind of dumb letting you do that when the code clearly says itโ€™s not a Fish?โ€

And I groan. And I sigh. Yep, got me there. So, itโ€™s not the case that a variable can just sometimes gain types that it is, it can also gain types that it isnโ€™t. That is to say, we now have And types, Or types, the Null type, and Not types. And the type of animal inside the above If block is: {Animal, Not Fish}

This leads to some fun display implicationsโ€ฆ would you rather see

{Animal, Not {Cat Or Dog Or Fish}}

Or

{Animal, Not Cat And Not Dog And Not Fish}

Or

{Animal, {Not Cat, Not Dog, Not Fish}}

How about

{Animal, {ICanFly And IEatInsects}, {Chordate Or Arthropod}, Not Bat}

Thatโ€™s going to be fun to bike-shed on later but there is a really cool implication of this with nullable

1: Let animal As Animal? = TryGetPet()
2:
3: If animal Is Null Then
4:    
5: Else
6:    
7: End If

The type of animal on line 4 is {Animal?, Null}, and on line 6 it is {Animal?, Not Null} which for feature tracking when values will and wonโ€™t be null, I think is pretty cool!

It was at this point that my occasional perusing of Wikipedia finally revealed the name of this design to me: Flow-Sensitive Typing, which I am now obliged to use for future SEO purposes. It wasโ€ฆ funnyโ€ฆ to stumble upon the correct terminology only after all the hard work was done, but truthfully it likely wouldnโ€™t have helped. Even this part (from a linked article):

(“you don’t say…”)

If I hadnโ€™t worked through the individual scenarios myself in the context of VB I wouldnโ€™t have appreciated them. So, seeing some prior art was more validating rather than humiliating and believe it or not the significant increase in scope over the original design somehow makes the implementation much simpler (and thus achievable).

For example, thereโ€™s this expectation that Select Case TypeOf implement a thing called subsumption checking. You can actually see an example of it in vanilla VB if you have on the โ€œDuplicate or overlapping catch blocksโ€ warning

Try
    ...
Catch ex As ArgumentNullException
    ...
Catch ex As ArgumentException
    ...
' Warning BC42029: 'Catch' block never reached,
' because 'ArgumentOutOfRangeException'
' inherits from 'ArgumentException'.
Catch ex As ArgumentOutOfRangeException
'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

End Try

Originally the compiler would have reported a warning because an extra step during Case block binding would go back and check previous Case blocks for the same or parent types. Now, youโ€™ll necessarily get an error because as a result of previous cases failing the type of the variable by that Case would be {Exception, Not ArgumentException} so the convertibility testing will fail, no extra work… maybe.

โ€œBut what about Gotos!?โ€

Well, I wrote a program using Goto statements which flowed from bottom to top changing a variableโ€™s type each step (worst case scenario) and tried to follow both how the compilerโ€™s implementation would track the changes and also how a calm and reasonable developer could understand the programโ€™s behavior. I suspect that what resulted was some kind of miniature stroke. As such, while Iโ€™m not necessarily convinced that itโ€™s impossible to implement a design that infers types in arbitrary execution orders (as opposed to top-to-bottom lexical order), I am convinced that doing so would cause grievous harm to the compiler, its implementers, and all who read such code. Therefore, the design is that if you use Goto statements in a way which allows execution to enter a region of code where a variable would otherwise have a certain type but bypass whatever flow-control which would guarantee that type AND then use that variable in a way which depends on it having that type, the compiler will raise an error (essentially a kind of definite assignment analysis)

1: Let animal As Animal = GetAnimal()
2:
3: If TypeOf animal Is Bat Then
4:
5:     animal.Fly() ' Illegal if code outside jumps to here.
6:    
7:     animal.Eat() ' Legal if code outside jumps to here.
8: End If
9:

That said, the design has re-stabilized and I donโ€™t have any expectation to further expand it. But, feel free to do some research on โ€œoccurrence typingโ€ and โ€œeffect type systemsโ€ and shoot me a mail if you think you can change my mind.

Oh, also note that everything Iโ€™ve said above regarding (non-Static) local variables also applies to ByVal parameters (including Me), but not ByRef parameters or fields (except those implicitly declared by the compiler for lambda closures, Async, and Iterator methods) maybe even ReadOnly ones, because their values can change on other threads or otherwise between when the variable is tested and when it is read. Alternately, some of these cases may be allowed for expressiveness but at the expensive of additional runtime checks at read-time. Please discuss.

The rest we can work out on GitHub during the specification phase where we talk about situations like aliasing, and the implications of Is checks (e.g. if TypeOf a Is {Bird Or Null} and TypeOf b Is {Fish Or Null} and a Is b does that mean that a Is Null and b Is Null? โ€œSomething, something, informationโ€. And we can talk about explicit ByRef and Out arguments (Spoiler: Iโ€™m adding explicit ByRef and Out arguments). Oh, not to mention the debugging experience (read: limitations).

Finally, for a discussion of ad-hoc โ€œanonymousโ€ union types (including exhaustiveness checking) as well as intersection types and what it would mean if ModVB supported them explicitly in declarations (not just as a consequence of flow control), look forward to the Section Summary #14, โ€œNew Kinds of Typesโ€ hopefully some time this month, health allowing.

1.2        Conditional and Null-Coalescing (If) Operators

1.2.1        Target-Typing

Both the If(expression, trueValue, falseValue) and If(expression, valueIfNull) operators will be target-typed in ModVB. This means when used in a context that dictates a specific result type, the result operands will themselves be target-typed or converted to that overall result type rather than attempting to find a common type between them. This is an essential change to support intuitive behavior around the 5 typeless/target-typed expressions already in vanilla VB (AddressOf, Nothing, lambda expressions, array literals, and interpolated strings), as well as any added in ModVB (hint). For example,

' error BC36911: Cannot infer a common type.
Dim action As Action =
      If(condition, AddressOf Console.Beep, AddressOf Console.WriteLine)
'     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In vanilla VB this reports and error, requiring you to explicitly cast one or both operands to Action. In ModVB, each operand is necessarily target-typed to the target-type of the If operator (Action) and no error is reported.

1.2.2        Improved Dominant Type

Additionally, as discussed above, the If(condition, trueValue, falseValue) operator will be enhanced to infer the most derived common base class or interface when not target-typed and trueValue and falseValue donโ€™t have a common type. This change should apply to any VB type inference which uses the โ€œDominant Typeโ€ algorithm (conditional expressions, array literals, type argument inference, and lambda functions).

1.2.3        Can Be an L-Value (Assignable)?

Lastly, it is worth discussing whether to change these operators to be L-values (assignable) when their result expressions are L-values, specifically to match the intuitive understanding of these operators as short-hand for the If statement

Let variableIfTrue As Object, variableIfFalse As Object

' Works.
If condition Then
    dictionary.TryGetValue("someKey", Out variableIfTrue)
Else
    dictionary.TryGetValue("someKey", Out variableIfFalse)
End If

' Analogous code fails silently in vanilla VB.
' Will either report an error in ModVB or work correctly.
dictionary.TryGetValue("someKey", Out If(condition,
                                         variableIfTrue, 
                                         variableIfFalse))

Honestly, now that Iโ€™ve written it out, it feels silly to ask, the right choice seems obvious, but it could be a little unexpected to folks coming from vanilla VB so, letโ€™s discuss it.

1.3        Array Literals

Like the If operators, array literals (when not target-typed) will also be able to infer the most derived common base type or interface with one difference: they will only infer a simple type, no supplemental Ands, Ors, or Nots. Itโ€™s a verifiability thing

Let animals = {New Bat, New Bird, New HoneyBee}

You might be thinking the element type of the array is {Animal And ICanFly} but that type only exists to the compiler; the CLR does not have a notion of an intersection or union type so while your code may have only filled it with Animal instances that are also ICanFly instances, any other code, perhaps not written by you, will only see an array of Animal and may fill it with Lion, Tiger, or Bear instances. Yes, the compiler could be made to do some kind of escape analysis to determine that if the reference to the array never escaped the containing method (including by a Static local, lambda capturing, or Async or Iterator hoisting) it could safely retain the more complex type but 1) thatโ€™s a very different and more expensive feature, 2) it would restrict the use of the array so much that it wouldnโ€™t be useful, I think. Much work, not much value. Happy to be convinced!

Youโ€™ll see similar restrictions elsewhere for basically the same reason.

1.4        Type Argument Inference

Type argument inference will likely have the same or greater restrictions as array literals with the same reasoning. Given the following

Module Cache(Of T)
    Public Shared Value As T
End Module

Function M(Of T)(a As T, b As T) As T
    Return Cache(Of T).Value        
End Function

Cache(Of Animal).Value = New Shark

Let result = M(New Bat, New HoneyBee)

Itโ€™s safe at the call-site of M to infer T as Animal but not that T is also ICanFly.

1.5        Multi-Line Function Lambdas

Multi-line Function lambdas (including Iterator lambdas), which also use the dominant type algorithm (when not target-typed), should also benefit from inferring the most derived common base class or interface when there is otherwise not a common type between Return or Yield expressions

' Dominant type only matters in a multi-line function lambda with multiple
' 'Return' or 'Yield' statements.
Let getFlyingAnimal =
      Function(mustMakeHoney As Boolean)
          If mustMakeHoney Then
              Return New Bee
          Else
              Return New Bird
          End If
      End Function

In the above, the type of the lambda expression is

<Function(Boolean) As {Animal, ICanFly, {Bee Or Bird}}>

Note that this inference does not have limitations on And or Or types as they are safe in this context. Furthermore, through VBโ€™s normal conversion rules it should be possible to convert this lambda (or the variable getFlyingAnimal) to a nominal delegate type which returns Animal such as Func(Of Boolean, Animal), or returns ICanFly such as Func(Of Boolean, ICanFly).

1.6        Bug Fixes & Other Minutiae

  • This code will no longer error and will correctly infer the return type of Create
Let customer = Customer.Create()
  • Static local variables will correctly infer their type from initializers
Static cache = New Dictionary(Of Integer, Integer)

1.7        Wrap up

Yikes! That was a lot!

As you can see, ModVB will level-up VBโ€™s already powerful type inference capabilities to make the language even more expressive with fewer explicit types, fewer temporary variable declarations, and fewer explicit casts required, all without sacrificing performance.

Thanks for reading! Please share and discuss everywhere!

Overview โ†‘ | Streamlining and Boilerplate Reduction โ†’

The Agenda

At long last, in the last hours of the last day of the week that marks the 33rd anniversary of Classic Visual Basic, I am finally ready to share with you my comprehensive vision for the future of the VB.NET language via ModVB.

As mentioned, last year I started out with an 11-page outline including some code examples in Word, noting ideas for some 110+ investments in completely new features or modifications to existing features collected over 12 years as a VB.NET developer and 8 years at Microsoft as a VB.NET language designer combed from countless personal experiences, customer interactions, bug reports, forum questions, etc. I set out to decompress this outline into effectively a book of concrete, thought-out designs that I could share to illustrate the language-specific scope of ModVB.

If you’re a new reader, ModVB is a Visual Studio extension I’m developing which adds new functionality to the VB language and tooling. You could think of it as a modernized extended Director’s Cut of VB to empower and be enjoyed by VB enthusiasts such as myself. I’m both attempting to crowdfund and crowdsource the project so it’s urgently important that potential supporters and contributors have an idea of what all they’re supporting.

Having said all that, while the exercise of methodically and holistically working through the entire outline has absolutely paid off, my previous draft had 120+ individual blog posts (I called chapters) written in specification-ese, and it became apparent that most folks wouldn’t see the forest for the trees. “What’s in ModVB?”

Any normal human: What’s in ModVB?

Me: Start at this link and click the “Next” link 120 times. If you skim it you’ll be done in a couple months and it’ll all make perfect sense to you.

So I’ve been aggressively editing and refactoring for the last several weeks and I decided this anniversary was my hard-deadline. Instead of my design process memoirs in 120 chapters, I’ll be sharing a mere 18 section summaries illustrating the various features primarily in concise, compelling, code. Not endless prose. Code with a few sentences here or there and I’m going to trust my audience to ask questions when they need, provide criticism when they feel, and to generally trust that I’ve done my homework on this. As short as an example may be, the process to vet the design (both for worthiness and achievability) was not short. All of these investments have a scenario somewhere. A question I was asked, probably more than once; an API I used or was blocked from using; I tool I or someone else wished they had. And every feature I propose is a feature I know I can personally implement in the Roslyn compiler (not that I will be implementing all of them). Maybe that’s a fault in my designs that I limit myself to the box of my own capabilities but it’s also my bedrock.

Further, this is not a wish list. It’s not an unfiltered collection of ideas accumulated that might or might not happen. As it stands upward of 85% of the things I show here are virtual certainties at this point. These features are shovel ready for an experienced compiler dev, and speclet ready for newbies. It’s not a question of if they’ll be implemented, just when. A small number have already been prototyped and demoed before on this blog or implemented already. There will be adjustments. Learnings. Scope changes. But this is what’s coming both because I am a VB enthusiast and am personally excited to use the vision of the language I describe, but also because creatively this is the thing which is in my soul and I have no choice but to see it through. Maybe I’ll end up homeless, destitute, coding at a public library in some warmer climate on an old laptop I aggressively guard with animal-like ferocity–I hope not–but a life creatively unfulfilled is far more terrifying.

For this first post I have included a brief description of each section and for all be the last 2 a highlight. This is not necessarily the “most important” feature in that section. In fact, it probably isn’t. Instead, it’s a selection meant to intrigue you. Especially in those sections where the top-of-mind feature is one I’ve demoed or discussed already. I want you to know there’s more and for you to feel interested in clicking the link to the full section summary after it’s posted. Each section has 6+ features it covers. A few will contain (link to) dedicated discussions of particular features which are on or near the line where your feedback and scenarios are critical. I’ll update this post as each section comes online.

Finally, the caveats for a better reading experience and better feedback:

  • Neither the sections nor the features in them are presented in priority order; do not assume because you see it first that I think it’s most important (and that you need to correct me). This isn’t a status update or a work item “backlog”; I’ll create one of those on GitHub with these items but having these descriptions available now massively facilitates a future discussion on when and how to realize these features.
  • This is not a list of everything required before releasing a v1. In fact, having now seen the entire scope I don’t think it would be good to dump that much on the world at once. It’s not a release schedule; that conversation will come later. There’s no value now in trying to “cull” some features to make it more likely some other features will happen sooner or at all.
  • While I am capable of implementing all of the features, the plan is that I won’t. Other VB community members have actually already implemented some in their own private branches and I’ll bring on contributors when I’m ready to support a proper functional responsive open source repo (with your help).
  • You might think a big part of this process was reviewing features already in other .NET languages but that’s not the case. The goal of the exercise was to find the ultimate expression of idiomatic VBness and see where that leads. I haven’t been closely looking at any other languages on .NET or off (except T-SQL) but I have a vague awareness of some new features here and there from names or screenshots I’ve seen on Twitter in the last few years. Even if a feature shares the same name as a capability in another language though the design is pure VB and hasn’t been influenced by how that capability was realized anywhere else. That said, for interop reasons alone there must be a period of reconciliation at some point in the future. This is expected, just not before the full VB-sourced vision is published.
  • I know the published VSIX and packages are woefully out-of-date. This is shameful to me. I published those manually (hand editing manifests) and my next step is to set up a proper CI/CD pipeline so that I can release effortlessly. If you’re an expert in that, please reach out!
  • If Patreon doesn’t work for you, I’m also considering other means: GitHub sponsors, Open Collective, etc. Open to suggestions. If you can’t give what you ideally would want to just give what you can. A once-annual $24 donation now is better than a $100/month donation never. If you can’t give there or now just leave a comment with your email or something, join the VB Discord, so I can keep you informed if new opportunities open up.
  • I need more scenarios for the designs I’ve created; if a scenario isn’t important to you personally, that’s ok, but don’t trash it to deprioritize. There’s someone for whom that scenario matters.
  • This is just the language/compiler/runtime stuff. There will be future (smaller) posts about IDE, tooling, and community. Look forward to it!
  • These are not “asks” or wishes. Please don’t go to Microsoft asking for individual items piecemeal. I’m not seeking permission, and we’re way past asking for permission.
  • Share and discuss everywhere!

And with that, here are the 18 sections. Right now these just include a teaser highlight but over the coming days and weeks I’ll post full section summaries separately covering all the functionality and then update this document with a link to the section summary after the highlight/short description.

The 18 Sections

Oh, just a heads-up. Because ModVB changes some semantics it might be confusing to developers whether a piece of code is using ModVB semantics or vanilla VB semantics. To make it more readily apparent the convention I’m recommending to distinguish them has two parts:

First, ModVB local variable declarations are introduced with the Let keyword (a callback to earlier BASICs). Functionally it’s identical except in a particular case I’ll cover later but mostly it’s just a visual indicator. All but the most trivial programs will include some local variables so making this stylistic change there makes it very likely that the semantics of the code will be clear.

Secondly, I’m trying out using lowercase keywords by default in ModVB. It’s still case-insensitive and I’ve actually modified the CSS on this page to only appear lowercase. When syntax colorization isn’t available I’ll likely still use PascalCase and/or bolding to emphasize keywords because I think it’s a little easier to read. I’d really love to hear feedback from readers about both of these conventions and how you feel about them. Thanks!

Finally, don’t let semantic changes confuse you, ModVB is 100% fully back-compatible with vanilla “in the box” VB provided by Microsoft, including all current and future updates. A project can choose to default to vanilla semantics and opt-in to modern semantics on a per-file basis, or vice versa. And tools to highlight and assist any transitions will be provided. It’s critically important that all existing vanilla VB code continue to work as is even while we push the frontiers or productivity.

Type-Inference Enhancements

ModVB will feature several enhancements to type inference to make it a lot smarter so you can get more benefits from static typing with fewer keystrokes.

Highlight: Control flow-sensitive local variable typing

Let control As Control = SomeFunction()

' Variable types narrow when tested or assigned.
If TypeOf control Is CheckBox Then
    control.Checked = True
End If

' Intersection (And) Types are possible.
If TypeOf control Is IButtonControl Then
    ' TypeOf control Is {Control And IButtonControl}.
    ' Members of both are available.
    control.Text = "Search"
    control.PerformClick()
End If

' Union (Or) Types are possible.
If enableFormatting Then
    control = New RichTextBox
Else
    control = New TextBox
End If

' TypeOf control Is {Control, TextBoxBase, {TextBox Or RichTextBox}}.
control.MaxLength = 200

Read the full section summary

Streamlining and Boilerplate Reduction

ModVB will include many new and improved features to make you more productive with fewer keystrokes across almost every category, but this section covers those that donโ€™t align with other themes.

Highlight: Developers can mark fields and properties with the Key modifier. This informs the compiler to generate the default constructor with matching parameters and (optionally) implementations of GetHashCode and Equals

Read the full section summary

' Sub New(id As Guid, accountNumber As String) is generated by default.
' camelCased (configurable) parameter names are inferred.
Class AccountInfo

    Key ReadOnly Property Id As Guid

    Key ReadOnly Property AccountNumber As String

End Class

Let info = New AccountInfo(Guid.NewGuid(), "12345")

General Modernization and Evolution I

Almost every basic kind of statement in VB is being revisited with some tweaks, including historically the most requested features from VB users!

Highlight: The Select Case statement will be enhanced with many new tests including identity, types, shapes, and more

' Identity
Select Case sender
    Case Is Button1
        
    Case Is Button2
        
    Case Is Null
        
End Select

' Type (works with control flow-sensitive typing)
Select Case TypeOf obj
    Case String
        
    Case Integer
        
End Select

' Shape (pattern matching)
Select Case ShapeOf int
    Case b As SByte
        
    Case s As Short
        
    Case Else
        
End Select

Full section summary coming soon!

UI, XML, and XAML

Itโ€™s no secret by now that ModVB is upgrading XML literals to full XAML literals to enable more productive uses such as web and cross-platform mobile development. Iโ€™ve published several videos of earlier prototypes of this work already so for this section Iโ€™ll highlight something you havenโ€™t seen already.

Highlight: XML/XAML literals will allow embedded block expressions allowing for the use of all statements when generating nested content

<html>
  <body>
    <{
      For Each customer In customers
          If TypeOf customer Is PreferredCustomer
              <div>
                <h1><{customer.Name}></h1>
                <p><{customer.PreferredStatus}></p>
                <p><{customer.EmailAddress}></p>
              </div>
          Else
              <div>
                <h2><{customer.Name}></h2>
                <p><{customer.EmailAddress}></p>
              </div>
          End If
      Next
    }>
  </body>
</html>

Full section summary coming soon!

JSON and JSON Pattern Matching

ModVB already supports JSON literal expressions and JSON pattern matching so Iโ€™ll highlight new functionality planned for these features.

Highlight: JSON literals will be enhanced to not only create JSON types like JsonObject and JObject but to initialize arbitrary .NET types. This will smooth transitions during development from weakly-typed APIs to strongly-typed APIs but also will accelerate automated test authoring in combination with JSON-producing technologies such as SQL Server and Postman

Let testContacts As InMemoryRepository(Of Contact) =
    [
      {
        "name": "Tyrion",
        "email": "gotaw@lannister.wes"
      },
      {
        "name": "Jon",
        "email": "bastard@stark.wes"
      },
      {
        "name": "Dany",
        "email": "khaleesi@targ.ess"
      }
    ]

Full section summary coming soon!

Strings and String Pattern Matching

Strings are everywhere! Several enhancements are planned for working with them more productively.

Highlight: Pattern Matching will be enhanced to support an interpolated string syntax for matching and inspecting strings; these can be used anywhere patterns can be specified, including within other patterns

Select Case ShapeOf username
    Case $"{user}โ€‹@{domain}โ€‹"
        ' External email
        If domain = "mydomain.com" Then
            Throw New Exception("Please use domain account.")
        End If
        
    Case $"MYDOMAIN\{user}โ€‹"
        ' Internal domain
        
    Case $"{domain}โ€‹\{user}โ€‹"
        ' Affiliated domain
        
End Select

Full section summary coming soon!

Generalized Pattern Matching

Beyond JSON and XML, ModVB will support generalized pattern matching for arbitrary .NET objects.

Highlight: Developers will be able to define, compose, and use their own custom named pattern methods

If ShapeOf coordinate Is Polar(r, theta) Then
    
ElseIf ShapeOf coordinate Is Cartesian(x, y) Then
    
End If


If ShapeOf str IsNot ip As IPAddress Then Return

Full section summary coming soon!

General Modernization and Evolution II

Basic statements arenโ€™t the only language elements being revisited in ModVB. Declarations and expressions are also getting overdue updates based on common usages and longstanding community requests.

Highlight: ModVB will add a pipeline operator (->) to VB, letting you call any method like an extension method (postfix syntax) and to make nested function calls, such as in data transformations, read in execution order (left-to-right, top-to-bottom) instead of inside-out

' Before
Dim last6 = Right(str, 6)

Dim result = ComputeResult(obj)

Dim emails = GenerateEmails(ComputeInvoices(GetMonthlyOrders(calendar), priceList), addressBook)

' After
Let last6 = str -> Right(6)

Let result = obj -> ComputeResult()

Let emails = GetMonthlyOrders(calendar) ->
             ComputeInvoices(priceList) ->
             GenerateEmails(addressBook)

Full section summary coming soon!

LINQ Enhancements

LINQ was introduced to VB 16 years ago. ModVB will add new query operators as well as integrate queries with other constructs. Iโ€™ve previously shown a prototype of integration of queries with For Each so Iโ€™ll highlight another example here.

Highlight: Object member and collection initializer enhancements

' Initialize any collection with a query.
Let dayNames =
      New HashSet(Of String)(StringComparer.OrdinalIgnoreCase)
            From
              day In DayOfWeek.Sunday To DayOfWeek.Saturday
            Select
              day.ToString()

' Nest member and collection initializers.                               
Let order = New Order(orderId) With {
                  .OrderNumber = orderNumber,
                  .Options With {
                     .ExpeditedShipping = True,
                     .GiftWrapping = True
                   },
                  .Items From item In items Where item.Quantity > 0
                }

Full section summary coming soon!

Late-Binding/Dynamic Programming Enhancements

Late-Binding has been a part of VB since the beginning. Whether interoperating with Office or other COM APIs, or emulating the untyped or gradually typed systems of dynamic languages for rapid development, late-binding has its place in modern development. To that end, ModVB will include several investments to make late-binding more powerful than before so that VB programs can achieve maximum productivity wherever they fall on the static-dynamic spectrum.

Highlight: ModVB will change the late-bound type from Object

Let lbo As LateBound = New ExpandoObject

lbo.Number = 10
lbo.Increment = Sub() lbo.Number += 1

' Before calling Increment.
Console.WriteLine(lbo.Number)

lbo.Increment()

' After calling Increment.
Console.WriteLine(lbo.Number)

' Implicit type of (non-local) declarations is LateBound
' rather than Object, for typeless programming a la
' Python/Ruby/JavaScript.
Class Person
    Key Name, Age

    Sub Say(Optional message)
        If message Is Null Then
            Console.WriteLine("Hello! My name is " & Name)
        Else
            Console.WriteLine(message)
        End If
    End Sub
    
    Function GetFutureAge(yearsFromNow)
        Return Age + yearsFromNow
    End Function
End Class

Full section summary coming soon!

Async Enhancements

Itโ€™s been 12 years since Async methods and Await expressions were added to VB. Itโ€™s time to take what weโ€™ve learned to make asynchronous programming even more productive and more deeply integrated into the language.

Highlight: In ModVB the result of an asynchronous method invocation can be โ€œdottedโ€ into (among other things) immediately; the asynchrony will propagate to the resulting value

' Before
Dim service = New RemoteStorageService

' Nobody wants to write it like this:
Dim blob = Await (Await (Await service.GetFileAsync("filepath")).OpenAsync()).ReadAllBytesAndCloseAsync()

' So, we write it like this:
Dim fileHandle = Await service.GetFileAsync("filepath")
Dim stream = Await fileHandle.OpenAsync()
Dim blob = Await stream.ReadAllBytesAndCloseAsync()

' After
Let service = New RemoteStorageService

' Write it as fluently as synchronous code.
Let blob = Await service.GetFileAsync("filepath") _
                        .OpenAsync() _
                        .ReadAllBytesAndCloseAsync()

Full section summary coming soon!

Null and Nothing

Nulls are real and inescapable. ModVB will treat them such but with deep language integration that makes working around them safer and more productive than ever!

Highlight: Expressions that use the new ? operator or various null-safe operators (e.g. ?.) will propagate null-safety to certain operators and statements that use them

' Before
' If obj is null, this will attempt to await a null task, and throw.
Dim collection = Await obj?.GetCollectionAsync()

' Is it this?
Dim collection = Await If(obj Is Nothing,
                          Task.FromResult(Nothing),
                          obj.GetCollectionAsync(cancellationToken)
                       ).ConfigureAwait(False)

' No, it's probably this.
Dim collection = If(obj Is Nothing,
                    Nothing,
                    Await obj.GetCollectionAsync(cancellationToken) _
                             .ConfigureAwait(False)
                 )

If collection IsNot Nothing Then
    For Each item In collection
        
    Next
End If

' After
Let collection = Await obj?.GetCollectionAsync()

For Each item In collection?
    
Next

Full section summary coming soon!

Declarative Programming and Code Generation

VB has a strong historical tendency toward declarative styles of programming. With the advent of source generators even more opportunities are on the horizon for developers to say what they want while delegating the details of how to the system. ModVB will include several investments to support a breadth of declarative scenarios both with and without source generators.

Highlight: Special โ€œSmart Attributesโ€ will enable auto-property and auto-event implementations to reuse common patterns without the use of source generators

Class ListingInfo
    Implements INotifyPropertyChanged
    Implements IDataErrorInfo

    ' Smart-Attributes can be combined to transform and react to values one
    ' after another.
    <Trim, Notify, Undoable, MaxLength(25)>
    Property Title As String = "New Listing"

    ' Compiler has no knowledge of specific Smart-Attributes, only patterns
    ' of integration at well-defined points such as 'on entry',
    ' 'before set', 'after set', 'on leave', etc.
    <ThrowOnNull, Notify, Undoable, MaxLength(50)>
    Property Description As String = "No description."

    ' Smart-Attributes can take dependencies on members of types in which 
    ' they are applied.
    <ForceToUtc, Notify>
    Property LastSaved As Date

    <Notify, Undoable,
     Regex("^\d{3}-\d{2}-\d{4}$", "Must be of the form '###-##-####'.")>
    Property ListingCode As String = "<None>"

    ' Simple implementation of INotifyPropertyChanged and IDataErrorInfo
    ' omitted for space.
    ' Could be user-declared, generator-declared, or in a base class.
End Class

Full section summary coming soon!

New Types and New Kinds of Types

This section mostly entails discussions of potentially integrating new types (e.g. Half and Int128), new type syntax, and new kinds of types (e.g. Or types) into the VB language.

Highlight: Could a library of units of measure be implemented in Vanilla VB? What would that look like? What are the minimal and ideal changes that could be made to the language to provide the best experience? Is it even worth it?

Let G = 9.8(In m/s^2)
Let speed = G * 5(In s)

Full section summary coming soon!

Inheritance, Interface Implementation, and Extension

Object-based programming techniques such as implementation inheritance are powerful but sometimes a little too rigid. What can we add to VB to make OOP less tedious?

Highlight: ModVB will allow types to delegate some or all of an interfaceโ€™s or base typeโ€™s overridable members to contained fields (or properties)

' Composition over inheritance made easier by delegating interface
' implementation partially or wholly to contained objects.
Private ErrorHelper As New DataErrorInfoHelper Implements IDataErrorInfo

Full section summary coming soon!

Performance and Interoperability

The .NET platform is always evolving and itโ€™s important to VB enthusiasts that we can continue to leverage new APIs (e.g. Span(Of T)) and write code that benefits from the latest platform performance improvements. ModVB will approach each scenario at the right level of abstraction while leaving escape hatches for edge cases.

Highlight: The ModVB compiler will recognize calls to certain well-known methods and emit these calls as CLR native instructions. This will give VB users access to the full expressivity of the .NET runtime even when a language-specific syntax for such instructions is not available

' Because IL literals would be too much.
Imports VB.Runtime.NativeInstructions.IL

' Emits IL similar to the following code in C#:
'
' int* ptr = stackalloc int[32];
Let ptr As IntPtr = 
      sizeof(Of Integer)() ->
      conv_u() ->
      mul(32U) ->
      localloc()

Full section summary coming soon!

Runtime Library Enhancements

The VB runtime library is more than a collection of useful functions, itโ€™s required for virtually any VB program to execute using the languageโ€™s semantics. From late-binding to simple intrinsic conversions this library must be kept up to date, both with the evolving language and its uses. Several investments will be made to optimize the performance of the runtime, fix bugs, and add new and improved functionality.

Full section summary coming soon!

Deprecations

As stated in my introduction, ModVB will be 100% back-compatible with vanilla VB insofar as projects may configure whether to use modern or vanilla semantics by default, and source files may further opt into (or out of) modern semantics individually. Combined with the partial types feature, code can migrate to ModVB semantics with method-level granularity.

That said, within the modern semantics there are obviously changesโ€”cases where the exact same syntax behaves differently under modern semantics. Iโ€™ve tried to be very intentional with these changes and to keep them few and either high value new functionality such as improved type inference, or to places where the entire language can be made more self-consistent for current and future users. However, beyond these cases, itโ€™s worth discussing a very small number of language constructs or semantics that exist in vanilla VB but which might not be supported moving forward under modern semantics. This section contains a summary of proposed deprecations for discussion.

Full section summary coming soon!

What’s Next?

Next, I upload the remaining 19 parts of this publication (18 section summaries + wrap up/next steps). I’m hoping to upload one full section summary every 2-3 days. It’s a process of moving content from Word to WordPress, along with manually colorizing, checking, and correcting dozens of code samples. Sometimes this may go faster or slower but I figure just for the sake of the audience it’s best to post no more frequently than once per day so that you have a change to read the entire section, noodle on it, leave some feedback, etc. I’ll be trying to keep active in the comments and on Twitter so feel free to reach out. When you’re staring down this much clerical work, it’s nice to come up from air now and then.

Please share and discuss… everywhere!

Regards,

-ADG

Special Thanks to ANYONE who is donating or has ever donated to me via Patreon! I could not pursue my dream of producing modern VB features and content without their support.

ALEX T., JONATHON M., DR. DONNA M., DR. ANNE J., KEVIN R., KIRILL O., NEAL G., GOVERT D., TAIWO A., PETER B., AND PAUL C.

Autumn 2023 Update #2

Announcements

OpenSilver 2.0 adds VB.NET supportthis is AWESOME! Please go show some love to those who support our community โค๏ธ

@DualBrain has set up a VB Discord server for all dialects of VB, Classic and .NETJoin Today!

Hey all,

In my last post I said I might be finished in November. That didn’t happen and the publication I’ve been working toward is now coming up on 200-ish pages. I’m aiming through editing to keep it in that range (for comparison, the entire VB language specification is < 700 pages). Editing has been a mix of expanding some chapters and reducing/combining others. I’m really liking the improvements to flow now with each section having 1-3 deep chapters (like XAML literals) and the rest be shallower chapters. I know this document is huge so I’m really trying to keep it from being boring or monotonous. I’ve added a section (technically I moved some chapters from other sections to a new section). I am considering (no promises) a stagger publication of one section at a time, but I haven’t yet committed to that approach.

A friend has convinced me to come up for air this month to get some other things in my life ready for the new year so I won’t promise this monster of a publication will be done this month. I’m still working on it but I’m trying to be more realistic about other end-of-year competition for my time, including holiday time with family.

Happy Holidays and Warmest Regards,

-ADG

Special Thanks to ANYONE who is donating or has ever donated to me via Patreon! I could not pursue my dream of producing modern VB features and content without their support.

Dr. Donna M., Alex T., Kirill O., Neal G., Govert D., Dr. Anne J., Taiwo A., Jonathon M., Peter B., Kevin R., and Paul C.

Autumn 2023 Update

Announcements

OpenSilver 2.0 adds VB.NET supportthis is AWESOME! Please go show some love to those who support our community โค๏ธ

@DualBrain has set up a VB Discord server for all dialects of VB, Classic and .NETJoin Today!

Hey all,

Itโ€™s November already. Iโ€™m still plugging away at the ModVB Language/Runtime/Compiler backlog reveal. Iโ€™m currently working on the section on โ€œNull and Nothingโ€. This was the last section to be added explicitly and is the least or second least developed. After this I just need to pump out a few pages on performance and interoperability and I think Iโ€™m on track to drop the massive collection of posts this month.

In the meantime, a community member over on the VB Discord server asked for a quick reminder of the highlights of whatโ€™s planned for ModVB. I paired down my documents to something โ€ฆ highlight-y and I thought others would benefit from the summary. The exact order of sections is subject to change but the spirit and highlights of each are below.

Remember, ModVB is not one feature or a few features but a focal point for the continued evolution of the VB.NET language, based on the Roslyn codebase. With regard to actual capabilities/features/fixes, there are currently over 100 distinct work items on the backlog in the language, compiler, and runtime library, broadly organized into a dozen or so buckets (subject to change). Please understand these highlights are nothing approaching exhaustive.

UI, XML, and XAML

Enhancing VBโ€™s existing XML capabilities for modern front-end development targeting the web and mobile.

Highlight:

  • XAML literals
  • + at least 5 more!

JSON

Replicating VBโ€™s amazing XML story with the JSON data format.

Highlight:

  • JSON literals and JSON pattern matching.
  • + at least 4 more!

Smart Casting and Pattern Matching

Features for testing types and shapes of data and inspecting complex data structures.

Highlights:

  • Checking a local variable/parameter with TypeOf will cause its type to change where the compiler can infer it is safe.
  • The top-requested VB feature: Select Case TypeOf
  • General pattern matching against types, tuples, wildcards, strings, and other user-defined named patterns.

Streamlining and Boilerplate Reduction

Does what it says on the tin.

Highlights:

  • Key Fields & Properties, and Auto-Constructors
  • Top-Level Code
  • Type-Inference improvements
  • + at least 10 more!

General Modernization and Evolution

Revisiting almost every statement in the language and a few other constructs to improve them in some way, long requested features, consistency fixes, etc.

Highlights:

  • Tuple deconstruction in several places in the language such as local variables, For Each loops, queries, etc.
  • Select Case on object identity.
  • โ€œClosedโ€, Generic, and Nestable Module declarations.
  • + at least 15 more!

Declarative Programming and Code Generation

Features which make it easier to reuse functionality across repetitive tasks both with and without source generators.

Highlight:

  • Source Generator support features
  • + at least 2 more

Asynchronous Programming Enhancements

Deeper integration of asynchrony with the language.

Highlights:

  • Await Each blocks
  • Async Iterator Functions
  • + at least 4 more!

LINQ Enhancements

Revisiting LINQ after 15 years with new features and fixes to make working with data even more productive.

Highlights:

  • Queries in For Each (and Await Each) blocks.
  • New operators
  • + at least 7 more!

Dynamic Programming  Enhancements

Revisiting VB late-binding to make it work better and work in more places.

  • At least 11 topics!

Interfaces, Inheritance, Extensions, & Delegates

Making OO fundamentals more productive.

Highlight:

  • Implicit Interface Implementations
  • + at least 6 more!

Null and Nothing

Making working with the reality of nulls in code safer, more elegant, and more consistent.

Highlight:

  • Nullable reference types
  • + at least 3 more!

Performance and Interoperability

Improvements to let VB developers write faster code for performance-critical components as well as interoperating with modern features of the .NET platform.

Highlight:

  • ByRef Structure story
  • + at least 4 more!

Versioning

Better features for dealing with or initiating change between library versions.

Runtime Library Enhancements

Revisiting the VB runtime library so that itโ€™s more portable, more performant, and more useful.

Highlights:

  • Runtime helper optimization
  • My namespace modernization
  • + more.

Summary

If you were keeping count thatโ€™s over 100 separate topics! After I finish writing Iโ€™ll move things around some to make things flow better and Iโ€™m trying to combine certain very similar topics where it makes sense and each topic may be anywhere from half a page to 10 pages for the largest topics so itโ€™s not an exact page count but as promised, this โ€œpostโ€ will be massive. My next update should be that the artifact is โ€œtext completeโ€. See you back here soon-ish.

Regards,

-ADG

Special Thanks to ANYONE who is donating or has ever donated to me via Patreon! I could not pursue my dream of producing modern VB features and content without their support.

Dr. Donna M., Alex T., Kirill O., Neal G., Govert D., Dr. Anne J., Taiwo A., Jonathon M., Peter B., Kevin R., and Paul C.

Update on the Colossal Backlog Publishing Endeavor

I need every drop of writing power in me to go toward writing this monument so I’ll keep this short. Per my last post I’m taking my 12-page outline for the ModVB language enhancements and expanding it out into a high-ish level conceptual overview of each enhancement with a short-ish motivating scenario and/or example of each. I hoped to be done by end of July. That was pure foolishness. As it stands this overview will definitely be over 100 pages printed!

Unlike my original Exhausting List of Differences Between VB.NET and C# post, this won’t be a single monolithic document. To make it easier to publish, revise, read, comment on, link, etc. instead I’ll be publishing each enhancement as a separate post. They’ll all be published at the same time and linked from a master table of contents (and maybe an index) which will be the “post”. Right now that means something over 110 (and counting) separate posts each averaging about a printed page, organized into 14 sections (and counting).

I’m trying to keep them all at a similar level of detail which is more than “I got an idea”/suggestion on GitHub/UserVoice level but not as detailed as a feature speclet that a inexperienced compiler dev could immediately start work off. I have several reasons for not wanting to publish each post one at a time over the next 4 months, one of which is that it would mean publishing each post one at a time over the next 4 months. But also, doing it this way will empower us to have a much better conversation about the entire scope of the backlog, prioritization, and how and where it could make sense for future contributors to pitch in.

Sorry it’s taking so long, but this is the culmination of 20 years of professional experience working in and on VB.NET, we’re all going to have to be patient with it. Thanks!

Regards,

-ADG

Scaling out

Announcements

  • Long-time VB MVP Cory Smith has set up a Visual Basic Discord server for all dialects of VB including VB.NET, Classic VB(6), VBA, and VB Script. All are welcome! Please consider joining today!
  • Several supporters have mentioned that Patreon is blocked in some regions of the world and that would be supporters would appreciate an alternative. I thank them for bringing this to my attention and am investigating additional platforms for crowdfunding, though more importantly Iโ€™m planning for more ways to contribute.
  • VB support for consuming C# required members has been implemented and is expected in VS 17.7!

Intro

Hey all,

Youโ€™re loooooooong overdue for some updates so letโ€™s get to it. So far this year has been an adventure and Iโ€™m going to give you a whole rundown of events in the second half of this post but first I want to talk about some changes (good I think) Iโ€™m going to be making to be more transparent as well as help capture and accelerate momentum for ModVB and related projects in the near future. The project isnโ€™t vaporware, I just suck at scaling (I apologize) and Iโ€™m pivoting to address that.

Whatโ€™s going to happen moving forward

Iโ€™ve been giving tremendous thought to how I can better scale. This ModVB project is important to me and a (slowly) growing number of followers and it deserves more than itโ€™s been getting. Also a few of community members, including past community contributors to the Roslyn project, have reached out offering to pitch in. Taking on more contributors has always been in the cards and weโ€™re rounding the corner to the right time to do so. One consideration to doing so has always been my own bandwidth for working with contributors and testing and reviewing code coming into the project. Iโ€™m not going to loosen up on my quality bar for the project, but I am readying myself mentally and practically to function in the role of project manager, tester, and code-reviewer. I also respect that simply having more hands involved in the work will lead to more steady progress and build confidence for would-be adopters that this project is a safer bet.

Here are some more concrete steps that Iโ€™m working on:

(Finally) publishing most of compiler/language/runtime backlog

Iโ€™ve frequently mentioned the 11 (now 12) page outline of language changes I have for tracking planned and potential modifications to VB. I know Iโ€™ve been pretty secretive about this list so far mostly just to avoid feedback flooding but also showmanship and other reasons. Those days are fast coming to a close. And Iโ€™m not going to just publish the outline as is but a true to form Anthony-style โ€œWall of Textโ€ laying out each feature, key design points, motivating scenarios, and in even my thoughts on what the โ€œdemo scriptโ€ for the feature would look like. This will go waaay faster than my current approach of waiting to do everything myself and revealing things only with a < 5 minute YouTube video and will empower others to go after the low-hanging fruit in parallel with my own work. In fact, one VB community member already has implemented a feature that I’m very interested in pulling into ModVB.

Iโ€™m putting this post together to inspire both contributors and supporters with my vision for the future of the VB experience that I truly believe is exciting and achievable by the community, completely independently. This document will be the basis for both project planning and myriad strategic and design discussions Iโ€™m literally aching to have so โ€ฆ Iโ€™m really excited to finally be preparing it to share! I sincerely hope that finally seeing most of the big picture will put new winds in this project’s sails.

Setting up the repos for more contributors (than me)

  • Setting up repos and testing infrastructure for
    • Compiler/IDE (Roslyn)
    • Runtime
    • Compiler extensions (e.g. JSON literal/XAML literal bindings)
    • Project templates
    • Moreโ€ฆ
  • Setting up docs for contributor success
    • Licensing
    • Contribution guidelines
    • Compiler tutorials
    • Feature speclets and full specifications (over time)

Timeframe

The timeframe Iโ€™m targeting for most of this is early-mid summer, so, either late June or sometime in July and just in time for the 1 year anniversary of my announcing ModVB.

And the XAML literals + top-level code work is still coming. Itโ€™s not vaporware, itโ€™s just approaching release asymptotically while Iโ€™ve been juggling other projects and life events. Which brings me toโ€ฆ

Whatโ€™s happened so far

When I decided, a year ago, to take some time off to kick start this ModVB thing, I imagined (and planned, and saved for) 5 months or so. Basically a nice self-funded sabbatical to dedicate myself wholly to a creative endeavor that truly matters to me. I want to re-emphasize that number: 5 months. I figured that was enough to get through the first couple of waves and parachute into a new job somewhere. SPOILER ALERT: The mass layoffs, various hiring freezes, and general contraction of the tech sector starting right about when I was supposed to deploy this metaphorical parachute quickly showed me the error of my presumption. So, when I write to you a year from the start line you can understand that at least 7 of those months of unemployment were completely unplanned for. Iโ€™ve basically been operating in survival mode since last fall and the great news is that I survived. I survived with a lot of help, from family, friends, clients, and those of my readers who became Patreon supporters.

Highlights

I was (briefly) writing a VB.NET book for a popular publisher

Thatโ€™s right! In the beginning of this year I was approached by a popular publisher about authoring a book on VB.NET, adding to my list of 3 or 4 projects I was juggling at the time. I was excited for the opportunity but ultimately after going through the initial steps of authoring, I realized that in just explaining the feature work in ModVB in the near future I was already committed to writing like a bookโ€™s worth of content and that it was insane to try to write TWO books at once while also juggling project work. Maybe next year.

I changed health insurersโ€ฆ and lost my entire healthcare team and all my appointments

I switched to a more reasonably priced health insurance plan which has been a major relief. However, because healthcare is never simple in this country, doing so moved my entire healthcare team โ€œout of networkโ€ and basically meant restarting my entire healthcare regimen from scratch with a new primary care person, a new team, etc. I still havenโ€™t really recovered from the switch though fortunately my medications have been maintained.

Stunlocked

Having said that, it turns out that managing a workload of software projects from multiple contracts, managing my (mental) health, and managing ModVB are all full-time jobs on their own. Being one person, Iโ€™ve naturally been sucking at all of them.

โ€œEnough about your personal problems, is this XAML stuff ever going to happen?โ€

Yes. Hereโ€™s what goes into a normal feature for me.

  • Designing/implementing/testing the feature in the compiler
  • Designing/implementing/testing basic IDE functionality around the feature in editor
  • Designing a compelling motivating scenario to demonstrate the feature in under 5 minutes and implementing any demoware sample projects that go with it

This is normally like ~2ish months of work. But XAML literals have extra steps because they implement a base pattern in the compiler but to really exercise the feature you need a mature implementation of the pattern. By comparison, the design of LINQ is a very meaty set of features in the compiler but imagine if LINQ couldnโ€™t be coherently released or demonstrated without first implementing several LINQ providersโ€”not just LINQ to SQL or Entity Framework but two or three other implementations. This is where the tail end of development has just been dragging on. I canโ€™t just say voila XAML is implemented in VB now but I need to release it with:

  • XAML bindings for .NET MAUI
  • XAML bindings for Avalonia

These are important because while they both use the same XAML syntax the way that XAML is translated is wildly different, which is good because it exercises the generality of the feature but I also have to prototype enough of XAML bindings for a hypothetical/prototypical VB Web library which doesnโ€™t yet exist but which weโ€™ll want to start building pretty much immediately after launch. And each of those bindings has to be tested as well. And consider that the feature has to both be tested in a unit test in the compiler which does not import any of these UI frameworks into the compiler test dependencies, i.e. Iโ€™ve had to mock compiler extensions representative of all the real scenarios that need to be supported in in the initial release but ALSO have separate automated test suites for each of the real extensions. Right now Iโ€™m trying to distill a kind of โ€œVB XAML standardโ€ test profile that can be reused across multiple implementations to hit a set of key scenarios (e.g. object graph deserialization, data binding, styles, templating). It might seem like Iโ€™m boiling the ocean and maybe I am but what I donโ€™t want to do is ship an incomplete pattern which then requires me to ship new revisions of the ModVB compiler every week or two as the implementations of compiler extensions uncover and fix more bugs. Now thereโ€™s a dependency hell where you need to upgrade the extension libraries AND the compiler constantly to get anything to work. I would like the compiler pattern to be mostly stable and the extensions themselves to do more of the churning.

All of that would be a lot if I werenโ€™t, as always, struggling with mental health issues, and if I werenโ€™t juggling multiple projects with multiple clients that Iโ€™m trying to meet obligations with. To put it into perspective, out of the entire backlog this is both the most important and the single largest feature I ever plan to implement in ModVB. Thatโ€™s why my original (flawed) plan was to do this kind of work during a period where I could focus almost entirely on ModVB. But itโ€™s OK, because soon talented folks are going to be able to help me scale better.

Summary/TL;DR

Juggling lots of smaller contract gigs and personal issues has really slowed progress but thereโ€™s more bandwidth on the horizon. Regardless, itโ€™s always been the expectation that ModVB would welcome some additional contributors and that time is fast approaching this summer; preparations are being made, most importantly of which is publishing a wall of text detailing the full backlog for the language so that both contributors and supporters can gauge their interest to chip in. XAML work still incoming, gated by a high quality bar.

XAML Literals Design Updates and Implications

Wave 2 is progressing steadily though I think itโ€™s approaching an inflection point. January was very challenging balancing out working on XAML literals alongside my contract work, but I still feel like Iโ€™m converging on a release. I want to share more insights into whatโ€™s going on behind the scenes, so I wrote this post. Itโ€™s really a chance for me to just nerd out about design so if it doesnโ€™t excite you thatโ€™s ok. I find it cool because Iโ€™m a language design nerd. To appreciate some of the recent design changes and their implications you need some background on compilation that Iโ€™m going to try to sum up. A few years back I wrote a primer on Roslyn and how compilers work. Folks have been asking me to pick that series back up (and I know Iโ€™ll need to to involve more folks in mod development) so think of the first part of this post as a draft of doing so.

A quick review of compilation

At a high level, compiling a VB.NET program is broken into 4 phases:

  • Syntaxโ€”what you type
  • Semanticsโ€”what it means
  • Loweringโ€”reducing high-level VB constructs like loops and Await into low-level CLR constructs like Goto (itโ€™s almost all gotos at the CLR level)
  • Emitโ€”writing the actual bytes that the CLR will load and run to an EXE or DLL file

The scope and separation between the first two are whatโ€™s relevant for this post.

Syntactic analysis means taking a sequence of characters literally one character at a time and assigning those characters to structure based on the set of rules that most people intuitively think of as โ€œthe languageโ€. Syntactical analysis itself happens in two โ€œphasesโ€. I put that in quotes because in a textbook college-level computer science class you might think of them as โ€œphasesโ€ that happen one after the other but technically in Roslyn they happen at the same time and are very interconnected. Those phases are scanning (also called lexing, or lexical analysis, or tokenizing) and parsing.

Anyway, if youโ€™re reading this blog youโ€™re almost certainly a VB.NET developer so I can just describe these concepts in terms of VB without straining analogies to natural languages like English.

Scanning

Given the string of characters: Await A.B(C) โ€˜ Comment, scanning chops the string up into a sequence of 7 tokens:

  • Await
  • A
  • .
  • B
  • (
  • C
  • )

Scanning of course uses whitespace to separate tokens but whitespace (in VB) isnโ€™t itself a token (other than newlines). Comments are, like whitespace, not tokens, theyโ€™re just something the scanner knows to skip over.

Parsing

The parser takes that flat sequence of 7 tokens and builds a tree structure

  • ExpressionStatement
    • AwaitExpression
      • AwaitKeyword: Await
      • InvocationExpression
        • MemberAccessExpression
          • Identifier: `A`
          • Dot: `.`
          • Identifer: `B`
        • ArgumentList
          • OpenParenthesis: `(`
          • Argument
            • Identifier: `C`
          • CloseParenthesis: `)`

So, in this example the scanner decides that A, B, and C are identifiers (as opposed to keywords, punctuation, operators, or literals) but not what they refer to. The parser is what uses those tokens to build structure of expressions and statements (and declarations): C is an argument to an invocation, B is (presumably) a named member of A, etc. But again, parsing doesnโ€™t decide what any of those identifiers refer to. Maybe none of those identifiers refer to anything and the program is an error. Maybe A is a namespace and B is a function in a module in that namespace. Maybe A is a local variable, B is a property that returns a collection, and C is being passed to the default property of that collection. The parser doesnโ€™t determine that, thatโ€™s what Semantic Analysis does.

Interdependence

Again, in an academic context lexical analysis and parsing are very neatly separated but in practice in VB.NET the parser directs the scanner with context that only it knows about. For example, the parser knows whether itโ€™s parsing inside of an Async method and thatโ€™s what determines whether Await is a keyword or an identifier (and thus whether Await (x) is a parenthesized expression being awaited or something called Await being invoked with argument x).

This familiarity between the parser and the scanner are what make it almost trivial to embed languages like XML, JSON, and the String.Format syntax for interpolated strings into VB; the parser is what knows whether itโ€™s โ€œinโ€ an XML literal, JSON literal, or interpolated string and thus instructs the scanner to tokenize characters appropriate to that context.

Token values

Part of scanning also involves assigning what in Roslyn we call the value of a token. So identifier A, escaped identifier [A], suffixed identifier A% all have the value โ€œAโ€โ€”thatโ€™s the name that identifier declares or refers to. Decimal integer literal 10, hexadecimal literal &H0A, and binary literal &B1010 all have the value of integer 10. The value is part of the token, is whatโ€™s used by the compiler during semantic analysis, and is separate from text of a token. Another more relevant example is the string literal โ€5โ€™8โ€โ€โ€ which contains an escaped quotation mark, the value of that string is 5โ€™8โ€ what would be written to the console if you printed that string token. Likewise the value for XML text tokens in an XML literal replace XML character and entity references like &amp;, &#38;, &#x26; with the character &.

Immutability

Now, the last thing to keep in mind about the phases of compilation is that the output of parsing is an immutable syntax treeโ€”a hierarchical data structure that cannot change once created and is fed into semantic analysis (along with project references) to determine the meaning of a program. Thereโ€™s limited examples of parsing re-interpreting a scanned token in the moment (e.g. around contextual keywords) but thatโ€™s still in the midst of parsing. Once parsed thereโ€™s no going back during semantic analysis to decide that the developer typed something else.

OK. I think thatโ€™s enough of a review for the rest of this post.

A XAML Literal is a semantic distinction, not a syntactic one

With that understanding, note that whether or not an XML literal in VB represents a XAML literal is a semantic distinction. XAML is XML and syntactically (to the parser) a XAML literal will look like any other XML literal. Itโ€™s during semantic analysis that the XML namespaces in scope (which, remember, may be imported at the file or project level) are examined and checked against the available runtime extensions to determine what that XML translates to. Whether thatโ€™s an XElement instance, a System.Windows.Controls.StackPanel instance, or a instance of a class from an entirely different library is all determined semantically.

Markup extensions

This is all mostly good and well until you come to implement markup extensions because markup extensions have their own syntax.

In my original prototypes, I didnโ€™t implement any special handling of markup extensions in the compiler. There were just hooks for extensions to process attribute values before assigning final values to members and those extensions could inspect XAML values for things they knew about like StartsWith(โ€{Binding โ€œ) and do whatever they wanted without the compiler having to know anything about it.

There are a several problems with this approach.

Markup extensions can be nested

OK, so one problem that comes up is that markup extensions arenโ€™t just a pattern of simple values, they can be nested:

Text=โ€{Binding Name, Source={StaticResource Company}}โ€

But this isnโ€™t the end of the world. Yes, it means the any implemented extension has to be able to recursively parse and process all of the supported markup extensions recursively. Annoying, but doable.

Markup extensions can be user-defined

Supporting all of the familiar markup extensions like Binding, StaticResource, DynamicResource, etc, would be very tedious on its own. But then you have to consider that XAML allows anyone to define their own markup extensions. And itโ€™s not unreasonable to expect that if a user has defined such an extension that theyโ€™d be able to use them in a XAML literal just as they would in WPF, Silverlight, or WinUI.

So, ok, the extensions can try to parse and resolve custom binding extensions. Since these extensions will likely be mapped to arbitrary XML/XAML namespaces rather than in the default namespace it does mean the compiler must have a way to supply the XAML namespace mappings in scope for the extensions to do this but for other scenarios (type converters) the compiler would have to do that anyway so itโ€™s not the end of the world. It does mean a lot of parsing, converting, and reflection at runtime which is a perf hit but again, itโ€™s doable.

Users should expect a rich IDE experience typing markup extensions

If you were typing a XAML literal what kind of editor support would you expect? Completion on XAML namespace prefixes? Markup extension type names? Maybe data-binding property names? What about colorization within the attribute?

Technically the Visual Studio and Roslyn APIs do supply enough hooks that someone could, with a lot of work, implement these things over an opaque string (e.g. RegEx syntax colorization). Butโ€ฆ

Markup extensions can have errors

Since all of this handling of markup extensions is deferred until runtime through extension helpers, when something goes wrong the way it would naturally be surfaced to a developer is at runtimeโ€”which stinks. What if you mistype a markup extension nameโ€”wouldnโ€™t you want a compile-time squiggle for that? Of course you would! And then what about syntax errorsโ€”if you fail to close a markup extension youโ€™d expect to hear about that early too.

But thatโ€™s where things get tricky! Because as I mentioned a while back, a XAML literal is a semantic distinction. In other words, โ€ฆ

<TextBox Text=โ€{Binding Nameโ€ />

โ€ฆis an error if and only if that XML element expression is also a XAML literal. It would be too grievous a breaking change if otherwise valid XML literals all of a sudden started reporting syntax or semantic errors based on the fact that they would be erroneous XAML.

At first glance this made me think that compile-time error reporting should be left to an analyzer rather than the compiler because the compiler has a higher bar for correctness than an analyzer, which can only report warnings which can be ignored. Now you have a situation where the XAML provider extension is reparsing attributes values at runtime, and the IDE and analyzers are each reparsing attribute values as you type which really erodes the promise of Roslyn. So, eventually I decided that I wanted to do some analysis of markup extensions in the compiler.

Now that the compiler is processing markup extensions, it has to be decided how this information is represented. Naively, youโ€™d add to the Roslyn Syntax API several nodes modeling markup extension syntax like you would when adding any new syntax to the language. But once again, remember that we only know that an XML literal represents a XAML expression at binding timeโ€”during semantic analysis. It would be โ€ฆ problematic for the syntax API to report that something โ€œlooks likeโ€ a special kind of XAML attribute. But, maybe not too crazy. By analogy, the compiler already has the concept of โ€œstructured triviaโ€. This technique allows the compiler to attach structure to a comment in a way that sits beneath its normal representation and this is what is used for XML documentation commentsโ€”interested parties can inquire on whether a comment has structure and other parties can treat it as just a comment. Maybe I could do something like that with the XML text tokens of an XML literal but Iโ€™m extremely hesitant as while structured trivia (comments) are a concept in the Syntax API, thereโ€™s no such analog for a potentially structured token.

Circling back to the question of how errors are represented, โ€ฆ

<TextBox Text=โ€{Binding Nameโ€ />

In our example we only want to report a syntax error here for a XAML literal, not a regular XML literal. But as I mentioned, once created a syntax tree is immutable. We canโ€™t decide later that it has syntax errors or that it does not have syntax errors. So this is the cleverness that I propose:

We parse the markup extension but throw away any errors we find. If there were no errors we produce a valid WellformedMarkupExtensionSyntax node and if there are any errors we produce a valid MalformedMarkupExtensionSyntax node. In either event we donโ€™t report ANY errors during parsing. When it comes time to bind a MarkupExtensionSyntax, if itโ€™s a WellformedMarkupExtensionSyntax AND contained in a semantic XAML literal we bind it using XAML rules. If itโ€™s a MalformedMarkupExtensionSyntax (inside a XAML literal) we reparse the entire expression but reporting this time reporting โ€œsyntaxโ€ errors alongside semantic errors. If itโ€™s not a XAML literal we ignore the structure and just use XML value of the text in a regular XML literal.

Voila! Now we can have our cake and eat it too. We can report โ€œsyntaxโ€ errors for markup extensions but only inside XAML literals, and we can provide rich structure to the IDE and analyzers that they donโ€™t have to recreate for themselves, and XAML provider extensions donโ€™t have to do almost anything because the compiler is handling the semantic analysis of the markup extension (which is a XAML concept by the way, so itโ€™s entirely reasonable that the compiler would do this). This is great because the actual rules for parsing and binding โ€œtextโ€ inside a markup extension are arduous enough that to force anyone else to re-implement them would simply be mean spirited.

Spoiler: This isnโ€™t what I ended up doing.

Markup extensions II

As I indicated in my last post, this whole thing has been part of the process of embracing the scope of the core feature of โ€œWave 2โ€ as a true to name XAML literal, not just XAML-like enhancements to XML literals. This shift in mindset both frees me and binds me to lean on the formal XAML specification. For the most part this has been liberating and the XAML spec is actually a bit of a fun read. It describes a type system for XAML without actually depending on that type system being implemented by the CLR or any specific XAML vocabulary such as WPF, with notions such as โ€œassignable to the x:MarkupExtension XamlTypeโ€ rather than the CLR notion of inheritance from the WPF MarkupExtension class. It describes relationships between โ€œtypesโ€ and โ€œmembersโ€ and lookup without depending on the concept of a CLR property or classes or even static typing per se. If this doesnโ€™t sound like a fun read to you, thatโ€™s normal. I am not. Not anymore.

So, there I am reading the XAML specification and for the most part everything is making sense and on page 72, in Section 8.6.3 โ€œMember Node Creation from an XML:attributeโ€, I come across this one line:

Let attributeText be the string xmlAttribute[normalized value].

โ€ฆand my head explodes.

The normalized value. Not the text as written. Thatโ€™s when it clicks that the processing of attributes isnโ€™t an alternate interpretation of XML but a subsequent step. In other words, in XAML (and you can verify this in the Visual Studio XAML editor right now), all of these (and more) are equivalent and perfectly valid markup extension usages:

  • {Binding Name}
  • &#x7B;&#x42;&#x69;&#x6E;&#x64;&#x69;&#x6E;&#x67;&#x20;&#x4E;&#x61;&#x6D;&#x65;&#x7D
  • &#x7B;Binding Name&#x7D;

So itโ€™s not enough to simply look at the text as weโ€™re scanning an XML text token and check for a { character, one has to first tokenize the entire attribute value as XMLโ€”then retokenize the normalized value (the value after all XML character references and entities have been resolved). And thereโ€™s a further layer to this because markup extension syntax itself has an escape syntax where the value of {}{Binding} (or e.g. {}{Binding}) is not a markup extension but whatever follows the {} (however theyโ€™re encoded). And I hear you right now thinking โ€œWho cares, though? Youโ€™re never going to run into this in the wildโ€ and youโ€™re right but weโ€™re doing this The Right Wayโ„ข๏ธ soโ€ฆ in for a pennyโ€ฆ

While at first this realization may seem like a complication, it actually simplified the design in that it left me firmly resolved that all computation of XAML โ€œvaluesโ€ should happen entirely in semantic analysis. That is to say that once again parsing will be completely pure of any involvement of the rules of XAML or markup extensions or markup extension escapes and will purely handle the interpretation of text as valid XML. This is great for all existing code that handles XML literals as no special handling would be needed to undo pre-emptive XAMLfication of any speculative WellformedMarkupExtensionSyntax, MalformedMarkupExtensionSyntax, EscapedXamlAttributeTextSyntax, or anything else like that.

Designing an API for efficient tooling

As I mentioned above, the XAML specification provides a whole type system describing XamlType and XamlMember instances so itโ€™s very natural to represent these concepts as first class in the compiler internals and to expose them through an API so that the IDE and analyzers can ask questions like โ€œWhat type does this element construct?โ€ and โ€œWhat member does this attribute initialize?โ€; reasonable questions for providing a rich editing experience with completion and quick info, etc.

However, the Roslyn APIs are designed such that semantic questions are normally asked in relation to syntax nodes that very efficiently convey which span of text is being asked about and with XAML literal translation happening entirely in semantic analysis there is now a need for some way to ask these questions about offsets in an attributeโ€™s text (and thatโ€™s a simplification). Long story short, Iโ€™m creating a pseudo-syntax API that captures the markup extension syntax so that tools can still be written efficiently. Itโ€™s very consistent with the true syntax API as I feel thatโ€™s ideal for API consumers. Having said that, I do think that XAML code editing tooling will have to wait for a future minor ModVB release because I need to get this feature out into your hands for testing/feedback ASAP. Maybe Iโ€™ll enlist help in building the code editing tooling from a community member.

Separations from and Interactions with top-level code

A few more design pointsโ€ฆ

InitializeComponent

In my earlier prototypes I planned to have the binding of top-level expressions special-case XML literals to support the common case of a XAML window or root object being initialized by a single literal. However, with this revision Iโ€™ve decided to move this behavior from being a special case of top-level code to being inherent behavior of a XAML literal expression statement inside a Sub (a method which does not return a value) whose name matches some configurable identifier such as InitializeComponent. This means that whether as a top-level expression statement, an expression statement in a top-level method, or an expression statement in a normal method in a normal class XAML-based initialization will work as expected. Aside from being a little cleaner this design is critical to enable testing of the XAML initialization scenario in isolation without top-level code, which is not implemented yet.

Name and x:Name/Code-behind

Another use case which is essential to reproducing the experience weโ€™re all familiar with from WPF is backing field generation and wire-up for named controls. As it stands, the design of XAML literals is that using the x:Name directive (defined by the XAML spec) or setting the equivalent member (also a specified concept) on a XAML object will lookup a field or property on the specified x:Class and initialize it with that object. It does not cause that field or property to be created.

Part of the reason for this is because of a separation of subphases within Semantic Analysisโ€”XAML literals are just normal expressions and are bound in the context of a method. Method body binding happens after the phase of semantic analysis that processes declarations. Of course, it would place undue burden on users if they were required to defined all their backing fields for named controls manually so the tension right now is between having these declarations created as a part of building declarations from top-level code (in contrast to the previous heading regarding InitializeComponent methods), or generating the fields explicitly through a source generator. The generator approach is closer to how WPF works today but the problem is to correctly determine which fields are generated and even their types requires semantic information. For example, if I assign a name to a control inside of a ControlTemplate or DataTemplate, Style, ResourceDictionary, or other deferred content that name technically exists within a separate namescope (A XAML spec concept). Only named objects within the root namescope should have fields generated. Which classes create their own namescopes is semantic knowledge which incurs a heavier performance penalty when using a source generator because the compilation must first be built up to the point where that information is available in order for a generator to ask about it.

Of course, a source generator could be hard coded with this knowledge (unlike the compiler) on a per-XAML vocabulary basis. One could even squint if the generator created too many fields in this case. But the alternative is to allow the compiler itself to do just enough semantic analysis of XAML types while building the declaration tree to create these fields. Both solutions can be elegant if you throw enough code at them and Iโ€™m currently trying to resolve this tension, though it wonโ€™t hold up testing/stabilization of the XAML literal feature because of my original design pointโ€”the XAML literal need only find the fields and initialize them, it takes no dependency on how those fields or properties are defined.

Another scenario that benefits from this separation, sort of, is the use of a XAML literal as an expression within a method. Having the literal take no responsibility for generating any new symbols means that a) I donโ€™t need to design what to generate in such a use case, and b) itโ€™s straight forward to design backing such objects with local variables.

The long tail of XAML scenarios

I hinted at the special handling of deferred content such as ControlTemplate and DataTemplate instances in the previous heading. Thereโ€™s more testing and implementation to be done there beyond just namescopes. For example, .NET MAUI handles these very differently from WPF or Avalonia UI so I have to implement and test both approaches.

Another trailing work item I need to implement is ambient properties, which are a WPF concept rather than a XAML concept but an essential one for styling. Iโ€™m tempted to defer supporting this until later but styling is a pretty central WPF use case. Either way I think this scenario defines the cut line for wrapping the initial implementation of XAML literals. There is bound to be a long tail of minute technology specific scenarios the community will find when trying to port examples of different technologies to use ModVB and Iโ€™ll simply have to address those bugs as they come up.

Once and only once I feel confident in the stability of this subset of XAML literal capabilities will I move on to the implementation of top-level code. I expect a lot more stability bugs to come out of that feature than XAML literals and I want to make sure Iโ€™m not fighting two fires at once by mixing the two. I do think my original design for top-level code is pretty elegant and still the correct way to go and should be fairly quick to implement so fingers crossed Wave 2 should release this month. I already know supplemental features and tooling will come in a future wave.

Wrap-up

Thanks for listening to a rambling nerd to the bitter end. My next post will be a short personal update talking about what itโ€™s been like juggling 4 projects in the month of January and other stuff.

Regards,

-ADG

This blog post was brought to you in part with the support of my wonderful Patreon community (which you can join). Special thanks to my Lunch tier patrons and higher: Dr. Donna M., Alex T., Kirill O., Neal G., Govert D., Dr. Anne J., Taiwo A., Jonathon M., Peter B., Kevin R., and Paul C.