Optional Fields In Type Generation: A Feature Discussion

Hey guys! Let's dive into a super important feature discussion about how we handle optional and undefined fields when generating types, especially when dealing with tools like Scalamando and directus-extension-ts-typegen. This is all about making our lives easier as developers and ensuring our code behaves the way we expect it to. So, buckle up, and let's get started!

The Issue: Forced Null Assignments and Update Confusion

The current type generation approach has a quirk that can be a bit of a headache. Non-required fields are often emitted as required properties with T | null unions. What does this mean in plain English? Well, it means that even if a field is optional, you're forced to explicitly pass null for it. This might not sound like a big deal, but it leads to a couple of usability issues that can make your coding experience less smooth.

First off, you're looking at forced null assignments when creating objects. Instead of being able to simply omit an optional field, you have to remember to set it to null. This can clutter your code and make it less readable. Imagine you have an object with ten optional fields; you'd have to explicitly set each one to null if you don't want to provide a value. That's a lot of extra typing and mental overhead, right? This not only increases the verbosity of your code but also adds an extra layer of cognitive load. Developers must remember to explicitly set optional fields to null, even when the absence of a value should be a valid state. This can lead to code that is harder to read and maintain, as the explicit null assignments obscure the intent behind the object creation.

Secondly, it introduces update semantics confusion. When you're updating an object, how do you differentiate between "don't change this field" and "clear this field"? With the current setup, both scenarios would involve null. This means you lose a level of expressiveness in your code. You can't clearly signal whether you're intentionally clearing a field or simply leaving it untouched. For example, consider a scenario where you have a user profile with an optional bio field. If you want to update the user's name without altering the bio, there's no straightforward way to do this without potentially clearing the bio field unintentionally. This ambiguity can lead to bugs and unexpected behavior, especially in complex applications where update operations are frequent and nuanced. The inability to distinguish between these two scenarios makes the code more prone to errors and harder to debug.

To further illustrate the problem, let's consider a real-world example. Imagine you are building a form where users can update their profile information. Some fields, like the user's job title or phone number, might be optional. With the current type generation behavior, if a user submits the form without filling in these optional fields, your code would need to explicitly set these fields to null in the database. This is not only cumbersome but also potentially misleading, as it suggests that the user intentionally cleared these fields, even if they simply left them blank. This can have implications for data analysis and reporting, as you might inadvertently count empty fields as intentional deletions. The current behavior forces developers to treat optional fields as mandatory with a null value, which goes against the very idea of optionality.

Current Behavior: A Concrete Example

Let's look at a quick example to make this crystal clear. Imagine we have a User interface:

interface User {
  name: string;
  subtitle: string | null; // Required property, must pass null explicitly
}

// Must do this:
const user: User = { name: "John", subtitle: null };

// Cannot do this:
const user: User = { name: "John" }; // TypeScript error

See the problem? Even though subtitle is technically optional, TypeScript will throw an error if you try to create a User object without explicitly setting it to null. This isn't ideal, and it's what we're trying to fix.

This example highlights the core issue: the type system is not accurately reflecting the optional nature of the subtitle field. By forcing developers to provide a null value, the type system is essentially treating the field as mandatory, even though it is intended to be optional. This discrepancy between the intended behavior and the actual type definition can lead to confusion and errors, especially for developers who are new to the codebase or the type system itself. The example clearly demonstrates how the current behavior deviates from the expected behavior of optional fields, making it necessary to explore alternative solutions.

Proposed Solution: Configuration Options for Flexibility

So, how do we tackle this? The idea is to add a configuration option that gives us more control over how non-required fields are represented in the generated types. This way, we can choose the behavior that best fits our needs. We're proposing three options:

• off (default): This would maintain the current behavior, where optional fields are represented as subtitle: string | null. This ensures backward compatibility and allows teams to stick with the existing approach if they prefer.
• questionMark: This option would represent optional fields with a question mark, like this: subtitle?: string | null. This is a common pattern in TypeScript for optional properties, and it aligns well with how developers typically expect optional fields to behave.
• undefinedUnion: This option would include undefined in the union type, resulting in subtitle: string | null | undefined. This approach provides explicit support for undefined as a valid value for optional fields, which can be useful in certain scenarios.

Diving Deeper into the Proposed Options

Let's break down each option a bit further to understand their implications and use cases. Each of these options offers a distinct way to handle optional fields, catering to different coding styles and application requirements. By providing this flexibility, we can ensure that the type generation process aligns with the specific needs of each project.

The off option, as mentioned, preserves the existing behavior. This is crucial for maintaining compatibility and avoiding breaking changes in existing codebases. For teams that have already adopted the current approach and have workflows built around it, this option allows them to continue working without disruption. It also serves as a baseline for comparison, allowing developers to evaluate the benefits of the other options in the context of their specific projects. The off option ensures that the transition to the new configuration system is smooth and doesn't force teams to adopt new patterns prematurely.

The questionMark option is arguably the most intuitive and widely used pattern for representing optional properties in TypeScript. By adding a question mark to the property name, we clearly signal that the field is optional and can be omitted when creating or updating objects. This approach aligns with the standard TypeScript conventions and makes the generated types more readable and easier to understand. When developers see a question mark, they immediately know that the field is not mandatory, which reduces the cognitive load and the likelihood of errors. This option also simplifies the process of creating objects with optional fields, as developers can simply omit the field without having to explicitly set it to null. The questionMark option provides a clean and idiomatic way to handle optional fields, making it a strong contender for the default behavior in future iterations.

The undefinedUnion option offers a more explicit way to represent optional fields by including undefined in the union type. This approach can be particularly useful in scenarios where undefined has a specific meaning or needs to be explicitly handled in the code. For example, in some applications, undefined might indicate that a field has never been set, while null might indicate that the field has been explicitly cleared. By including undefined in the type, we can ensure that these distinctions are preserved and that the code correctly handles all possible states of the optional field. This option provides a higher level of precision and control over the representation of optional fields, which can be valuable in complex applications where data integrity and state management are critical. However, it's worth noting that this option might also increase the verbosity of the code, as developers might need to explicitly handle undefined in certain situations. The undefinedUnion option caters to developers who require fine-grained control over the representation of optional fields and are willing to handle the additional complexity that it might introduce.

Examples: Seeing the Options in Action

Let's see how these options would look in practice.

Current:

subtitle: string | null

With questionMark:

subtitle?: string | null

With undefinedUnion:

subtitle: string | null | undefined

You can see how each option changes the way the subtitle field is represented. The questionMark option makes it clearly optional, while the undefinedUnion option explicitly includes undefined as a possible value.

These examples clearly demonstrate the impact of each configuration option on the generated types. The questionMark option provides the most concise and readable representation of optional fields, while the undefinedUnion option offers a more explicit and potentially more precise representation. By allowing developers to choose the option that best suits their needs, we can ensure that the type generation process is flexible and adaptable to different coding styles and application requirements. The examples serve as a practical guide for developers to understand the implications of each option and make informed decisions about how to configure the type generation process.

Conclusion: Flexibility for Better Development

By adding this configuration option, we're giving developers more flexibility and control over how their types are generated. This, in turn, leads to cleaner, more readable code and a smoother development experience. Guys, this is a big step forward in making our tools work for us, not against us! This enhancement not only simplifies the development process but also reduces the potential for errors and improves the overall maintainability of the codebase. The ability to choose the representation that best fits the specific needs of the project is a significant advantage, as it allows developers to tailor the type generation process to their preferred coding style and application requirements. This flexibility ultimately translates into increased productivity and higher-quality code.