Creating Subcommands Whose Parents Can Be Invoked?

Introduction

When building command-line interfaces (CLI) using the Tyro library, it's common to encounter scenarios where you need to create subcommands with their parent commands also being invocable. In this article, we'll explore how to achieve this using Tyro and Python.

Understanding the Issue

Let's consider a simple CLI that we want to build using Tyro. The CLI should have two modes: one where it can be invoked with the bare cli.py command, and another where it can be invoked with a subcommand, e.g., cli.py sub. Internally, each of these modes should be parsed into a different data structure.

Defining the Data Structures

We'll define two data structures using Python's dataclasses module: Model1 and Model2. These data structures will represent the internal state of our CLI in each of the two modes.

from dataclasses import dataclass

# python3 cli.py --arg1 1 --arg2 22
@dataclass
class Model1:
    """Data structure for the bare CLI invocation."""
    arg1: int | None = None
    arg2: int | None = None

# python3 cli.py sub --arg3 3 --arg4 4
@dataclass
class Model2:
    """Data structure for the subcommand invocation."""
    arg3: int | None = None
    arg4: int | None = None

Using Union to Represent the CLI Modes

To represent the two modes of our CLI, we'll use the Union type from the typing module. This will allow us to define a single type that can be either Model1 or Model2.

from typing import Union

# Define the CLI type as a Union of Model1 and Model2
cli_type = Union[Model1, Annotated[Model2, tyro.conf.subcommand("sub")]]

The Problem with Subcommands

However, when we try to use the Union type to define our CLI, we encounter a problem. The Model1 type becomes a subcommand, which means that it can no longer be invoked with the bare cli.py command.

usage: cli.py [-h] {model1,sub}

╭─ options ───────────────────────────────────────────╮
│ -h, --help          show this help message and exit │
╰─────────────────────────────────────────────────────╯
╭─ subcommands ─────────────────────────────────────��─╮
│ {model1,sub}                                        │
│     model1                                          │
│     sub                                             │
╰─────────────────────────────────────────────────────╯

Solving the Problem

To solve this problem, we need to find a way to define our CLI type as a Union of Model1 and Model2 without making Model1 a subcommand. One possible solution is to use the Annotated type from the typing module to add a subcommand annotation to Model2 without Model1.

from typing import Annotated

# Define the CLI type as a Union of Model1 and Model2
cli_type = Union[Model1, Annotated[Model2, tyro.conf.subcommand("")]]

However, this solution still doesn't work as expected. When we try to invoke the CLI with the bare cli.py command, we need to pass the empty string "" explicitly on the command line.

The Final Solution

After some experimentation, we find that the final solution involves using the tyro.conf.subcommand function to add a subcommand annotation to Model2 without affecting Model1. We can do this by creating a separate type for the subcommand and using that type in the Union definition.

from typing import Union, TypeVar

# Define a type variable for the subcommand
T = TypeVar("T")

# Define the subcommand type
Subcommand = Annotated[T, tyro.conf.subcommand("sub")]

# Define the CLI type as a Union of Model1 and the subcommand type
cli_type = Union[Model1, Subcommand[Model2]]

With this final solution, we can define our CLI type as a Union of Model1 and the subcommand type without making Model1 a subcommand. This allows us to invoke the CLI with the bare cli.py command and also with the subcommand cli.py sub.

Conclusion

Q: What is the problem with using Union to represent the CLI modes?

A: When you use the Union type to represent the two modes of your CLI, the Model1 type becomes a subcommand. This means that it can no longer be invoked with the bare cli.py command.

Q: How do I define a subcommand in Tyro?

A: To define a subcommand in Tyro, you can use the tyro.conf.subcommand function. This function takes a string argument that represents the name of the subcommand.

Q: Can I use the empty string as the subcommand name?

A: No, you cannot use the empty string as the subcommand name. If you try to do so, you'll need to pass the empty string explicitly on the command line when invoking the CLI.

Q: How do I create a separate type for the subcommand?

A: To create a separate type for the subcommand, you can use the TypeVar type from the typing module. This will allow you to define a type variable that can be used to represent the subcommand.

Q: What is the difference between using Annotated and using TypeVar to define the subcommand type?

A: When you use Annotated to define the subcommand type, you're adding a subcommand annotation to the type. When you use TypeVar, you're creating a separate type variable that can be used to represent the subcommand. The TypeVar approach is more flexible and allows you to define the subcommand type independently of the main CLI type.

Q: Can I use the subcommand type in other parts of my code?

A: Yes, you can use the subcommand type in other parts of your code. Since you've defined the subcommand type as a separate type variable, you can use it anywhere in your code where you need to represent the subcommand.

Q: How do I invoke the CLI with the bare cli.py command?

A: To invoke the CLI with the bare cli.py command, you can simply run the command without any arguments. The CLI will use the Model1 type to represent the internal state.

Q: How do I invoke the CLI with the subcommand cli.py sub?

A: To invoke the CLI with the subcommand cli.py sub, you can run the command with the sub argument. The CLI will use the Model2 type to represent the internal state.

Q: Can I add more subcommands to my CLI?

A: Yes, you can add more subcommands to your CLI by defining additional types and using the tyro.conf.subcommand function to add subcommand annotations to them.

Q: How do I handle errors and exceptions in my CLI?

A: To handle errors and exceptions in your CLI, you can use try-except blocks to catch and handle any exceptions that may occur during the execution of your CLI. You can also use the tyro.conf.error_handler function to define custom error handlers for your CLI.

Q: Can I use Tyro with other Python libraries and frameworks?

A: Yes, you can use Tyro with other Python libraries and frameworks. Tyro is designed to be flexible and can be used with a wide range of libraries and frameworks.