# (Part2) Hate YAML? Build your next tool with HCL!

This is the second part of my HCL series. You find the first part [here (Part 1)](https://blog.weakpixel.com/hate-yaml-build-your-next-tool-with-hcl)

In the second post of my HCL series, I want to extend our example with:

* Cobra Commandline
    
* Variables
    
* Functions
    

## Cobra

[Cobra](https://github.com/spf13/cobra) is my favourite library to build command-line tools.

We start with the example program from the first post ([source](https://gist.github.com/weakpixel/356f44e8a3d0a74c6e542967ade5eb00)).

As I write before I want to introduce you to the Cobra command-line tool. To use it we have to add a new import:

```golang
import (
    "fmt"
    "os"

    "github.com/spf13/cobra"
    // ...
```

Next, rename the `main()` function to `newRunCommand()` and refactor it to return a [cobra.Command](https://pkg.go.dev/github.com/spf13/cobra#Command)

```golang
func newRunCommand() *cobra.Command {
    // contains all variables given by the user with --var "key=value"
    vars := []string{}
    cmd := cobra.Command{
        Use: "run"
        Short: "Executes tasks",
        RunE: func(cmd *cobra.Command, args []string) error {
            config := &Config{}
            err := hclsimple.Decode("example.hcl", []byte(exampleHCL), nil, config)
            if err != nil {
                return err
            }
            for _, task := range config.Tasks {
                fmt.Printf("Task: %s\n", task.Name)
                for _, step := range task.Steps {
                    fmt.Printf("    Step: %s %s\n", step.Type, step.Name)
                    var runner Runner
                    switch step.Type {
                    case "mkdir":
                        runner = &MkdirStep{}
                    case "exec":
                        runner = &ExecStep{}
                    default:
                        return fmt.Errorf("unknown step type %q", step.Type)
                    }

                    diags := gohcl.DecodeBody(step.Remain, nil, runner)
                    if diags.HasErrors() {
                        return diags
                    }
                    err = runner.Run()
                    if err != nil {
                        return err
                    }
                }
            }

            return nil
        },
    }
    // Define an optional "var" flag for the commnd
    cmd.Flags().StringArrayVar(&vars, "var", nil, "Sets variable. Format <name>=<value>")
    return &cmd
}
```

The `Use` field describes the subcommand name. The `Short` field allows defining a short command description. The `RunE` implements the execution of the (sub-)command. It contains our HCL parsing code. Since `RunE` allows us to return an error we also have refactored the code to just return an error instead of using `os.Exit(1)`.

After that, we implement a new `main` function looking like:

```golang
func main() {
    root := cobra.Command{
        Use: "taskexec",
    }
    root.AddCommand(newRunCommand())
    err := root.Execute()
    if err != nil {
        fmt.Println(err)
        os.Exit(1)
    }
}
```

The root command is just an empty `cobra.Command`. To the root command we add our subcommand with `root.AddCommand(newRunCommand())`.

Let's try out what happens if we run our program:

```plaintext
go run main.go 
Usage:
  taskexec [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  help        Help about any command
  run         Executes tasks

Flags:
  -h, --help   help for taskexec
```

Let's try to show the help for the subcommand:

```plaintext
go run main.go run -h
Executes tasks

Usage:
  taskexec run [flags]

Flags:
  -h, --help              help for run
      --var stringArray   Sets variable. Format <name>=<value>
```

Great! Next, we want to make use of the variables. To use variables in our HCL config, we must learn about the `hcl.EvalContext`

## EvalContext

The [hcl.](https://pkg.go.dev/github.com/hashicorp/hcl2/hcl#EvalContext)EvalContext allows us to define variables and functions

```golang
type EvalContext struct {
    Variables map[string]cty.Value
    Functions map[string]function.Function
}
```

For now, we focus on the variables. The `Variables` map allows us to define the variable name as key and as value a `cty.Value`. The `cty.Value` is part of the `github.com/zclconf/go-cty/cty` package. The package provides a dynamic type system.

You can read more about `cty` on the [github project](https://github.com/zclconf/go-cty).

Let's come back to `hcl.EvalContext`. Where is this context struct actually used? In our example code, we have two instances:

```golang
hclsimple.Decode("example.hcl", []byte(exampleHCL), 
    /*&hcl.EvalContext{}*/ nil, config)
```

and

```golang
diags := gohcl.DecodeBody(step.Remain,
     /*&hcl.EvalContext{}*/ nil, runner)
```

## Variables

In our command, we have defined a `vars` slice which contains the user-defined variables in the format:

```plaintext
--var "key=value" ...
```

So let's get started and create `hcl.EvalContext` and populate it with the `vars` parameters from the command line.

```golang
func newEvalContext(vars []string) (*hcl.EvalContext, error) {
    varMap := map[string]cty.Value{}
    for _, v := range vars {
        el := strings.Split(v, "=")
        if len(el) != 2 {
            return nil, fmt.Errorf("invalid format: %s", v)
        }
        varMap[el[0]] = cty.StringVal(el[1])
    }

    ctx := &hcl.EvalContext{}
    ctx.Variables = map[string]cty.Value{
        "var": cty.ObjectVal(varMap),
    }
    return ctx, nil
}
```

We use the `newEvalContext()` function in our subcommand to create the EvalContext and use the context in all places where we decode the HCL document:

```golang
// ...
RunE: func(cmd *cobra.Command, args []string) error {
    ctx, err := newEvalContext(vars)
    if err != nil {
        return err
    }
    config := &Config{}
    err = hclsimple.Decode("example.hcl", []byte(exampleHCL), ctx, config)
    // ...
    for _, task := range config.Tasks {
        fmt.Printf("Task: %s\n", task.Name)
        for _, step := range task.Steps {
            // ...
            diags := gohcl.DecodeBody(step.Remain, ctx, runner)
            // ...
        }
    }

    return nil
},
// ...
```

And finally, we change our `exampleHCL` to make use of variables:

```golang
exampleHCL = `
    task "first_task" {
        step "mkdir" "build_dir" {
            path = var.buildDir
        }
        step "exec" "list_build_dir" {
            command = "ls ${var.buildDir}"
        }
    }
`
```

Let's try to execute the command without defining the `buildDir` variable:

```plaintext
go run main.go run 
...
example.hcl:4,15-24: Unsupported attribute; This object does not have an attribute named "buildDir"., and 1 other diagnostic(s)
exit status 1
```

Good, it fails with a detailed error message.

Now we try to execute the command with the needed variable:

```plaintext
go run main.go run --var buildDir=./build
Task: first_task
    Step: mkdir build_dir
    Step: exec list_build_dir
```

And it works as expected!

You can see the full source code [here](https://gist.github.com/weakpixel/c92f8427b6197a501c1a8d0595e5b5db)

## Functions

Next, we want to explore how e.g. Terraform provides these nice inline functions which makes life so much easier to deal with input variables. It might not make much sense in our example but let's try to implement a function that converts all cased letters into uppercase:

```plaintext
helloValue = "${upper("hello")} World"
```

To implement a function we must add a new module to our import `"github.com/zclconf/go-cty/cty/function"`. We have to use the `function.Spec` struct to create with `function.New` our function implementation:

```golang
var upperFn = function.New(&function.Spec{
    // Define the required parameters.
    Params: []function.Parameter{
        {
            Name:             "str",
            Type:             cty.String,
            AllowDynamicType: true,
        },
    },
    // Define the return type
    Type: function.StaticReturnType(cty.String),
    // Function implementation:
    Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
        in := args[0].AsString()
        out := strings.ToUpper(in)
        return cty.StringVal(out), nil
    },
})
```

And last we add the new function to our EvalContext:

```golang
func newEvalContext(vars []string) (*hcl.EvalContext, error) {
    
    // ...

    ctx.Functions = map[string]function.Function{
        "upper": upperFn,
    }
    return ctx, nil
}
```

Update the `exampleHCL` to make use of our brand new defined function:

```plaintext
exampleHCL = `
    task "first_task" {
        step "mkdir" "build_dir" {
            path = upper(var.buildDir)
        }
        step "exec" "list_build_dir" {
            command = "ls ${ upper(var.buildDir) }"
        }
    }
`
```

Add some debug output to our example Step execution (mkdir, exec) and run the program:

```plaintext
go run main.go run --var "buildDir=./build"

Task: first_task
    Step: mkdir build_dir
        Path:./build
    Step: exec list_build_dir
        Command: ls ./BUILD
```

and as expected we have an upper-case build directory.

If you don't want to implement all the functions yourself or you need some inspiration to implement a function you find want you looking for here:

* https://pkg.go.dev/github.com/zclconf/go-cty/cty/function/stdlib
    
* https://github.com/hashicorp/terraform/blob/main/internal/lang/functions.go
    

## Resources

Resources:

* [Part 1 - Hate YAML? Build your next tool with HCL!](https://blog.weakpixel.com/hate-yaml-build-your-next-tool-with-hcl)
    
* [Full Source Code Gist](https://gist.github.com/weakpixel/40147127fdcac0e11f74967a9ef5aaad)
    

## Full Source Code

```golang
package main

import (
    "fmt"
    "os"
    "strings"

    "github.com/spf13/cobra"
    "github.com/zclconf/go-cty/cty"

    "github.com/hashicorp/hcl/v2"
    "github.com/hashicorp/hcl/v2/gohcl"
    "github.com/hashicorp/hcl/v2/hclsimple"

    "github.com/zclconf/go-cty/cty/function"
)

var (
    exampleHCL = `
        task "first_task" {
            step "mkdir" "build_dir" {
                path = upper(var.buildDir)
            }
            step "exec" "list_build_dir" {
                command = "ls ${ upper(var.buildDir) }"
            }
        }
    `
)

func main() {
    root := cobra.Command{
        Use: "taskexec",
    }
    root.AddCommand(newRunCommand())
    err := root.Execute()
    if err != nil {
        fmt.Println(err)
        os.Exit(1)
    }
}

func newRunCommand() *cobra.Command {
    vars := []string{}
    cmd := cobra.Command{
        Use:   "run",
        Short: "Executes tasks",
        RunE: func(cmd *cobra.Command, args []string) error {
            ctx, err := newEvalContext(vars)
            if err != nil {
                return err
            }
            config := &Config{}
            err = hclsimple.Decode("example.hcl", []byte(exampleHCL), ctx, config)
            if err != nil {
                return err
            }
            for _, task := range config.Tasks {
                fmt.Printf("Task: %s\n", task.Name)
                for _, step := range task.Steps {
                    fmt.Printf("    Step: %s %s\n", step.Type, step.Name)
                    var runner Runner
                    switch step.Type {
                    case "mkdir":
                        runner = &MkdirStep{}
                    case "exec":
                        runner = &ExecStep{}
                    default:
                        return fmt.Errorf("unknown step type %q", step.Type)
                    }

                    diags := gohcl.DecodeBody(step.Remain, ctx, runner)
                    if diags.HasErrors() {
                        return diags
                    }
                    err = runner.Run()
                    if err != nil {
                        return err
                    }
                }
            }

            return nil
        },
    }

    cmd.Flags().StringArrayVar(&vars, "var", nil, "Sets variable. Format <name>=<value>")

    return &cmd
}

func newEvalContext(vars []string) (*hcl.EvalContext, error) {
    varMap := map[string]cty.Value{}
    for _, v := range vars {
        el := strings.Split(v, "=")
        if len(el) != 2 {
            return nil, fmt.Errorf("invalid format: %s", v)
        }
        varMap[el[0]] = cty.StringVal(el[1])
    }

    ctx := &hcl.EvalContext{}
    ctx.Variables = map[string]cty.Value{
        "var": cty.ObjectVal(varMap),
    }
    ctx.Functions = map[string]function.Function{
        "upper": upperFn,
    }
    return ctx, nil
}

var upperFn = function.New(&function.Spec{
    // Define the required parameters.
    Params: []function.Parameter{
        {
            Name:             "str",
            Type:             cty.String,
            AllowDynamicType: true,
        },
    },
    // Define the return type
    Type: function.StaticReturnType(cty.String),
    // Function implementation:
    Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
        in := args[0].AsString()
        out := strings.ToUpper(in)
        return cty.StringVal(out), nil
    },
})

type Config struct {
    Tasks []*Task `hcl:"task,block"`
}
type Task struct {
    Name  string  `hcl:"name,label"`
    Steps []*Step `hcl:"step,block"`
}

type Step struct {
    Type   string   `hcl:"type,label"`
    Name   string   `hcl:"name,label"`
    Remain hcl.Body `hcl:",remain"`
}

type ExecStep struct {
    Command string `hcl:"command"`
}

func (s *ExecStep) Run() error {
    fmt.Println("\tCommand: " + s.Command)
    return nil
}

type MkdirStep struct {
    Path string `hcl:"path"`
}

func (s *MkdirStep) Run() error {
    fmt.Println("\tPath:" + s.Path)
    return nil
}

type Runner interface {
    Run() error
}
```