How to perform Metaprogramming in Scala

This post shows how to automatically generate Scala case classes from a schema file using Scalameta. I’ll explain metaprogramming, explore where it’s useful, and provide working code.

Code generation appears in many domains: dbt handles data engineering, Jinja generates website pages, and Lisp DSLs let you build custom tools.

I wanted to learn metaprogramming in Scala after seeing its power in other languages. A practical problem I faced: working with a new Spark dataset without knowing its schema.

Metaprogramming, as Wikipedia defines it:

Metaprogramming is a programming technique in which computer programs have the ability to treat other programs as their data. It means that a program can be designed to read, generate, analyze or transform other programs, and even modify itself while running.

Scala uses Scalameta for metaprogramming. Like most Scala tooling, it takes a hands-on learning approach: thorough API docs but limited higher-level documentation. Scalameta is built on the principle that code translates into an Abstract Syntax Tree (AST), which enables programmatic manipulation. The AST Explorer tool lets you see in real time how Scala code translates into AST.

If you’ve worked with Scala before, you’ll know Scalafmt, which is also built on Scalameta. Scala is well-suited for writing Domain Specific Languages (DSLs) for specific problem classes. Coupling metaprogramming with DSLs expands what can be solved.

Note: This post uses Scala 2, but references Scala 3 documentation since it’s actively maintained. The principles remain the same.

Now for the problem: generate a case class that mirrors a data source’s schema. Case Classes are immutable data structures tied to specific domains. They typically describe data structures at the start of ETL pipelines, making automation valuable both for efficiency and for learning how Scala handles code generation.

The solution has three steps:

1. Accept a csv file defining the data source schema defined in terms of name,type, where type is usually String but needs to be one of the Scala types.
2. Generate a valid Scala case class in a file in the same directory as the data file.
3. Apply formatting to the Scala case class

Here’s another way of describing this problem:

Loading diagram…

A core principle when using Scala is leaning on its expressive type system—creating custom Types as Case Classes. The diagram below illustrates the three data structures used:

Loading diagram…

The ColumnSpecification type pairs a column name with its type. The CaseClass type groups a class name with a list of ColumnSpecification items. Finally, the ScalaFile type combines generated Scala code (as a Source) with the file’s name and path.

Note: This approach is called Domain Modelling

Next, convert the schema types into an AST, then render it as Scala code. Represent each type as a Term. Here’s how to convert a ColumnSpecification into a Term.Param:

Term.Param(
      Nil,
      Term.Name(field.colName),
      Some(Type.Name(field.colType)),
      None
    )

Use a for-comprehension to create a Seq of Term.Params. For-comprehensions are syntactic sugar for sequence computations—see the documentation. This touches on functional programming’s deeper topics like monads; the Haskell tutorial offers a good introduction ¹.

private def createFields(fields: Seq[ColumnSpecification]): Seq[Term.Param] =
    for {
        field <- fields
    } yield Term.Param(
        Nil,
        Term.Name(field.colName),
        Some(Type.Name(field.colType)),
        None
    )

The final method inserts the case-class fields into the package’s AST structure.

  private def generateCode(caseClass: CaseClass): Source = {
    val packageName =
      Term.Select(Term.Name("metaprogramming"), Term.Name("generatedCode"))
    val name = Type.Name(caseClass.name)
    val fields = List(createFields(caseClass.fields).toList)
    val parameters = Ctor.Primary(
      Nil,
      Name(""),
      fields
    )
    val template = Template(Nil, Nil, Self(Name(""), None), Nil, Nil)
    Source(
      List(
        Pkg(
          packageName,
          List(
            Defn.Class(
              List(Mod.Final(), Mod.Case()),
              name,
              Nil,
              parameters,
              template
            )
          )
        )
      )
    )
  }

At this point I’m just going to point you to the entire script to see how these different methods tie together.

This post introduced metaprogramming and ASTs, showed how to generate code in Scala, and demonstrated a practical solution: automating case-class generation from schema files. This technique extends to any code generation task—templates, API clients, or domain-specific types—making it a valuable tool for automating repetitive structure in large projects.

Footnotes

1. For this specific example we can use a for-comprehension because Seq is monadic which means it implemens a pure and a flatMap method - I will have to point you back to google to read more about the topic of monads. ↩