Column-major typst table definitions - a rabbit hole
Foreword: This post pertains to Typst v0.11.1, and the post most likely won’t be updated to account for any future Typst changes. So to future netizens: your mileage may vary.
Anything worthy of being displayed in a table is, more likely than not, at least a 2D array and frequently an array of dictionaries. It is probably stored as a csv file, but for the sake of brevity, let us assume that it is encoded in the source field already.
#table(
columns: (auto,) + (1fr,)*4,
align: center + horizon,
// Header with hlines, ...
// ... followed by body
($ H_v^"pc"times $), ($ 1 $),
($ (rho_"solvent")/(M_"solvent") $),
($ (rho_"solvent")/(M_"solute") $),
($ 1 / ( R T ) $),
($ H_v^"px"times $), ($ (M_"solvent")/(rho_"solvent") $),
($ 1 $),
($ (M_"solvent")/(M_"solute") $),
($ (M_"solvent")/(R T rho_"solvent") $),
($ H_v^"pw"times $), ($ (M_"solute")/(rho_"solvent") $),
($ (M_"solute")/(M_"solvent") $),
($ 1 $),
( $ (M_"solute")/(R T rho_"solvent") $),
($ H_v^"cc"times $), ($ R T$),
($ (R T rho_"solvent")/(M_"solvent") $),
($ (R T rho_"solvent")/(M_"solute") $),
($ 1 $),
)
Suppose now you want to remove a column. You have to go through each and every positional argument and check that everything lines up where its meant to. You could check that the output is correct in the preview window, but for complicated maths like shown above, the task is no easier. All of this is without complicating factors like rowspans or colspans (shudder) and their associated hlines or vlines. All these problems arrise because we have to chose the order in which we enter information - by row or by column. In Typst, entry is done in row-major (that is, we must fill a row before starting the next). This is useful for cases where you want to remove a row, but removing a row is perhaps done more correctly by removing it from the dataset in the first place.
Column-major definitions
Refusing to take the L on the problems listed above, I set out to create a function that would translate column-major into row-major. I also wanted to make an easy way of relating the column definitions to how the data is actually laid out. Because I don’t want to make life any easier, I wanted nested columns (children) to be grouped with their parent. Lastly, I wanted per-column variables (fill, align, width, gutter) to be grouped together too, plus inheritence. The design I came up with ended up looking like this:
#let example = (
(
key: "date",
display: [Date],
// fill: bg-fill-1,
// align: left,
width: 5em,
gutter: 0.5em,
),
(
key: "particulars",
display: text(tracking: 5pt)[Particulars],
width: 1fr,
gutter: 0.5em,
),
(
key: "ledger",
display: [Ledger],
// fill: bg-fill-2,
width: 2cm,
// align: center,
gutter: 0.5em,
),
(
key: "amount",
display: align(center)[Amount],
// fill: bg-fill-1,
gutter: 0.5em,
hline: arguments(stroke: booktabs.lightrule),
children: (
(
key: "unit",
display: align(left)[£],
width: 5em,
align: right,
vline: arguments(stroke: booktabs.lightrule),
gutter: 0em,
),
(
key: "decimal",
display: align(right, text(number-type: "old-style")[.00]),
width: 2em,
// align: left
),
)
),
(
key: "total",
display: align(center)[Total],
gutter: 0.5em,
hline: arguments(stroke: booktabs.lightrule),
children: (
(
key: "unit",
display: align(left)[£],
width: 5em,
align: right,
vline: arguments(stroke: booktabs.lightrule),
gutter: 0em,
),
(
key: "decimal",
display: align(right, text(number-type: "old-style")[.00]),
width: 2em,
align: left
),
)
),
)
and I want to give it the information in the following form:
#let data = (
(
date: [00/11/1234],
particulars: lorem(05),
ledger: [JRS123] + booktabs.footnotes.make[Hello World],
amount: (unit: $100$, decimal: $00$),
total: (unit: $99$, decimal: $00$),
),
)*7 +(
(
date: [01/09/1994],
particulars: [Just buying something extra this week before I run out of stuff],
ledger: [JRS123] + booktabs.footnotes.make[Special reference because it has a label],
amount: (unit: $12,222$, decimal: $99$),
total: (unit: $99$, decimal: $00$),
),
)
Recursion
The first step towards implementing this is to deal with nested columns. There isn’t a limit on how deep these nests can be (outside of hardware limitations), so this task (and most that follow) will be accomplished using recursion. Recursive functions are functions that end up calling themselves in some circumstances. In our case, we are going to go down each rabit hole, and record the depth of each column aswell as a recursive count of how many child elements it has. Finally, we will also want to know what the maximum depth is a bit later on, so we may aswell calculate that now.
#let sanitize-input(columns, depth: 0, max-depth: 1, length: 0) = {
// For every column
for (key, entry) in columns.enumerate() {
// if it has children
if "children" in entry {
// Recurse
let (children, child-depth, child-length) = sanitize-input(
entry.children,
depth: depth + 1,
max-depth: max-depth + 1
)
// record the recursive length
columns.at(key).insert("length", child-length)
columns.at(key).children = children
length += child-length
// Keep track of the deepest yet seen rabit hole
max-depth = calc.max(max-depth, child-depth)
// Bottom of the rabit hole, must have a length of 1
} else {
length += 1
columns.at(key).insert("length", 1)
}
// In all cases, keep track of depth
columns.at(key).insert("depth", depth)
}
// Pass the results on
return (columns, max-depth, length)
}
All this function does is go through, column by column, row by row, and calculates depths, recursive lengths, and keeps a track of the maximum depth it has come across. We can use this information to build the header:
#let build-header(columns, max-depth: 1, start: 0) = {
// For every column
for entry in columns {
// Make a cell that spans its recusive length (and limit rowspan if it has children)
(table.cell(
x: start,
y: entry.depth,
rowspan: if entry.length == 1 {max-depth - entry.depth} else {1},
colspan: entry.length,
// Header cells should be horizon aligned. Ideally it should default to `start`
// but I've shadowed that variable.
align: horizon + entry.at("align", default: left),
entry.display
),)
// If it has nested columns, build those too.
// NOTE: Return values are collated using automatic array joining!
if "children" in entry {
build-header(
entry.children,
max-depth: max-depth,
start: start // Pass along
)
}
// If it has a hline, add it under the cell
if ("hline" in entry){
(
table.hline(
y: entry.depth + 1,
start: start,
end: start + entry.length,
..entry.hline
),
)
}
if ("vline" in entry){ (table.vline(x: start + 1, ..entry.vline),) }
// Keep track of which column we are in. This could be precalculated.
start += entry.length
}
}
At this point, we are just about ready to start rendering this table, but if you remember those inherited values I mentioned above (fill, align, widith, gutter), we still need to calculate those and have them ready in a data structure that typst can use. Let’s implement a general solution which takes the parameter name, a boolean for whether the value should be inherited by its children, and what it should default to if its missing.
#let recurse-columns(columns, key, default: auto, inherit: true) = {
// For every column
for child in columns {
// If it has children, recurse. If we should inherit, update the default.
if "children" in child {
recurse-columns(
child.children,
key,
inherit: inherit,
default: {
if inherit {child.at(key, default: default)} else {default}
}
)
} else {
// Bottom of rabbit hole: fetch the value (or default) and array join
// everything together
(child.at(key, default: default),)
}
}
}
Time to put all of this together into a neat function:
#let make(
columns: (),
toprule: toprule,
midrule: midrule,
bottomrule: bottomrule,
..args
) = {
// Calculate those useful values I mentioned earlier
let (columns, max-depth, length) = sanitize-input(columns)
table(
stroke: none,
fill: recurse-columns(columns, "fill", default: none),
align: recurse-columns(columns, "align", default: start),
column-gutter: recurse-columns(columns, "gutter", default: 0em),
columns: recurse-columns(columns, "width"),
table.header(
table.hline(stroke: toprule),
..build-header(columns, max-depth: max-depth),
table.hline(stroke: midrule, y: max-depth),
),
..args,
table.hline(stroke: bottomrule)
)
}
Data
Out of this we get a beautiful table, but we still need to enter in the data manually, so lets automate that by creating another recursive function to traverse the columns and the data, and to figure out what needs to be output.
#let recurse-data(columns, data) = {
// For every column
for column in columns {
// Handle nested columns
if ("children" in column){
recurse-data(
column.children,
// If the parent column has a key, lets assume that its not a mistake, and
// lets use this to slice the data before we pass it onto the child columns.
// If the key is missing, from the data, lets assume it has been removed from
// the data set
if ("key" in column){
data.at(column.key, default: (:))
} else {
data
}
)
} else { // Bottom of the rabbit hole
// Return the data, but if it doesn't exist, return empty content instead so
// we don't mess up our alignments.
// Collate returned values using joined arrays.
if "key" in column {
(data.at(column.key, default: []),)
} else {
([],)
}
}
}
}
and lets update make so we can put this new function to work. While we are at it, lets allow the user to specify a hline that gets put between rows inside the table body, as this is the best chance we are going to get to make this a smooth thing to do.
#let make(
columns: (),
data: (), // ADDED
hline: none, // ADDED
toprule: toprule,
midrule: midrule,
bottomrule: bottomrule,
..args
) = {
// Calculate those useful values I mentioned earlier
let (columns, max-depth, length) = sanitize-input(columns, depth: 0)
// footnotes.clear() +
table(
stroke: none,
fill: recurse-columns(columns, "fill", default: none),
align: recurse-columns(columns, "align", default: start),
column-gutter: recurse-columns(columns, "gutter", default: 0em),
columns: recurse-columns(columns, "width"),
table.header(
table.hline(stroke: toprule),
..build-header(columns, max-depth: max-depth),
table.hline(stroke: midrule, y: max-depth),
),
..args,
..( // --- ADDED -------------------------------------->
for entry in data{
recurse-data(columns, entry)
if hline != none {(hline,)}
}
), // <-------------------------------------- ADDED ---
table.hline(stroke: bottomrule)
)
}
Tablenotes
Finally, sometimes we are going to want to attach notes to some of the data we show in our tables. You might consider using #footnote however this adds it to the bottom of the page instead of the bottom of the table, which puts distance between the asterix and your words. After all, notes are meant to make the table easier to read, not more difficult, so lets fix that.
Ultimately, tables notes are a combination of two things: a mark at the location of the annotated content, and then at the end of the table, an ordered list with both the mark and the annotation itself. These can be stored in a stateful array, so lets define one (aswell as some helper functions). We want to keep track of an array of content, and the numbering function used to display the mark. We will also want a quick function to clear the list of annotations so we don’t end up capturing another tables annotations.
// I want my argument names to be intuitive, but these sometimes shadow
// important functions, so lets copy them first.
#let std-state = state
#let std-numbering = numbering
// This will store an array of content
#let state = std-state("__booktabs:footnote", ())
#let clear() = state.update(())
// This will store the numbering function
#let numbering = std-state("__booktabs:numbering", "a")
#let set-numbering(new) = numbering.update(new)
When we make the footnote, all we need is the corresponding mark. Let’s assume that an annotation is numbered based on its order (pretty sensible), and the numbering function won’t change between creating the mark and rendering the footnote at the end of the table (somewhat sensible, but lets just say that anything else is outside the scope of this work).
// as we are rendering the mark before actually storing it, we offset the length of the array by 1.
// I had tried doing it the other way arround, but ran into convergence problems and didn't pursue it much further.
#let make-numbering() = context std-numbering(numbering.get(), state.get().len() + 1)
#let make( body) = {
// Weak horizontal space collapses preceding space, and is followed by a narrow space
// This is so the citation always "hugs" whatever comes before it. This might not be
// appropriate for RTL texts.
h(0em, weak: true) + sym.space.narrow + super(make-numbering())
state.update(it=>{it.push(body);it})
}
To render them, we get the state value inside a context block, map the key and the value of each entry into content, and then join that array of content (with an oxford comma).
#let display-numbering(key) = super(context std-numbering(numbering.get(), key + 1))
#let display-list = context {
state.get()
.enumerate()
.map( ((key, value)) => box[#display-numbering(key) #value]) // Box prevents breaking (perhaps not the best idea)
.join(", ", last: ", and ")
}
Lastly, and only because I have several types of tables but I want the tablenotes to be drawn consistently, I’ve made a helper function to style the footnotes by using some set rules and collapsing some space above it
#let display-style(notes) = {
v(-0.5em)
set text(0.888em)
set block(spacing: 0.5em)
set par(leading: 0.5em)
align(start, notes)
}
Lets do the same as what we did when adding in data, and add these functions to our table make definition.
#let make(
columns: (),
data: (),
hline: none,
toprule: toprule,
midrule: midrule,
bottomrule: bottomrule,
notes: footnotes.display-list, // ADDED
..args
) = {
let (columns, max-depth, length) = sanitize-input(columns, depth: 0)
footnotes.clear() + table( // ADDED
stroke: none,
fill: recurse-columns(columns, "fill", default: none),
align: recurse-columns(columns, "align", default: start),
column-gutter: recurse-columns(columns, "gutter", default: 0em),
columns: recurse-columns(columns, "width"),
table.header(
table.hline(stroke: toprule),
..build-header(columns, max-depth: max-depth),
table.hline(stroke: midrule, y: max-depth),
),
..args,
..(
for entry in data{
recurse-data(columns, entry)
if hline != none {(hline,)}
}
),
table.hline(stroke: bottomrule)
) + if (notes != none) {footnotes.display-style(notes)} // ADDED
}
Further work
For each of “fill”, “gap”, “width”, and “align”, we are recursing the columns array anew every time. It would be more sensible to collect this information right at the start in the sanitize-input function, however I haven’t spent any time on a solution. A similar fact is true with respect to the displaying of data: the columns array is recursively read for each row of data, which is pretty wasteful.
Currently, the column gutters need to be specified manually as I couldn’t find a satisfying heuristic for determining the column gutter based on the difference in depths of 2 adjacent columns.
One of the driving forces (outside of a looming deadline for my thesis submission) is that this is a first step towards aligning a column’s content by decimal point (like the S column type in LaTeX).
Some missing features that would make for excellent quality tables: Sparklines (if we are taking data as a parameter, we might aswell let it be put to good use). Because we now know how long the header is, and how many rows of data we are showing, we have enough information to construct a separate show rule for the header and the body (which previously wasn’t possible).
Lastly, while this was pretty fun to make and would probably make for a useful package, I haven’t got any interest in maintaining a package for it. Hopefully someone will pick up the task, and if they do, I’ll add a link to it’s Typst Universe page at the top.