ZqUserOperatorsNeedParens

July 19, 2024
from LearningZq

examples use zq version 1.16.0.

In zq you can define your own dataflow operators and functions. All functions need to be called with parens, even argumentless ones, whether built-in or user-defined.

Operators, however, are quirky in that built-in ones cannot be called with parens, but user-defined ones must be.

First off, when do I need to create a function vs. an operator?

The syntax for a user-defined function is:
func <id> ( [<param> [, <param> ...]] ) : ( <expr> )

“where <id> and <param> are identifiers and <expr> is an expression that may refer to parameters but not to runtime state such as this.”

The syntax for a user-defined operator is:
op <id> ( [<param> [, <param> ...]] ) : (
<sequence>
)

“where <id> is the operator identifier, <param> are the [optional] parameters for the operator, and <sequence> is the chain of operators (e.g., operator | ...) where the operator does its work.” And it can refer to this.

To over-simplify:
User-defined FunctionCan only call other functions. Cannot refer to this.
User-defined OperatorCan call operators and functions. Can refer to this.

Though it's more precise to say the body of a user-defined function is an Expression, and the body of a user-defined operator is a Sequence of Dataflow Operators.

Let's make a user-defined operator.
❯ zq -z \
'op add_ids(): (
over this
| yield {id:count(), ...this}
)

yield [{name:"lorem"},{name:"ipsum"}] | add_ids()'

{id:1(uint64),name:"lorem"}
{id:2(uint64),name:"ipsum"}

The user-defined operator add_ids will take an array of records and add an auto-incrementing id integer to each one.

There's some other cool zq things going on here, but for now we'll just focus on calling the operator, with parens. A call without, and again this will fall to the search ImpliedOperator:
❯ zq -z \
'op add_ids(): (
over this
| yield {id:count(), ...this}
)

yield [{name:"lorem"},{name:"ipsum"}] | add_ids'



add_ids without parens is interpreted by zq as search add_ids and the string “add_ids” can't be found in the array of two records.



Prev: ZqFunctionsNeedParens | Up: LearningZq | Next: ZqLateralSubquery