POKI_PUT_TOC_HERE
No output at all
Check the line-terminators of the data, e.g. with the command-line
file program. Example: for CSV, Miller’s default line terminator
is CR/LF (carriage return followed by linefeed, following
RFC4180). Yet if your CSV has
*nix-standard LF line endings, Miller will keep reading the file looking for a
CR/LF which never appears. Solution in this case: tell Miller the input has LF line-terminator, e.g. mlr --csv --rs
lf {remaining arguments ...}.
Fields not selected
Check the field-separators of the data, e.g. with the command-line
head program. Example: for CSV, Miller’s default record
separator is comma; if your data is tab-delimited, e.g. aTABbTABc,
then Miller won’t find three fields named a, b, and
c but rather just one named aTABbTABc. Solution in this
case: mlr --fs tab {remaining arguments ...}.
Error-output in certain string cases
mlr put '$y = string($x); $z=$y.$y' gives (error) on
numeric data such as x=123 while mlr put
'$z=string($x).string($x)' does not. This is because in the former case
y is computed and stored as a string, then re-parsed as an integer,
for which string-concatenation is an invalid operator.
How do I parse log-file output?
Suppose you have log-file lines such as
POKI_CARDIFY(2015-10-08 08:29:09,445 INFO com.company.path.to.ClassName @ [sometext] various/sorts/of data {& punctuation} hits=1 status=0 time=2.378)HERE
I prefer to pre-filter with grep and/or sed to extract the structured text, then hand that to Miller. Example:
POKI_CARDIFY(grep 'various sorts' *.log | sed 's/.*} //' | mlr --fs space --repifs --oxtab stats1 -a min,p10,p50,p90,max -f time -g status)HERE
How do I examine then-chaining?
Then-chaining found in Miller is intended to function the same as Unix
pipes. You can print your data one pipeline step at a time, to see what
intermediate
output at one step becomes the input to the next step.
First, review the input data:
POKI_RUN_COMMAND{{cat data/then-example.csv}}HERE
Next, run the first step of your command, omitting anything from the first then onward:
POKI_RUN_COMMAND{{mlr --icsv --rs lf --opprint count-distinct -f Status,Payment_Type data/then-example.csv}}HERE
After that, run it with the next then step included:
POKI_RUN_COMMAND{{mlr --icsv --rs lf --opprint count-distinct -f Status,Payment_Type then sort -nr count data/then-example.csv}}HERE
Now if you include another then step after this, the columns Status,
Payment_Type, and count will be its input.
Note, by the way, that you’ll get the same results using pipes:
POKI_RUN_COMMAND{{mlr --csv --rs lf count-distinct -f Status,Payment_Type data/then-example.csv | mlr --icsv --rs lf --opprint sort -nr count}}HERE
How do I do arithmetic on fields with currency symbols?
POKI_INCLUDE_ESCAPED(data/dollar-sign.txt)HERE