\chapter[\mineXp Scripting] {\mineXp\\JavaScript-based Scripting}
\label{chap:minexpert-javascript-based-scripting}

When mining data, often, the user finds herself repeating tasks time and time
again. Scripting allows to craft text files in which commands are run by the
software program one after the other.

\mineXp makes a scripting environment available to the user. That environment is
materialized by a window, shown on
Figure~\vref{fig:minexpert-scripting-wnd-scripting-tab}.

\begin{figure}[H]

	\centering

	\includegraphics[width=0.75\textwidth]{figures/minexpert-scripting-wnd-scripting-tab.png}

	\renewcommand{\captionText}{Scripting console, scripting tab}
	\caption[\captionText]{\textbf{\captionText}}

	\label{fig:minexpert-scripting-wnd-scripting-tab}

\end{figure}

The left panel is a tree view listing all the objects or function names that are
made available to the scripter. The right panel has three tabs. The
\guilabel{Scripting} tab has two sections; the upper section is where the
scripting engine output is recorded; the lower section is the actual console
where the user enters the script text.


\renewcommand{\sectitle}{Using the scripting console}
\section*{\textcolor{sectioningcolor}{\sectitle}}
\addcontentsline{toc}{section}{\numberline{}\sectitle}

The scripting console is a multiline text edit widget. The user may enter
multiple lines in that widget, using the carriage \kbdKey{Return} key.
Once all the lines have been entered to make one or more complete JavaScript
statements, the whole set of lines is executed by pressing the
\kbdKey{Ctrl}+\kbdKey{Return} key combination.

Upon execution of the script statements, any output is recorded in the upper
section. When the output obtained from the engine evaluates to true, it is
recorded in green color, if it evaluates to undefined, its color is black, and if
it evaluates to error, it is colored in red.

Looking at the script engine output section, it is possible to read indented
text, colored in grey (Figure~\ref{fig:minexpert-scripting-comments}).

\begin{figure}[H]

	\centering

	\includegraphics[width=0.9\textwidth]
	{figures/minexpert-scripting-comments.png}

	\renewcommand{\captionText}{Informative comments to the user}
	\caption[\captionText]{\textbf{\captionText}}

	\label{fig:minexpert-scripting-comments}

\end{figure}

These are comments that \mineXp provides when it runs
script statements itself as part of making available to the user globally
accessible objects. For example \guivalue{Make available the console window
under name ``consoleWnd''} is an explanatory text for the script statement
(that the user does neither enter or see anywhere in the interface because it is
run in the guts of \mineXp) that creates a globally accessible object for the
scripter to use if necessary. The \guilabel{consoleWnd} object that is created
is listed in the tree view in the left panel of the window.

\renewcommand{\sectitle}{The available objects list}
\section*{\textcolor{sectioningcolor}{\sectitle}}
\addcontentsline{toc}{section}{\numberline{}\sectitle}

The left panel of the scripting window lists objects that are made available by
\mineXp to the scripter. The main objects that \mineXp makes available are shown
in Figure~\vref{fig:minexpert-scripting-wnd-scripting-tab}

\begin{itemize}

	\item \guilabel{mainWindow}: the window of the program, where the
		\guimenu{File} menu resides;

	\item \guilabel{consoleWnd}: the console window where the various
		feedback messages to the user are displayed;

	\item \guilabel{ticChromWnd}: the window where all the total ion current
		chromatograms are shown;

	\item \guilabel{massSpecWnd}: the window displaying all the mass spectra;

	\item \guilabel{driftSpecWnd}: the window displaying all the drift spectra;

	\item \guilabel{colorMapWnd}: the window showing all the mz=f(dt) color maps;

	\item \guilabel{xicExtractionWnd}: the window displaying the XIC integration
		parameters;

	\item \guilabel{mzIntegrationParamsWnd}: the window displaying the m/z
		integration parameters;

	\item \guilabel{sqlMassDataSlicerWnd}: the window displaying the mass data
		file slicer configuration;

	\item \guilabel{openSpectraDlg}: the window listing all the currently opened
		mass spectrometry data files;

	\item \guilabel{scriptingWnd}: the where all the scripting occurs.

\end{itemize}

All the items listed above  are systematically made available upon launching of
the application. For some, they are of no use until a new mass spectrometry file
is loaded. Upon opening of a mass spectrometric data file, new objects are
created that are made available and listed in this same treeview. For example,
upon loading of a mass spectrometry data file, a total ion current chromatogram
is computed by \mineXp\ and displayed in a TicChromPlotWidget object that is
made available under the \guilabel{ticChromWnd} object in the tree view. If the
data file were for ion mobility mass spectrometry, a mz = f(dt) color map would
be computed also and made available as a new plot widget object under the
\guilabel{colorMapWnd} object in the tree view.

When a given object is used for scripting, a shortcut to writing its name in the
scripting editor is to simply double-click its item in the treeview and the name
will be copied into the scripting editor.


\renewcommand{\sectitle}{The scripting history list}
\section*{\textcolor{sectioningcolor}{\sectitle}}
\addcontentsline{toc}{section}{\numberline{}\sectitle}

Each time a JavaScript statement (single- or multi-line) is executed, that
statement gets stored in the \guilabel{History} tab that contains a list, as
shown in Figure~\vref{fig:minexpert-scripting-wnd-history-tab}.

\begin{figure}[H]

	\centering

	\includegraphics[width=0.9\textwidth]
	{figures/minexpert-scripting-wnd-history-tab.png}

	\renewcommand{\captionText}{Scripting console, history tab}
	\caption[\captionText]{\textbf{\captionText}}

	\label{fig:minexpert-scripting-wnd-history-tab}

\end{figure}

The items in the history list are reusable by double-clicking on them (or
selecting them and hitting \kbdKey{Return}). By doing so, the selected items are
copied to the \guilabel{Scripting} tab, in the script input widget. It is then
only required to hit \kbdKey{Ctrl}+\kbdKey{Return} to execute the script
line(s).

Note that the history list might become very long, and the user might want to
filter the entries of the list. By typing a regular expression in the edit
widget below the list and pressing \kbdKey{Return}, the user can select history
items matching that expression.

\clearpage

\renewcommand{\sectitle}{Exploring the available objects' features}
\section*{\textcolor{sectioningcolor}{\sectitle}}
\addcontentsline{toc}{section}{\numberline{}\sectitle}

There are three kinds of objects that are available to the scripting environment:

\begin{itemize}

	\item High-level graphical user interface objects that are semi-automatically
		made available to the JavaScript environment by the \libname{Qt} libraries.
		These objects are not fully scriptable and comprise the highest-level
		windows that are automatically made available upon running the program as
		top level branches of the tree view in the left panel of the scripting
		window;

	\item Middle-level graphical user interface objects that are created ``on
		demand'', like the various plot widgets that are created to display data.
		These objects have a useful range of features, typically listed in the upper
		pane of the scripting window by double-clicking their item in the tree view
		(see below);

	\item Low-level C++ classes that are mapped to first-class JavaScript objects
		(with support for the \verb|new| constructor operator, for example) such
		that they expose their full functionality to the scripting environment.  The
		methods of these classes are described in detail in the JavaScript reference
		section number \ref{sec:javascript-reference}.

\end{itemize}

The functional capabilities of the various objects listed in the tree view can
be explored by double-clicking the item of interest while maintaining the
\kbdKey{Ctrl} key pressed. The list of functional capabilities is printed in the
script output widget, as shown in
Figure~\vref{fig:minexpert-scripting-wnd-object-functional-capabilities}.


\begin{figure}[H]

	\centering

	\includegraphics[width=0.75\textwidth]{figures/minexpert-scripting-wnd-object-functional-capabilities.png}

	\renewcommand{\captionText}{Scripting console, checking object methods}
	\caption[\captionText]{\textbf{\captionText}}

	\label{fig:minexpert-scripting-wnd-object-functional-capabilities}

\end{figure}

As shown in the figure, the mainWindow has a number of member functions that can
be called according to the conventional <object>.<function> notation. For
example, entering \command{mainWindow.quit()} in the script input text widget
will trigger the \command{quit()} function of the mainWindow object,
effectively quitting the application (the mainWindow is actually the main
\mineXp window).

Calling \command{mainWindow.createMassSpectrumFromClipboard()}, for example,
will create a new mass spectrum on the basis of data presently in the clipboard,
if the data are in the conventional x<sep>y format, with x the \mz{}, y the
intensity and <sep> any character that is neither a cipher or a dot (decimal
separator).

\alertbox{It is essential to understand that not all functions are of use in the
	scripting context. This is because the listing of the functional capabilities
	of the objects also comprise functions that are not aimed at the scripting
	environment but are for internal \libname{Qt} or \mineXp use.}

This is more specifically the case of the plot objects, like ticChromPlotWidget0,
massSpecPlotWidget0 or driftSpecPlotWidget0, all listed in the tree view in
Figure~\vref{fig:minexpert-scripting-wnd-object-functional-capabilities}. The
creation of these plot objects is discussed in the section below.

Another way to iterate in the JavaScript properties of the objects made
available in the JavaScript environment is by using this JavaScript code (using
\guilabel{massSpecPlotWidget0} as an example object):\\

\begin{verbatim}

for (var prop in massSpecPlotWidget0)
  {
    print("prop: " + prop)
  }

\end{verbatim}

This is particularly useful to explore the various functions that are available
for any given C++ object that is ``exported'' to the JavaScript environment (the
GlobalObject, really).


\renewcommand{\sectitle}{The JavaScript reference}
\section*{\textcolor{sectioningcolor}{\sectitle}}
\addcontentsline{toc}{section}{\numberline{}\sectitle}

The JavaScript reference material related to all the classes available in the
JavaScript environment is located in the form of formatted plain text in the
\guilabel{JS reference} tab of the scripting window
(Figure~\vref{fig:minexpert-scripting-window-reference-tab}).

\begin{figure}[H]

	\centering

	\includegraphics[width=0.75\textwidth]
	{figures/minexpert-scripting-window-reference-tab.png}

	\renewcommand{\captionText}{Scripting console, JavaScript reference tab}
	\caption[\captionText]{\textbf{\captionText} The reference tab displays all
	the JavaScript-related documentation extracted from the source code. This
	documentation describes all the JavaScript classes available to the scripting
environment. The description details the members and the methods of the class.}

	\label{fig:minexpert-scripting-window-reference-tab}

\end{figure}

The line edit widget at the bottom of the tab provides an easy means to filter
the documentation such that only the matching lines are displayed in the text
widget. To filter the documentation, enter a regular expression (or a simple
text) and press the \kbdKey{Return} key. To displays the original contents anew,
remove the regular expression and press the \kbdKey{Return} key.

Sometimes, some context around the matching lines might be desirable. The number
of lines upstream and downstream of the matching lines that need to be output
can be defined with the \guilabel{Context lines} spin box. A value of
\guivalue{0} indicates that only the matching lines are output.  A value of
\guivalue{5} indicates that five lines above the matching line and below that
matching line are output.

That JavaScript reference is also available in identical form in the JavaScript
reference section~\vref{sec:javascript-reference} below.


\renewcommand{\sectitle}{Creation of the plot objects as a result of data mining}
\section*{\textcolor{sectioningcolor}{\sectitle}}
\addcontentsline{toc}{section}{\numberline{}\sectitle}

When new plots are created as a result of the user making data mining
operations, new JavaScript objects are created. The JavaScript objects mirror
the C++ objects that live in the guts of \mineXp. The convention is to name the
plot objects made available to the scripting environment using an index, like in
\guivalue{ticChromPlotWidget0}, \guivalue{ticChromPlotWidget1}\dots\ For example, when a new
mass spectrometry data file is loaded, \mineXp automatically computes the TIC
chromatogram and makes it available to the scripting environment under object
name \guivalue{ticChromPlotWidget0}. That name is listed as a child of the
\guivalue{ticChromWnd} object in the tree view of the available objects. This
object will remain available as long as the plot lives in the
\guivalue{ticChromWnd}. Note that simulateanously, the
\guivalue{lastTicChromPlotWidget}
alias is provided to the scripter to capture that newly created object. The
scripter willing to store the name of a newly created plot can thus make an
assignment like in the following code:

{\small
	\begin{verbatim}

	mainWindow.openMassSpectrometryFile(``massSpecFile1.mzml'');

	// The last TIC chromatogram plot is made available under the
	// temporary name lastTicChromPlotWidget, take advantage to store it
	// under a meaningful name for later use

	msFile1TicPlot = lastTicChromPlotWidget

	// Open a new file so that comparisons can be made
	// against the previous one

	mainWindow.openMassSpectrometryFile(``massSpecFile2.mzml'');

	// The lastTicChromPlotWidget is made available, take advantage to store
	// it under a meaningful name for later use

	msFile2TicPlot = lastTicChromPlotWidget

	// Now it is possible to reference the two TIC chromatograms
	// with their new names:

	msFile1TicPlot.integrateToMz(4.5, 7);
	msFile2TicPlot.integrateToMz(4.5, 7);

	\end{verbatim}
}

\clearpage


\renewcommand{\sectitle}{JavaScript-oriented classes}
\section*{\textcolor{sectioningcolor}{\sectitle}}
\addcontentsline{toc}{section}{\numberline{}\sectitle}

At the time of writing, there are a number of C++ classes that have a full
JavaScript counterpart: DataPoint, Trace, MassSpectrum, MzIntegrationParams,
SavGolFilter, for example.  In the following sections, a number of use cases are
detailed to give the reader a flavor of what is scriptable in \mineXp. Note that
the classes are detailed in the scripting reference section
\vref{sec:javascript-reference}.

\renewcommand{\sectitle}{Creating a DataPoint object}
\subsection*{\textcolor{sectioningcolor}{\sectitle}}
\addcontentsline{toc}{subsection}{\numberline{}\sectitle}

The C++ class DataPoint has two member data: m\_key and m\_val. Both are high
precision double values. The JavaScript counter part is also named DataPoint and
can be initialized in two ways:

{\scriptsize

	\begin{verbatim}

	// create a new DataPoint object, initialized using double values
	var dp1 = new DataPoint(123.321,456.654);

	// check that this worked fine:
	dp1.key;
	// --> returns 123.321 (same as dp1["key"])
	dp1.val;
	// --> returns 456.654 (same as dp1["val"])

	dp1.isValid();
	// --> returns true

	// create a new DataPoint object, initialized using an xy-formatted string
	var dp2 = new DataPoint("147.741 258.852");

	// check that this worked fine:
	dp2.key;
	// --> returns 147.741
	dp2.val;
	// --> returns 258.852

	dp2.isValid();
	// --> returns true

	// create an empty DataPoint object
	var dp3 = new DataPoint()

	// an empty DataPoint object is invalid
	dp3.isValid();
	// --> returns false

	// a DataPoint can be initialized a posteriori
	dp1.initialize(159.951, 357.753)

	// or using a string
	dp2.initialize("258.852 741.147");

	\end{verbatim}

} % \scriptsize

The DataPoint class is the main building block Trace objects are made of. A
Trace object is nothing but a list of DataPoint instances.


\renewcommand{\sectitle}{Creating Trace objects}
\subsection*{\textcolor{sectioningcolor}{\sectitle}}
\addcontentsline{toc}{subsection}{\numberline{}\sectitle}

A JavaScript Trace object is nothing but an array of DataPoint objects.  One
useful application of scripting is to perform a variety of calculations on a
given Trace or on a set of Trace\_s.

A JavaScript Trace object can be created and initialized in a number of ways.

\renewcommand{\sectitle}{Creating an empty Trace and initializing it}
\subsubsection*{\textcolor{sectioningcolor}{\sectitle}}
\addcontentsline{toc}{subsection}{\numberline{}\sectitle}

{\scriptsize

	\begin{verbatim}

	// create a new Trace using an xy-formatted string
	var t1 = new Trace("This is the title of the trace object");
	t1.title
	// --> returns ``This is the title of the trace object''

	t1.initialize("123.321 456.654\n789.987 147.741\n258.852 369.963");
	// --> returns 3 // the number of created DataPoint objects

	// check that the trace has effectively 3 items:
	t1.length
	// --> returns 3

	// check that the initialization was ok
	print("(" + t1[0 ].key + "," + t1[0].val + ")")
	// --> returns (123.321,456.654)

	// create a new Trace using two arrays of numbers (key list and value list)
	var t2 = new Trace();
	t2.initialize([123.321,456.654,789.987],[147.741,258.852,369.963])
	// --> returns 3 (the number of created DataPoint objects)

	// check actual length
	t2.length
	// --> returns 3

	// check that the initialization went fine
	t2[0].key
	// --> returns 123.321
	t2[0].val
	// --> returns 147.741

	// get the key and value lists
	t2.keyArray()
	// --> returns 123.321,456.654,789.987
	t2.valArray()
	// --> returns 147.741,258.852,369.963

	// once a Trace has been initialized, it can undergo interesting actions:

	// calculate the sum of all the values:
	t2.valSum()
	// --> returns 776.556

	// a Trace object can combine into itself another Trace object
	t2.combine(t1)

	// now check if the t2 values have increased accordingly
	t2.valArray()
	// --> returns 604.395,369.963,258.852,517.704 // correct combination

	// now subtract t2 from t2:
	t2.subtract(t2)

	// now check that subtraction worked fine
	t2.valArray()
	// --> returns 0,0,0

	// it is possible to initialize a trace with limitation to a given key range
	var t3 = new Trace()
	t3.initializeRange([123.321,456.654,789.987], [147.741,258.852,369.963], 200, 500);

	// there should be only one DataPoint in t3
	t3.length
	// --> returns 1
	t3.keyArray();
	// --> returns 456.654

	\end{verbatim}

} % \scriptsize


\renewcommand{\sectitle}{Creating a Trace object starting from a plot}
\subsubsection*{\textcolor{sectioningcolor}{\sectitle}}
\addcontentsline{toc}{subsection}{\numberline{}\sectitle}

A Trace object can be easily created by using data already obtained from using
\mineXp. For example, a Trace object can be made so that it reproduces a plot
dispalyed in the graphical user interface. The process is shown below:

{\scriptsize

	\begin{verbatim}

	// load mobility data file
	mainWnd.openMassSpectrometryFile(``mobility.mzml'')

	// integrate over the full TIC chromatogram range
	// creates the  massSpecPlotWidget0 object

	ticChromPlotWidget0.integrateToMz()

	// create a new Trace JavaScript object
	var t0 = new Trace()

	// now initialize t0 with the plot data of ticChromPlotWidget0
	t0.initialize(massSpecPlotWidget0.keys(),massSpecPlotWidget0.values());

	// check that the initialization actually worked
	t0.length // should return the number of DataPoint objects in that Trace

	// load a blank data file that acts as a baseline data file
	mainWnd.openMassSpectrometryFile(``mobility-blank.mzml'')

	// integrate over the full TIC chromatogram range
	// creates the massSpecPlotWidget1 object

	ticChromPlotWidget1.integrateToMz()

	// create Trace JavaScript object
	var t1 = new Trace()

	// now initialize t1 with the plot data of ticChromPlotWidget1
	t1.initialize(massSpecPlotWidget1.keys(),massSpecPlotWidget1.values());

	// check that the initialization actually worked
	t1.length // should return the number of DataPoint objects in that Trace

	// finally perform some useful task: subtract noise from t0
	t0.subtract(trace1)

	// Trace object t0 now has the baseline removed.

	\end{verbatim}

}%\scriptsize


\renewcommand{\sectitle}{Exporting a Trace object to a file}
\subsubsection*{\textcolor{sectioningcolor}{\sectitle}}
\addcontentsline{toc}{subsection}{\numberline{}\sectitle}

It is possible at each moment to export a given Trace object (or plot) to a
file.

\medskip

This is possible either at the JavaScript Trace object level:

{\scriptsize

	\begin{verbatim}

	mainWnd.openMassSpectrometryFile("mobility.mzml")
	ticChromPlotWidget0.integrateToMz()
	var t0 = new Trace()
	t0.initialize(massSpecPlotWidget0.keys(),massSpecPlotWidget0.values());
	t0.exportToFile("/home/rusconi/demo.xy");

	\end{verbatim}

} %\scriptsize


Or, at the plot level:

{\scriptsize

	\begin{verbatim}

	mainWnd.openMassSpectrometryFile("mobility.mzml")
	ticChromPlotWidget0.integrateToMz()

	// note how the export function name has ``Plot'' in it.
	massSpecPlotWidget0.exportPlotToFile("/home/rusconi/demobis.xy");

	\end{verbatim}

} %\scriptsize


At this point both files demo.xy and demobis.xy are bit-by-bit identical.


\renewcommand{\sectitle}{Plotting a JavaScript Trace object to a plot wiget
window}
\subsubsection*{\textcolor{sectioningcolor}{\sectitle}}
\addcontentsline{toc}{subsection}{\numberline{}\sectitle}

It is necessary for the data miner to be able to look at her data graphically.
Nobody wants to scrutinize data in ASCII files.

Anytime the scripter has a Trace object, she can plot it in the window that
makes sense for the data. For example, if a plot derives from mass spectral
data, then logically, the new plot should derive for these mass spectral data,
as exemplified below:

{\scriptsize

	\begin{verbatim}

	mainWnd.openMassSpectrometryFile("mobility.mzml");
	ticChromPlotWidget0.integrateToMz();

	var t1 = new Trace();
	t1.initialize(massSpecPlotWidget0.keys(), massSpecPlotWidget0.values());

	// and now plot that Trace object as a descendant of massSpecPlotWidget0.
	// if massSpecPlotWidget0 is removed, then the descendant will be removed also
	massSpecPlotWidget0.newPlot(t1);

	\end{verbatim}

} %\scriptsize

Displaying a new plot using the \command{<existingPlot Object>.newPlot(<Trace>)}
call occurs in the window that matches the type of <existingPlot>. If
<existingPlot> is a mass spectrum, then the descendant plot will be displayed in
the Mass~Spectrum~Window.

Any of the plot widget windows can receive a new plot according to the mechanics
described above.


\renewcommand{\sectitle}{Printing a JavaScript Trace object to the console}
\subsubsection*{\textcolor{sectioningcolor}{\sectitle}}
\addcontentsline{toc}{subsection}{\numberline{}\sectitle}

It is possible to have a look at the numerical data of a Trace object like so:

{\scriptsize

	\begin{verbatim}

	mainWnd.openMassSpectrometryFile("mobility.mzml");
	ticChromPlotWidget0.integrateToMz();

	var t1 = new Trace();
	t1.initialize(massSpecPlotWidget0.keys(), massSpecPlotWidget0.values());

	// print the data to the console
	t1.asText()

	// outputs a series of xy-formatted lines like so:
	// 2504.3185084749 1262.0000000000
	// 2504.3503266026 1277.0000000000
	// ...

	// doing this is also possible at the plot widget level
	// note the function name change
	massSpecPlotWidget0.asXyText()

	\end{verbatim}

} %\scriptsize


\renewcommand{\sectitle}{Running a JavaScript script from file}
\subsubsection*{\textcolor{sectioningcolor}{\sectitle}}
\addcontentsline{toc}{subsection}{\numberline{}\sectitle}

It is possible to run a script directly from file by using either the following
flag to the \mineXp command line:

\medskip
\verb| minexpert -j <scriptFile>|
\medskip

or by using the \guimenu{File}\guimenuitem{Run a JavaScript from file} menu of
the scripting window. In that case the script is immediately run and the results
are printed in the upper part of the \guilabel{Scripting} tab of the scripting
window.


\clearpage


