How To Use Gflags (formerly Google Commandline Flags)

(as of )
Table of contents
Introduction
DEFINE: Defining Flags In Program
Accessing the Flag
DECLARE: Using the Flag in a Different File
RegisterFlagValidator: Sanity-checking Flag Values
Putting It Together: How to Set Up Flags
Setting Flags on the Command Line
Setting Flags at Runtime
Changing the Default Flag Value
Special Flags
The API

Introduction, and Comparison to Other Commandline Flags Libraries

Commandline flags are flags that users specify on the command line when they run an executable. In the command

   fgrep -l -f /var/tmp/foo johannes brahms

-l and -f /var/tmp/foo are the two commandline flags. (johannes and brahms, which don't start with a dash, are commandline arguments.)

Typically, an application lists what flags the user is allowed to pass in, and what arguments they take -- in this example, -l takes no argument, and -f takes a string (in particular, a filename) as an argument. Users can use a library to help parse the commandline and store the flags in some data structure.

Gflags, the commandline flags library used within Google, differs from other libraries, such as getopt(), in that flag definitions can be scattered around the source code, and not just listed in one place such as main(). In practice, this means that a single source-code file will define and use flags that are meaningful to that file. Any application that links in that file will get the flags, and the gflags library will automatically handle that flag appropriately.

There's significant gain in flexibility, and ease of code reuse, due to this technique. However, there is a danger that two files will define the same flag, and then give an error when they're linked together.

The rest of this document describes how to use the commandlineflag library. It's a C++ library, so examples are in C++. However, there is a Python port with the same functionality, and this discussion translates directly to Python.

DEFINE: Defining Flags In Program

Defining a flag is easy: just use the appropriate macro for the type you want the flag to be, as defined at the bottom of gflags/gflags.h. Here's an example file, foo.cc:

   #include <gflags/gflags.h>

   DEFINE_bool(big_menu, true, "Include 'advanced' options in the menu listing");
   DEFINE_string(languages, "english,french,german",
                 "comma-separated list of languages to offer in the 'lang' menu");

DEFINE_bool defines a boolean flag. Here are the types supported:

Note that there are no 'complex' types like lists: the "languages" flag in our example is a list of strings, but is defined of type "string", not "list_of_string" or similar. This is by design. We'd rather use only simple types for the flags, and allow for complex, arbitrary parsing routines to parse them, than to try to put the logic inside the flags library proper.

All DEFINE macros take the same three arguments: the name of the flag, its default value, and a 'help' string that describes its use. The 'help' string is displayed when the user runs the application with the --help flag.

You can define a flag in any source-code file in your executable. Only define a flag once! If you want to access a flag in more than one source file, DEFINE it in one file, and DECLARE it in the others. Even better, DEFINE it in foo.cc and DECLARE it in foo.h; then everyone who #includes foo.h can use the flag.

Defining flags in libraries rather than in main() is powerful, but does have some costs. One is that a library might not have a good default value for its flags, for example if the flag holds a filename that might not exist in some environments. To mitigate such problems, you can use flag validators to ensure prompt notification (in the form of a crash) of an invalid flag value.

Note that while most functions in this library are defined in the google namespace, DEFINE_foo (and DECLARE_foo, below), should always be in the global namespace.

Accessing the Flag

All defined flags are available to the program as just a normal variable, with the prefix FLAGS_ prepended. In the above example, the macros define two variables, FLAGS_big_menu (a bool), and FLAGS_languages (a C++ string).

You can read and write to the flag just like any other variable:

   if (FLAGS_consider_made_up_languages)
     FLAGS_languages += ",klingon";   // implied by --consider_made_up_languages
   if (FLAGS_languages.find("finnish") != string::npos)
     HandleFinnish();

You can also get and set flag values via special functions in gflags.h. That's a rarer use case, though.

DECLARE: Using the Flag in a Different File

Accessing a flag in the manner of the previous section only works if the flag was DEFINE-ed at the top of the file. If it wasn't, you'll get an 'unknown variable' error.

The DECLARE_type macro is available when you want to use a flag that's defined in another file. For instance, if I were writing bar.cc but wanted to access the big_menu, flag, I would put this near the top of bar.cc:

   DECLARE_bool(big_menu);

This is functionally equivalent to saying extern FLAGS_big_menu.

Note that such an extern declaration introduces a dependency between your file and the file that defines the big_menu flag: foo.cc, in this case. Such implicit dependencies can be difficult to manage in large projects. For that reason we recommend the following guideline:

If you DEFINE a flag in foo.cc, either don't DECLARE it at all, only DECLARE it in tightly related tests, or only DECLARE it in foo.h.

You should go the do-not-DECLARE route when the flag is only needed by foo.cc, and not in any other file. If you want to modify the value of the flag in the related test file to see if it is functioning as expected, DECLARE it in the foo_test.cc file.

If the flag does span multiple files, DECLARE it in the associated .h file, and make others #include that .h file if they want to access the flag. The #include will make explicit the dependency between the two files. This causes the flag to be a global variable.

RegisterFlagValidator: Sanity-checking Flag Values

After DEFINE-ing a flag, you may optionally register a validator function with the flag. If you do this, after the flag is parsed from the commandline, and whenever its value is changed via a call to SetCommandLineOption(), the validator function is called with the new value as an argument. The validator function should return 'true' if the flag value is valid, and false otherwise. If the function returns false for the new setting of the flag, the flag will retain its current value. If it returns false for the default value, ParseCommandLineFlags will die.

Here is an example use of this functionality:

static bool ValidatePort(const char* flagname, int32 value) {
   if (value > 0 && value < 32768)   // value is ok
     return true;
   printf("Invalid value for --%s: %d\n", flagname, (int)value);
   return false;
}
DEFINE_int32(port, 0, "What port to listen on");
static const bool port_dummy = RegisterFlagValidator(&FLAGS_port, &ValidatePort);

By doing the registration at global initialization time (right after the DEFINE), we ensure that the registration happens before the commandline is parsed at the beginning of main().

RegisterFlagValidator() returns true if the registration is successful. It return false if the registration fails because a) the first argument does not refer to a commandline flag, or b) a different validator has already been registered for this flag.

Putting It Together: How to Set Up Flags

The final piece is the one that tells the executable to process the commandline flags, and set the FLAGS_* variables to the appropriate, non-default value based on what is seen on the commandline. This is equivalent to the getopt() call in the getopt library, but has much less overhead to use. In fact, it's just a single function call:

   google::ParseCommandLineFlags(&argc, &argv, true);

Usually, this code is at the beginning of main(). argc and argv are exactly as passed in to main(). This routine might modify them, which is why pointers to them are passed in.

The last argument is called "remove_flags". If true, then ParseCommandLineFlags removes the flags and their arguments from argv, and modifies argc appropriately. In this case, after the function call, argv will hold only commandline arguments, and not commandline flags.

If, on the other hand, remove_flags is false, then ParseCommandLineFlags will leave argc unchanged, but will rearrange the arguments in argv so that the flags are all at the beginning. For example, if the input is "/bin/foo" "arg1" "-q" "arg2" (which is legal but weird), the function will rearrange argv so it reads "/bin/foo", "-q", "arg1", "arg2". In this case, ParseCommandLineFlags returns the index into argv that holds the first commandline argument: that is, the index past the last flag. (In this example, it would return 2, since argv[2] points to arg1.)

In either case, the FLAGS_* variables are modified based on what was passed in on the commandline.

Setting Flags on the Command Line

The reason you make something a flag instead of a compile-time constant, is so users can specify a non-default value on the commandline. Here's how they might do it for an application that links in foo.cc:

   app_containing_foo --nobig_menu -languages="chinese,japanese,korean" ...

This sets FLAGS_big_menu = false; and FLAGS_languages = "chinese,japanese,korean", when ParseCommandLineFlags is run.

Note the atypical syntax for setting a boolean flag to false: putting "no" in front of its name. There's a fair bit of flexibility to how flags may be specified. Here's an example of all the ways to specify the "languages" flag:

For boolean flags, the possibilities are slightly different:

(as well as the single-dash variant on all of these).

Despite this flexibility, we recommend using only a single form: --variable=value for non-boolean flags, and --variable/--novariable for boolean flags. This consistency will make your code more readable, and is also the format required for certain special-use cases like flagfiles.

It is a fatal error to specify a flag on the commandline that has not been DEFINED somewhere in the executable. If you need that functionality for some reason -- say you want to use the same set of flags for several executables, but not all of them DEFINE every flag in your list -- you can specify --undefok to suppress the error.

As in getopt(), -- by itself will terminate flags processing. So in foo -f1 1 -- -f2 2, f1 is considered a flag, but -f2 is not.

If a flag is specified more than once, only the last specification is used; the others are ignored.

Note that flags do not have single-letter synonyms, like they do in the getopt library, nor do we allow "combining" flags behind a single dash, as in ls -la.

Changing the Default Flag Value

Sometimes a flag is defined in a library, and you want to change its default value in one application but not others. It's simple to do this: just assign a new value to the flag in main(), before calling ParseCommandLineFlags():

   DECLARE_bool(lib_verbose);   // mylib has a lib_verbose flag, default is false
   int main(int argc, char** argv) {
     FLAGS_lib_verbose = true;  // in my app, I want a verbose lib by default
     ParseCommandLineFlags(...);
   }

For this application, users can still set the flag value on the commandline, but if they do not, the flag's value will default to true.

Special Flags

There are a few flags defined by the commandlineflags module itself, and are available to all applications that use commandlineflags. These fall into three categories. First are the 'reporting' flags that, when found, cause the application to print some information about itself and exit.

--help shows all flags from all files, sorted by file and then by name; shows the flagname, its default value, and its help string
--helpfull same as -help, but unambiguously asks for all flags (in case -help changes in the future)
--helpshort shows only flags for the file with the same name as the executable (usually the one containing main())
--helpxml like --help, but output is in xml for easier parsing
--helpon=FILE   shows only flags defined in FILE.*
--helpmatch=S shows only flags defined in *S*.*
--helppackage shows flags defined in files in same directory as main()
--version prints version info for the executable

Second are the flags that affect how other flags are parsed.

--undefok=flagname,flagname,... for those names listed as the argument to --undefok, suppress the normal error-exit that occurs when --name is seen on the commandline, but name has not been DEFINED anywhere in the application

Third are the 'recursive' flags, that cause other flag values to be set: --fromenv, --tryfromenv, --flagfile. These are described below in more detail.

--fromenv

--fromenv=foo,bar says to read the values for the foo and bar flags from the environment. In concert with this flag, you must actually set the values in the environment, via a line like one of the two below:

   export FLAGS_foo=xxx; export FLAGS_bar=yyy   # sh
   setenv FLAGS_foo xxx; setenv FLAGS_bar yyy   # tcsh

This is equivalent to specifying --foo=xxx, --bar=yyy on the commandline.

Note it is a fatal error to say --fromenv=foo if foo is not DEFINED somewhere in the application. (Though you can suppress this error via --undefok=foo, just like for any other flag.)

It is also a fatal error to say --fromenv=foo if FLAGS_foo is not actually defined in the environment.

--tryfromenv

--tryfromenv is exactly like --fromenv, except it is not a fatal error to say --tryfromenv=foo if FLAGS_foo is not actually defined in the environment. Instead, in such cases, FLAGS_foo just keeps its default value as specified in the application.

Note it is still an error to say --tryfromenv=foo if foo is not DEFINED somewhere in the application.

--flagfile

--flagfile=f tells the commandlineflags module to read the file f, and to run all the flag-assignments found in that file as if these flags had been specified on the commandline.

In its simplest form, f should just be a list of flag assignments, one per line. Unlike on the commandline, the equals sign separating a flagname from its argument is required for flagfiles. An example flagfile, /tmp/myflags:

--nobig_menus
--languages=english,french

With this flagfile, the following two lines are equivalent:

   ./myapp --foo --nobig_menus --languages=english,french --bar
   ./myapp --foo --flagfile=/tmp/myflags --bar

Note that many errors are silently suppressed in flagfiles. In particular, unrecognized flagnames are silently ignored, as are flags that are missing a required value (e.g., a flagfile that just says --languages).

The general format of a flagfile is a bit more complicated than the simple, common case above. It is: a sequence of filenames, one per line, followed by a sequence of flags, one per line, repeated as many times as desired. Filenames in a flagfile can use wildcards (* and ?), and the sequence of flags located after a sequence of filenames is processed only if the current executable's name matches one of the filenames. It is possible to start the flagfile with a sequence of flags instead of a sequence of filenames; if such a sequence of flags is present, these flags are applied to the current executable no matter what it is.

Lines that start with a # are ignored as comments. Leading whitespace is also ignored in flagfiles, as are blank lines.

It is possible for a flagfile to use the --flagfile flag to include another flagfile.

Flags are always processed in the expected order. That is, processing begins by examining the flags specified directly on the command line. If a flagfile is specified, its contents are processed, and then processing continues with remaining flags from the command line.

The API

In addition to accessing FLAGS_foo directly, it is possible to access the flags programmatically, through an API. It is also possible to access information about a flag, such as its default value and help-string. A FlagSaver makes it easy to modify flags and then automatically undo the modifications later. Finally, there are somewhat unrelated, but useful, routines to easily access parts of argv outside main, including the program name (argv[0]).

For more information about these routines, and other useful helper methods such as google::SetUsageMessage() and google::SetVersionString, see gflags.h.

Miscellaneous Notes

If your application has code like this:

   #define STRIP_FLAG_HELP 1    // this must go before the #include!
   #include <gflags/gflags.h>

we will remove the help messages from the compiled source. This can reduce the size of the resulting binary somewhat, and may also be useful for security reasons.

Craig Silverstein