The Go Text Protocol
====================

GNU Go 3.0 introduced a new interface, the Go Text Protocol, abbreviated GTP. The intention was to make an interface that is better suited for machine-machine communication than the ascii interface and simpler, more powerful, and more flexible than the Go Modem Protocol.

There are two versions of the protocol. Version 1 was used with GNU Go 3.0 and 3.2. GNU Go 3.4 and later versions use protocol version 2. The specification of GTP version 2 is available at http://www.lysator.liu.se/~gunnar/gtp/. GNU Go 3.4 is the reference implementation for GTP version 2, but all but the most common commands are to be regarded as private extensions of the protocol.

The GTP has a variety of applications. For GNU Go the first use was in regression testing (see Regression), followed by communication with the NNGS go server and for automated test games against itself and other programs. Now there are also many graphical user interfaces available supporting GTP, as well as bridges to other Go servers than NNGS.

Running GNU Go in GTP mode
==========================

To start GNU Go in GTP mode, simply invoke it with the option --mode gtp. You will not get a prompt or any other output to start with but GNU Go is silently waiting for GTP commands.

A sample GTP session may look as follows:

     virihaure 462% ./gnugo --mode gtp
     1 boardsize 7
     =1

     2 clear_board
     =2

     3 play black D5
     =3

     4 genmove white
     =4 C3

     5 play black C3
     ?5 illegal move

     6 play black E3
     =6

     7 showboard
     =7
        A B C D E F G
      7 . . . . . . . 7
      6 . . . . . . . 6
      5 . . + X + . . 5
      4 . . . + . . . 4
      3 . . O . X . . 3
      2 . . . . . . . 2     WHITE (O) has captured 0 stones
      1 . . . . . . . 1     BLACK (X) has captured 0 stones
        A B C D E F G

     8 quit
     =8

Commands are given on a single line, starting by an optional identity number, followed by the command name and its arguments.

If the command is successful, the response starts by an equals sign (=), followed by the identity number of the command (if any) and then the result. In this example all results were empty strings except for command 4 where the answer was the white move at C3, and command 7 where the result was a diagram of the current board position. The response ends by two consecutive newlines.

Failing commands are signified by a question mark (?) instead of an equals sign, as in the response to command 5.

The detailed specification of the protocol can be found at http://www.lysator.liu.se/~gunnar/gtp/. The available commands in GNU Go may always be listed using the command list_commands. They are also documented in see GTP command reference.

GTP applications
================

GTP is an asymmetric protocol involving two parties which we call controller and engine. The controller sends all commands and the engine only responds to these commands. GNU Go implements the engine end of the protocol.

With the source code of GNU Go is also distributed a number of applications implementing the controller end. Among the most interesting of these are:

  * regression/regress.awk

          Script to run regressions. The script sends GTP commands to set up and evaluate positions to the engine and then analyzes the responses from the engine. More information about GTP based regression testing can be found in the regression chapter (see Regression).

  * regression/regress.pl

          Perl script to run regressions, giving output which together with the CGI script regression/regress.plx generates HTML views of the regressions.

  * regression/regress.pike

          Pike script to run regressions. More feature-rich and powerful than regress.awk.

  * regression/view.pike

          Pike script to examine a single regression testcase through a graphical board. This gives an easy way to inspect many of the GNU Go internals.

  * interface/gtp_examples/twogtp

          Perl script to play two engines against each other. The script essentially sets up both engines with desired boardsize, handicap, and komi, then relays moves back and forth between the engines.

  * interface/gtp_examples/twogtp-a

          An alternative Perl implementation of twogtp.

  * interface/gtp_examples/twogtp.py

          Implementation of twogtp in Python. Has more features than the Perl variants.

  * interface/gtp_examples/twogtp.pike

          Implementation of twogtp in Pike. Has even more features than the Python variant.

  * interface/gtp_examples/2ptkgo.pl

          Variation of twogtp which includes a graphical board.

More GTP applications, including bridges to go servers and graphical user interfaces, are listed at http://www.lysator.liu.se/~gunnar/gtp/.

The Metamachine
===============

An interesting application of the GTP is the concept of using GNU Go as an ``Oracle'' that can be consulted by another process. This could be another computer program that asks GNU Go to generate future board positions, then evaluate them.

David Doshay at the University of California at Santa Cruz has done interesting experiments with a parallel engine, known as SlugGo, that is based on GNU Go.  These are described in http://lists.gnu.org/archive/html/gnugo-devel/2004-08/msg00060.html.

The ``Metamachine'' experiment is a more modest approach using the GTP to communicate with a GNU Go process that is used as an oracle. The following scheme is used.

  * The GNU Go ``oracle'' is asked to generate its top moves using the GTP top_moves commands.

  * Both moves are tried and estimate_score is called from the resulting board position.

  * The higher scoring position is selected as the engine's move.

This scheme does not produce a stronger engine, but it is suggestive, and the SlugGo experiment seems to show that a more elaborate scheme along the same lines could produce a stronger engine.

Two implementations are distributed with GNU Go. Both make use of fork and pipe system calls, so they require a Unix-like environment.  The Metamachine has been tested under GNU/Linux.

Important: If the Metamachine terminates normally, the GNU Go process will be killed. However there is a danger that something will go wrong. When you are finished running the Metamachine, it is a good idea to run ps -A|grep gnugo or ps -aux|grep gnugo to make sure there are no unterminated processes. (If there are, just kill them.)

The Standalone Metamachine
--------------------------

In interface/gtp_examples/metamachine.c is a standalone implementation of the Metamachine. Compile it with cc -o metamachine metamachine.c and run it. It forks a gnugo process with which it communicates through the GTP, to use as an oracle.

The following scheme is followed:

                  stdin             pipe a
       GTP client ----> Metamachine -----> GNU Go
                  <----             <-----
                 stdout             pipe b

Most commands issued by the client are passed along verbatim to GNU Go by the Metamachine. The exception is gg_genmove, which is intercepted then processed differently, as described above. The client is unaware of this, and only knows that it issued a gg_genmove command and received a reply. Thus to the the Metamachine appears as an ordinary GTP engine.

Usage: no arguments gives normal GTP behavior. metamachine --debug sends diagnostics to stderr.

GNU Go as a Metamachine
-----------------------

Alternatively, you may compile GNU Go with the configure option --enable-metamachine. This causes the file oracle.c to be compiled, which contains the Metamachine code. This has no effect on the engine unless you run GNU Go with the runtime option --metamachine. Thus you must use both the configure and the runtime option to get the Metamachine.

This method is better than the standalone program since you have access to GNU Go's facilities. For example, you can run the Metamachine with CGoban or in Ascii mode this way.

You can get traces by adding the command line -d0x1000000. In debugging the Metamachine, a danger is that any small oversight in designing the program can cause the forked process and the controller to hang, each one waiting for a response from the other. If this seems to happen it is useful to know that you can attach gdb to a running process and find out what it is doing.

Adding new GTP commands
=======================

The implementation of GTP in GNU Go is distributed over three files, interface/gtp.h, interface/gtp.c, and interface/play_gtp.c. The first two implement a small library of helper functions which can be used also by other programs. In the interest of promoting the GTP they are licensed with minimal restrictions (see GTP License). The actual GTP commands are implemented in play_gtp.c, which has knowledge about the engine internals.

To see how a simple but fairly typical command is implemented we look at gtp_countlib() (a GNU Go private extension command):

     static int
     gtp_countlib(char *s)
     {
       int i, j;
       if (!gtp_decode_coord(s, &i, &j))
         return gtp_failure("invalid coordinate");

       if (BOARD(i, j) == EMPTY)
         return gtp_failure("vertex must not be empty");

       return gtp_success("%d", countlib(POS(i, j)));
     }

The arguments to the command are passed in the string s. In this case we expect a vertex as argument and thus try to read it with gtp_decode_coord() from gtp.c.

A correctly formatted response should start with either = or ?, followed by the identity number (if one was sent), the actual result, and finally two consecutive newlines. It is important to get this formatting correct since the controller in the other end relies on it. Naturally the result itself cannot contain two consecutive newlines but it may be split over several lines by single newlines.

The easiest way to generate a correctly formatted response is with one of the functions gtp_failure() and gtp_success(), assuming that their formatted output does not end with a newline.

Sometimes the output is too complex for use with gtp_success, e.g. if we want to print vertices, which gtp_success() does not support. Then we have to fall back to the construction in e.g. gtp_genmove():

     static int
     gtp_genmove(char *s)
     {
       [...]
       gtp_start_response(GTP_SUCCESS);
       gtp_print_vertex(i, j);
       return gtp_finish_response();
     }

Here gtp_start_response() writes the equal sign and the identity number while gtp_finish_response() adds the final two newlines. The next example is from gtp_list_commands():

     static int
     gtp_list_commands(char *s)
     {
       int k;
       UNUSED(s);

       gtp_start_response(GTP_SUCCESS);

       for (k = 0; commands[k].name != NULL; k++)
         gtp_printf("%s\n", commands[k].name);

       gtp_printf("\n");
       return GTP_OK;
     }

As we have said, the response should be finished with two newlines. Here we have to finish up the response ourselves since we already have one newline in place from the last command printed in the loop.

In order to add a new GTP command to GNU Go, the following pieces of code need to be inserted in play_gtp.c:

 1. A function declaration using the DECLARE macro in the list starting at line 68.

 2. An entry in the commands[] array starting at line 200.

 3. An implementation of the function handling the command.

Useful helper functions in gtp.c/gtp.h are:

  * gtp_printf() for basic formatted printing.

  * gtp_mprintf() for printing with special format codes for vertices and colors.

  * gtp_success() and gtp_failure() for simple responses.

  * gtp_start_response() and gtp_end_response() for more complex responses.

  * gtp_print_vertex() and gtp_print_vertices() for printing one or multiple vertices.

  * gtp_decode_color() to read in a color from the command arguments.

  * gtp_decode_coord() to read in a vertex from the command arguments.

  * gtp_decode_move() to read in a move, i.e. color plus vertex, from the command arguments.

GTP command reference
=====================

This section lists the GTP commands implemented in GNU Go along with some information about each command. Each entry in the list has the following fields:

  * Function: What this command does.

  * Arguments: What other information, if any, this command requires. Typical values include none or vertex or integer (there are others).

  * Fails: Circumstances which cause this command to fail.

  * Returns: What is displayed after the = and before the two newlines.  Typical values include nothing or a move coordinate or some status string (there are others).

  * Status: How this command relates to the standard GTP version 2 commands. If nothing else is specified it is a GNU Go private extension.

Without further ado, here is the big list (in no particular order).

Note: if new commands are added by editing interface/play_gtp.c this list could become incomplete. You may rebuild this list in doc/gtp-commands.texi with the command make gtp-commands in the doc/ directory. This may require GNU sed.

          [file gtp-commands.texi not included]