doc.go

     1  // Copyright 2018 The Go Authors. All rights reserved.
     2  // Use of this source code is governed by a BSD-style
     3  // license that can be found in the LICENSE file.
     4  
     5  /*
     6  Package analysis defines the interface between a modular static
     7  analysis and an analysis driver program.
     8  
     9  # Background
    10  
    11  A static analysis is a function that inspects a package of Go code and
    12  reports a set of diagnostics (typically mistakes in the code), and
    13  perhaps produces other results as well, such as suggested refactorings
    14  or other facts. An analysis that reports mistakes is informally called a
    15  "checker". For example, the printf checker reports mistakes in
    16  fmt.Printf format strings.
    17  
    18  A "modular" analysis is one that inspects one package at a time but can
    19  save information from a lower-level package and use it when inspecting a
    20  higher-level package, analogous to separate compilation in a toolchain.
    21  The printf checker is modular: when it discovers that a function such as
    22  log.Fatalf delegates to fmt.Printf, it records this fact, and checks
    23  calls to that function too, including calls made from another package.
    24  
    25  By implementing a common interface, checkers from a variety of sources
    26  can be easily selected, incorporated, and reused in a wide range of
    27  driver programs including command-line tools (such as vet), text editors and
    28  IDEs, build and test systems (such as go build, Bazel, or Buck), test
    29  frameworks, code review tools, code-base indexers (such as SourceGraph),
    30  documentation viewers (such as godoc), batch pipelines for large code
    31  bases, and so on.
    32  
    33  # Analyzer
    34  
    35  The primary type in the API is [Analyzer]. An Analyzer statically
    36  describes an analysis function: its name, documentation, flags,
    37  relationship to other analyzers, and of course, its logic.
    38  
    39  To define an analysis, a user declares a (logically constant) variable
    40  of type Analyzer. Here is a typical example from one of the analyzers in
    41  the go/analysis/passes/ subdirectory:
    42  
    43  	package unusedresult
    44  
    45  	var Analyzer = &analysis.Analyzer{
    46  		Name: "unusedresult",
    47  		Doc:  "check for unused results of calls to some functions",
    48  		Run:  run,
    49  		...
    50  	}
    51  
    52  	func run(pass *analysis.Pass) (interface{}, error) {
    53  		...
    54  	}
    55  
    56  An analysis driver is a program such as vet that runs a set of
    57  analyses and prints the diagnostics that they report.
    58  The driver program must import the list of Analyzers it needs.
    59  Typically each Analyzer resides in a separate package.
    60  To add a new Analyzer to an existing driver, add another item to the list:
    61  
    62  	import ( "unusedresult"; "nilness"; "printf" )
    63  
    64  	var analyses = []*analysis.Analyzer{
    65  		unusedresult.Analyzer,
    66  		nilness.Analyzer,
    67  		printf.Analyzer,
    68  	}
    69  
    70  A driver may use the name, flags, and documentation to provide on-line
    71  help that describes the analyses it performs.
    72  The doc comment contains a brief one-line summary,
    73  optionally followed by paragraphs of explanation.
    74  
    75  The [Analyzer] type has more fields besides those shown above:
    76  
    77  	type Analyzer struct {
    78  		Name             string
    79  		Doc              string
    80  		Flags            flag.FlagSet
    81  		Run              func(*Pass) (interface{}, error)
    82  		RunDespiteErrors bool
    83  		ResultType       reflect.Type
    84  		Requires         []*Analyzer
    85  		FactTypes        []Fact
    86  	}
    87  
    88  The Flags field declares a set of named (global) flag variables that
    89  control analysis behavior. Unlike vet, analysis flags are not declared
    90  directly in the command line FlagSet; it is up to the driver to set the
    91  flag variables. A driver for a single analysis, a, might expose its flag
    92  f directly on the command line as -f, whereas a driver for multiple
    93  analyses might prefix the flag name by the analysis name (-a.f) to avoid
    94  ambiguity. An IDE might expose the flags through a graphical interface,
    95  and a batch pipeline might configure them from a config file.
    96  See the "findcall" analyzer for an example of flags in action.
    97  
    98  The RunDespiteErrors flag indicates whether the analysis is equipped to
    99  handle ill-typed code. If not, the driver will skip the analysis if
   100  there were parse or type errors.
   101  The optional ResultType field specifies the type of the result value
   102  computed by this analysis and made available to other analyses.
   103  The Requires field specifies a list of analyses upon which
   104  this one depends and whose results it may access, and it constrains the
   105  order in which a driver may run analyses.
   106  The FactTypes field is discussed in the section on Modularity.
   107  The analysis package provides a Validate function to perform basic
   108  sanity checks on an Analyzer, such as that its Requires graph is
   109  acyclic, its fact and result types are unique, and so on.
   110  
   111  Finally, the Run field contains a function to be called by the driver to
   112  execute the analysis on a single package. The driver passes it an
   113  instance of the Pass type.
   114  
   115  # Pass
   116  
   117  A [Pass] describes a single unit of work: the application of a particular
   118  Analyzer to a particular package of Go code.
   119  The Pass provides information to the Analyzer's Run function about the
   120  package being analyzed, and provides operations to the Run function for
   121  reporting diagnostics and other information back to the driver.
   122  
   123  	type Pass struct {
   124  		Fset         *token.FileSet
   125  		Files        []*ast.File
   126  		OtherFiles   []string
   127  		IgnoredFiles []string
   128  		Pkg          *types.Package
   129  		TypesInfo    *types.Info
   130  		ResultOf     map[*Analyzer]interface{}
   131  		Report       func(Diagnostic)
   132  		...
   133  	}
   134  
   135  The Fset, Files, Pkg, and TypesInfo fields provide the syntax trees,
   136  type information, and source positions for a single package of Go code.
   137  
   138  The OtherFiles field provides the names of non-Go
   139  files such as assembly that are part of this package.
   140  Similarly, the IgnoredFiles field provides the names of Go and non-Go
   141  source files that are not part of this package with the current build
   142  configuration but may be part of other build configurations.
   143  The contents of these files may be read using Pass.ReadFile;
   144  see the "asmdecl" or "buildtags" analyzers for examples of loading
   145  non-Go files and reporting diagnostics against them.
   146  
   147  The ResultOf field provides the results computed by the analyzers
   148  required by this one, as expressed in its Analyzer.Requires field. The
   149  driver runs the required analyzers first and makes their results
   150  available in this map. Each Analyzer must return a value of the type
   151  described in its Analyzer.ResultType field.
   152  For example, the "ctrlflow" analyzer returns a *ctrlflow.CFGs, which
   153  provides a control-flow graph for each function in the package (see
   154  golang.org/x/tools/go/cfg); the "inspect" analyzer returns a value that
   155  enables other Analyzers to traverse the syntax trees of the package more
   156  efficiently; and the "buildssa" analyzer constructs an SSA-form
   157  intermediate representation.
   158  Each of these Analyzers extends the capabilities of later Analyzers
   159  without adding a dependency to the core API, so an analysis tool pays
   160  only for the extensions it needs.
   161  
   162  The Report function emits a diagnostic, a message associated with a
   163  source position. For most analyses, diagnostics are their primary
   164  result.
   165  For convenience, Pass provides a helper method, Reportf, to report a new
   166  diagnostic by formatting a string.
   167  Diagnostic is defined as:
   168  
   169  	type Diagnostic struct {
   170  		Pos      token.Pos
   171  		Category string // optional
   172  		Message  string
   173  	}
   174  
   175  The optional Category field is a short identifier that classifies the
   176  kind of message when an analysis produces several kinds of diagnostic.
   177  
   178  The [Diagnostic] struct does not have a field to indicate its severity
   179  because opinions about the relative importance of Analyzers and their
   180  diagnostics vary widely among users. The design of this framework does
   181  not hold each Analyzer responsible for identifying the severity of its
   182  diagnostics. Instead, we expect that drivers will allow the user to
   183  customize the filtering and prioritization of diagnostics based on the
   184  producing Analyzer and optional Category, according to the user's
   185  preferences.
   186  
   187  Most Analyzers inspect typed Go syntax trees, but a few, such as asmdecl
   188  and buildtag, inspect the raw text of Go source files or even non-Go
   189  files such as assembly. To report a diagnostic against a line of a
   190  raw text file, use the following sequence:
   191  
   192  	content, err := pass.ReadFile(filename)
   193  	if err != nil { ... }
   194  	tf := fset.AddFile(filename, -1, len(content))
   195  	tf.SetLinesForContent(content)
   196  	...
   197  	pass.Reportf(tf.LineStart(line), "oops")
   198  
   199  # Modular analysis with Facts
   200  
   201  To improve efficiency and scalability, large programs are routinely
   202  built using separate compilation: units of the program are compiled
   203  separately, and recompiled only when one of their dependencies changes;
   204  independent modules may be compiled in parallel. The same technique may
   205  be applied to static analyses, for the same benefits. Such analyses are
   206  described as "modular".
   207  
   208  A compiler’s type checker is an example of a modular static analysis.
   209  Many other checkers we would like to apply to Go programs can be
   210  understood as alternative or non-standard type systems. For example,
   211  vet's printf checker infers whether a function has the "printf wrapper"
   212  type, and it applies stricter checks to calls of such functions. In
   213  addition, it records which functions are printf wrappers for use by
   214  later analysis passes to identify other printf wrappers by induction.
   215  A result such as “f is a printf wrapper” that is not interesting by
   216  itself but serves as a stepping stone to an interesting result (such as
   217  a diagnostic) is called a [Fact].
   218  
   219  The analysis API allows an analysis to define new types of facts, to
   220  associate facts of these types with objects (named entities) declared
   221  within the current package, or with the package as a whole, and to query
   222  for an existing fact of a given type associated with an object or
   223  package.
   224  
   225  An Analyzer that uses facts must declare their types:
   226  
   227  	var Analyzer = &analysis.Analyzer{
   228  		Name:      "printf",
   229  		FactTypes: []analysis.Fact{new(isWrapper)},
   230  		...
   231  	}
   232  
   233  	type isWrapper struct{} // => *types.Func f “is a printf wrapper”
   234  
   235  The driver program ensures that facts for a pass’s dependencies are
   236  generated before analyzing the package and is responsible for propagating
   237  facts from one package to another, possibly across address spaces.
   238  Consequently, Facts must be serializable. The API requires that drivers
   239  use the gob encoding, an efficient, robust, self-describing binary
   240  protocol. A fact type may implement the GobEncoder/GobDecoder interfaces
   241  if the default encoding is unsuitable. Facts should be stateless.
   242  Because serialized facts may appear within build outputs, the gob encoding
   243  of a fact must be deterministic, to avoid spurious cache misses in
   244  build systems that use content-addressable caches.
   245  The driver makes a single call to the gob encoder for all facts
   246  exported by a given analysis pass, so that the topology of
   247  shared data structures referenced by multiple facts is preserved.
   248  
   249  The Pass type has functions to import and export facts,
   250  associated either with an object or with a package:
   251  
   252  	type Pass struct {
   253  		...
   254  		ExportObjectFact func(types.Object, Fact)
   255  		ImportObjectFact func(types.Object, Fact) bool
   256  
   257  		ExportPackageFact func(fact Fact)
   258  		ImportPackageFact func(*types.Package, Fact) bool
   259  	}
   260  
   261  An Analyzer may only export facts associated with the current package or
   262  its objects, though it may import facts from any package or object that
   263  is an import dependency of the current package.
   264  
   265  Conceptually, ExportObjectFact(obj, fact) inserts fact into a hidden map keyed by
   266  the pair (obj, TypeOf(fact)), and the ImportObjectFact function
   267  retrieves the entry from this map and copies its value into the variable
   268  pointed to by fact. This scheme assumes that the concrete type of fact
   269  is a pointer; this assumption is checked by the Validate function.
   270  See the "printf" analyzer for an example of object facts in action.
   271  
   272  Some driver implementations (such as those based on Bazel and Blaze) do
   273  not currently apply analyzers to packages of the standard library.
   274  Therefore, for best results, analyzer authors should not rely on
   275  analysis facts being available for standard packages.
   276  For example, although the printf checker is capable of deducing during
   277  analysis of the log package that log.Printf is a printf wrapper,
   278  this fact is built in to the analyzer so that it correctly checks
   279  calls to log.Printf even when run in a driver that does not apply
   280  it to standard packages. We would like to remove this limitation in future.
   281  
   282  # Testing an Analyzer
   283  
   284  The analysistest subpackage provides utilities for testing an Analyzer.
   285  In a few lines of code, it is possible to run an analyzer on a package
   286  of testdata files and check that it reported all the expected
   287  diagnostics and facts (and no more). Expectations are expressed using
   288  "// want ..." comments in the input code.
   289  
   290  # Standalone commands
   291  
   292  Analyzers are provided in the form of packages that a driver program is
   293  expected to import. The vet command imports a set of several analyzers,
   294  but users may wish to define their own analysis commands that perform
   295  additional checks. To simplify the task of creating an analysis command,
   296  either for a single analyzer or for a whole suite, we provide the
   297  singlechecker and multichecker subpackages.
   298  
   299  The singlechecker package provides the main function for a command that
   300  runs one analyzer. By convention, each analyzer such as
   301  go/analysis/passes/findcall should be accompanied by a singlechecker-based
   302  command such as go/analysis/passes/findcall/cmd/findcall, defined in its
   303  entirety as:
   304  
   305  	package main
   306  
   307  	import (
   308  		"golang.org/x/tools/go/analysis/passes/findcall"
   309  		"golang.org/x/tools/go/analysis/singlechecker"
   310  	)
   311  
   312  	func main() { singlechecker.Main(findcall.Analyzer) }
   313  
   314  A tool that provides multiple analyzers can use multichecker in a
   315  similar way, giving it the list of Analyzers.
   316  */
   317  package analysis
   318
View as plain text