Text file talks/2016/refactor.article

     1  Codebase Refactoring (with help from Go)
     2  
     3  Russ Cox
     4  rsc@golang.org
     5  
     6  * Abstract
     7  
     8  Go should add the ability to create alternate equivalent names for types,
     9  in order to enable gradual code repair during codebase refactoring.
    10  This article explains the need for that ability and the implications of not having it
    11  for today’s large Go codebases.
    12  This article also examines some potential solutions,
    13  including the alias feature proposed during the development of
    14  (but not included in) Go 1.8.
    15  However, this article is _not_ a proposal of any specific solution.
    16  Instead, it is intended as the start of a discussion by the Go community
    17  about what solution should be included in Go 1.9.
    18  
    19  This article is an extended version of a talk given at
    20  GothamGo in New York on November 18, 2016.
    21  
    22  * Introduction
    23  
    24  Go’s goal is to make it easy to build software that scales.
    25  There are two kinds of scale that we care about.
    26  One kind of scale is the size of the systems that you can build with Go,
    27  meaning how easy it is to use large numbers of computers,
    28  process large amounts of data, and so on.
    29  That’s an important focus for Go but not for this article.
    30  Instead, this article focuses on another kind of scale,
    31  the size of Go programs,
    32  meaning how easy it is to work in large codebases
    33  with large numbers of engineers
    34  making large numbers of changes independently.
    35  
    36  One such codebase is
    37  [[http://m.cacm.acm.org/magazines/2016/7/204032-why-google-stores-billions-of-lines-of-code-in-a-single-repository/pdf][Google’s single repository]]
    38  that nearly all engineers work in on a daily basis.
    39  As of January 2015,
    40  that repository was seeing 40,000 commits per day
    41  across 9 million source files
    42  and 2 billion lines of code.
    43  Of course, there is more in the repository than just Go code.
    44  
    45  Another large codebase is the set of all the open source Go code
    46  that people have made available on GitHub
    47  and other code hosting sites.
    48  You might think of this as `go` `get`’s codebase.
    49  In contrast to Google’s codebase,
    50  `go` `get`’s codebase is completely decentralized,
    51  so it’s more difficult to get exact numbers.
    52  In November 2016, there were 140,000 packages known to godoc.org
    53  and over 160,000
    54  [[https://github.com/search?utf8=%E2%9C%93&q=language%3AGo&type=Repositories&ref=searchresults][GitHub repos written in Go]].
    55  
    56  Supporting software development at this scale was in our
    57  minds from the very beginning of Go.
    58  We paid a lot of attention to implementing imports efficiently.
    59  We made sure that it was difficult to import code but forget to use it, to avoid code bloat.
    60  We made sure that there weren’t unnecessary dependencies
    61  between packages, both to simplify programs and to make it
    62  easier to test and refactor them.
    63  For more detail about these considerations, see Rob Pike’s 2012 article
    64  “[[/talks/2012/splash.article][Go at Google: Language Design in the Service of Software Engineering]].”
    65  
    66  Over the past few years we’ve come to realize that there’s
    67  more that can and should be done to make it easier
    68  to refactor whole codebases,
    69  especially at the broad package structure level,
    70  to help Go scale to ever-larger programs.
    71  
    72  * Codebase refactoring
    73  
    74  Most programs start with one package.
    75  As you add code, occasionally you recognize
    76  a coherent section of code that could stand on its own,
    77  so you move that section into its own package.
    78  Codebase refactoring is the process of rethinking
    79  and revising decisions about both the grouping of code
    80  into packages and the relationships between those packages.
    81  There are a few reasons you might want to change the way
    82  a codebase is organized into packages.
    83  
    84  The first reason is to split a package into more manageable pieces for users.
    85  For example, most users of [[/pkg/regexp/][package regexp]] don’t need access to the
    86  regular expression parser, although [[https://pkg.go.dev/github.com/google/codesearch/regexp][advanced uses may]],
    87  so the parser is exported in [[/pkg/regexp/syntax][a separate regexp/syntax package]].
    88  
    89  The second reason is to [[/blog/package-names][improve naming]].
    90  For example, early versions of Go had an `io.ByteBuffer`,
    91  but we decided `bytes.Buffer` was a better name and package bytes a better place for the code.
    92  
    93  The third reason is to lighten dependencies.
    94  For example, we moved `os.EOF` to `io.EOF` so that code not using the operating system
    95  can avoid importing the fairly heavyweight [[/pkg/os][package os]].
    96  
    97  The fourth reason is to change the dependency graph
    98  so that one package can import another.
    99  For example, as part of the preparation for Go 1, we looked at the explicit dependencies
   100  between packages and how they constrained the APIs.
   101  Then we changed the dependency graph to make the APIs better.
   102  
   103  Before Go 1, the `os.FileInfo` struct contained these fields:
   104  
   105  	type FileInfo struct {
   106  		Dev      uint64 // device number
   107  		Ino      uint64 // inode number
   108  		...
   109  		Atime_ns int64  // access time; ns since epoch
   110  		Mtime_ns int64  // modified time; ns since epoch
   111  		Ctime_ns int64  // change time; ns since epoch
   112  		Name     string // name of file
   113  	}
   114  
   115  Notice the times `Atime_ns`, `Mtime_ns`, `Ctime_ns` have type int64,
   116  an `_ns` suffix, and are commented as “nanoseconds since epoch.”
   117  These fields would clearly be nicer using [[/pkg/time/#Time][`time.Time`]],
   118  but mistakes in the design of the package structure of the codebase
   119  prevented that.
   120  To be able to use `time.Time` here, we refactored the codebase.
   121  
   122  This graph shows eight packages from the standard library
   123  before Go 1, with an arrow from P to Q indicating that P imports Q.
   124  
   125  .html refactor/import1.html
   126  
   127  Nearly every package has to consider errors,
   128  so nearly every package, including package time, imported package os for `os.Error`.
   129  To avoid cycles, anything that imports package os cannot itself be used by package os.
   130  As a result, operating system APIs could not use `time.Time`.
   131  
   132  This kind of problem convinced us that
   133  `os.Error` and its constructor `os.NewError` were so fundamental
   134  that they should be moved out of package os.
   135  In the end, we moved `os.Error` into the language as [[/ref/spec/#Errors][`error`]]
   136  and `os.NewError` into the new
   137  [[/pkg/errors][package errors]]
   138  as `errors.New`.
   139  After this and other refactoring, the import graph in Go 1 looked like:
   140  
   141  .html refactor/import2.html
   142  
   143  Package io and package time had few enough dependencies
   144  to be used by package os, and
   145  the Go 1 definition of [[/pkg/os/#FileInfo][`os.FileInfo`]] does use `time.Time`.
   146  
   147  (As a side note, our first idea was to move `os.Error` and `os.NewError`
   148  to a new package named error (singular) as `error.Value` and `error.New`.
   149  Feedback from Roger Peppe and others in the Go community helped us
   150  see that making the error type predefined in the language would
   151  allow its use even in low-level contexts like the specification of
   152  [[/ref/spec#Run_time_panics][run-time panics]].
   153  Since the type was named `error`, the package became errors (plural)
   154  and the constructor `errors.New`.
   155  Andrew Gerrand’s 2015 talk
   156  “[[/talks/2015/how-go-was-made.slide#37][How Go was Made]]” has more detail.)
   157  
   158  * Gradual code repair
   159  
   160  The benefits of a codebase refactoring apply throughout the codebase.
   161  Unfortunately, so do the costs:
   162  often a large number of repairs must be made as a result of the refactoring.
   163  As codebases grow, it becomes infeasible to do all the repairs at one time.
   164  The repairs must be done gradually,
   165  and the programming language must make that possible.
   166  
   167  As a simple example,
   168  when we moved `io.ByteBuffer` to `bytes.Buffer` in 2009, the [[https://go.googlesource.com/go/+/d3a412a5abf1ee8815b2e70a18ee092154af7672][initial commit]]
   169  moved two files, adjusted three makefiles, and repaired 43 other Go source files.
   170  The repairs outweighed the actual API change by a factor of twenty,
   171  and the entire codebase was only 250 files.
   172  As codebases grow, so does the repair multiplier.
   173  Similar changes in large Go codebases,
   174  such as Docker, and Juju, and Kubernetes,
   175  can have repair multipliers ranging from 10X to 100X.
   176  Inside Google we’ve seen repair multipliers well over 1000X.
   177  
   178  The conventional wisdom is that when making a codebase-wide API change,
   179  the API change and the associated code repairs should be committed
   180  together in one big commit:
   181  
   182  .html refactor/atomic.html
   183  
   184  The argument in favor of this approach,
   185  which we will call “atomic code repair,”
   186  is that it is conceptually simple:
   187  by updating the API and the code repairs in the same commit,
   188  the codebase transitions in one step from the old API to the new API,
   189  without ever breaking the codebase.
   190  The atomic step avoids the need to plan for a transition
   191  during which both old and new API must coexist.
   192  In large codebases, however, the conceptual simplicity
   193  is quickly outweighed by a practical complexity:
   194  the one big commit can be very big.
   195  Big commits are hard to prepare, hard to review,
   196  and are fundamentally racing against other work in the tree.
   197  It’s easy to start doing a conversion, prepare your one big commit,
   198  finally get it submitted, and only then find out that another developer added
   199  a use of the old API while you were working.
   200  There were no merge conflicts,
   201  so you missed that use, and despite all your effort
   202  the one big commit broke the codebase.
   203  As codebases get larger,
   204  atomic code repairs become more difficult
   205  and more likely to break the codebase inadvertently.
   206  
   207  In our experience,
   208  an approach that scales better is to plan for a transition period
   209  during which the code repair proceeds gradually,
   210  across as many commits as needed:
   211  
   212  .html refactor/gradual.html
   213  
   214  Typically this means the overall process runs in three stages.
   215  First, introduce the new API.
   216  The old and new API must be _interchangeable_,
   217  meaning that it must be possible to convert individual uses
   218  from the old to the new API without changing the overall
   219  behavior of the program,
   220  and uses of the old and new APIs must be able to coexist
   221  in a single program.
   222  Second, across as many commits as you need,
   223  convert all the uses of the old API to the new API.
   224  Third, remove the old API.
   225  
   226  “Gradual code repair” is usually more work
   227  than the atomic code repair,
   228  but the work itself is easier:
   229  you don’t have to get everything right in one try.
   230  Also, the individual commits are much smaller,
   231  making them easier to review and submit
   232  and, if needed, roll back.
   233  Maybe most important of all, a gradual code repair
   234  works in situations when one big commit would be impossible,
   235  for example when code that needs repairs
   236  is spread across multiple repositories.
   237  
   238  The `bytes.Buffer` change looks like an atomic code repair, but it wasn’t.
   239  Even though the commit updated 43 source files,
   240  the commit message says,
   241  “left io.ByteBuffer stub around for now, for protocol compiler.”
   242  That stub was in a new file named `io/xxx.go` that read:
   243  
   244  	// This file defines the type io.ByteBuffer
   245  	// so that the protocol compiler's output
   246  	// still works. Once the protocol compiler
   247  	// gets fixed, this goes away.
   248  
   249  	package io
   250  
   251  	import "bytes"
   252  
   253  	type ByteBuffer struct {
   254  		bytes.Buffer;
   255  	}
   256  
   257  Back then, just like today,
   258  Go was developed in a separate source repository
   259  from the rest of Google’s source code.
   260  The protocol compiler in Google’s main repository was
   261  responsible for generating Go source files from protocol buffer definitions;
   262  the generated code used `io.ByteBuffer`.
   263  This stub was enough to keep the generated code working
   264  until the protocol compiler could be updated.
   265  Then [[https://go.googlesource.com/go/+/832e72beff62e4fe4897699e9b40a2b228e8503b][a later commit]] removed `xxx.go`.
   266  
   267  Even though there were many fixes included in the original commit,
   268  this change was still a gradual code repair, not an atomic one,
   269  because the old API was only removed in a separate stage
   270  after the existing code was converted.
   271  
   272  In this specific case the gradual repair did succeed, but
   273  the old and new API were not completely interchangeable:
   274  if there had been a function taking an `*io.ByteBuffer` argument
   275  and code calling that function with an `*io.ByteBuffer`,
   276  those two pieces of code could not have been updated independently:
   277  code that passed an `*io.ByteBuffer` to a function expecting a `*bytes.Buffer`,
   278  or vice versa, would not compile.
   279  
   280  Again, a gradual code repair consists of three stages:
   281  
   282  .html refactor/template.html
   283  
   284  These stages apply to a gradual code repair for any API change.
   285  In the specific case of codebase refactoring—moving
   286  an API from one package to another, changing its full name in the process—making the old and new API
   287  interchangeable means making the old and new names interchangeable,
   288  so that code using the old name has exactly the same behavior
   289  as if it used the new name.
   290  
   291  Let’s look at examples of how Go makes that possible (or not).
   292  
   293  ** Constants
   294  
   295  Let’s start with a simple example of moving a constant.
   296  
   297  Package io defines the [[/pkg/io/#Seeker][Seeker interface]],
   298  but the named constants that developers prefer to use
   299  when invoking the `Seek` method came from package os.
   300  Go 1.7 moved the constants to package io and gave them more idiomatic names;
   301  for example, `os.SEEK_SET` is now available as `io.SeekStart`.
   302  
   303  For a constant, one name is interchangeable with another
   304  when the definitions use the same type and value:
   305  
   306  	package io
   307  	const SeekStart int = 0
   308  
   309  	package os
   310  	const SEEK_SET int = 0
   311  
   312  Due to [[/doc/go1compat][Go 1 compatibility]],
   313  we’re blocked in stage 2 of this gradual code change.
   314  We can’t delete the old constants,
   315  but making the new ones available in package io allows
   316  developers to avoid importing package os in code that
   317  does not actually depend on operating system functionality.
   318  
   319  This is also an example of a gradual code repair being done
   320  across many repositories.
   321  Go 1.7 introduced the new API,
   322  and now it’s up to everyone with Go code to update their code
   323  as they see fit.
   324  There’s no rush, no forced breakage of existing code.
   325  
   326  ** Functions
   327  
   328  Now let’s look at moving a function from one package to another.
   329  
   330  As mentioned above,
   331  in 2011 we replaced `os.Error` with the predefined type `error`
   332  and moved the constructor `os.NewError` to a new package as
   333  [[/pkg/errors/#New][`errors.New`]].
   334  
   335  For a function, one name is interchangeable with another
   336  when the definitions use the same signature and implementation.
   337  In this case, we can define the old function as a wrapper calling
   338  the new function:
   339  
   340  	package errors
   341  	func New(msg string) error { ... }
   342  
   343  	package os
   344  	func NewError(msg string) os.Error {
   345  	    return errors.New(msg)
   346  	}
   347  
   348  Since Go does not allow comparing functions for equality,
   349  there is no way to tell these two functions apart.
   350  The old and new API are interchangeable,
   351  so we can proceed to stages 2 and 3.
   352  
   353  (We are ignoring a small detail here: the original
   354  `os.NewError` returned an `os.Error`, not an `error`,
   355  and two functions with different signatures _are_ distinguishable.
   356  To really make these functions indistinguishable,
   357  we would also need to make `os.Error` and `error` indistinguishable.
   358  We will return to that detail in the discussion of types below.)
   359  
   360  ** Variables
   361  
   362  Now let’s look at moving a variable from one package to another.
   363  
   364  We are discussing exported package-level API, so the variable
   365  in question must be an exported global variable.
   366  Such variables are almost always set at init time
   367  and then only intended to be read from, never written again,
   368  to avoid races between reading and writing goroutines.
   369  For exported global variables that follow this pattern,
   370  one name is nearly interchangeable with another when the two have
   371  the same type and value.
   372  The simplest way to arrange that is to initialize one from the other:
   373  
   374  	package io
   375  	var EOF = ...
   376  
   377  	package os
   378  	var EOF = io.EOF
   379  
   380  In this example, io.EOF and os.EOF are the same value.
   381  The variable values are completely interchangeable.
   382  
   383  There is one small problem.
   384  Although the variable values are interchangeable,
   385  the variable addresses are not.
   386  In this example, `&io.EOF` and `&os.EOF` are different pointers.
   387  However, it is rare to export a read-only variable
   388  from a package and expect clients to take its address:
   389  it would be better for clients if the package exported a variable set to the address instead,
   390  and then the pattern works.
   391  
   392  ** Types
   393  
   394  Finally let’s look at moving a type from one package to another.
   395  This is much harder to do in Go today, as the following three examples demonstrate.
   396  
   397  *** Go’s os.Error
   398  
   399  Consider once more the conversion from `os.Error` to `error`.
   400  There’s no way in Go to make two names of types interchangeable.
   401  The closest we can come in Go is to give `os.Error` and `error` the same underlying definition:
   402  
   403  	package os
   404  	type Error error
   405  
   406  Even with this definition, and even though these are interface types,
   407  Go still considers these two types [[/ref/spec#Type_identity][different]],
   408  so that a function returning an os.Error
   409  is not the same as a function returning an error.
   410  Consider the [[/pkg/io/#Reader][`io.Reader`]] interface:
   411  
   412  	package io
   413  	type Reader interface {
   414  	    Read(b []byte) (n int, err error)
   415  	}
   416  
   417  If `io.Reader` is defined using `error`, as above, then a `Read` method
   418  returning `os.Error` will not satisfy the interface.
   419  
   420  If there’s no way to make two names for a type interchangeable,
   421  that raises two questions.
   422  First, how do we enable a gradual code repair for a moved or renamed type?
   423  Second, what did we do for `os.Error` in 2011?
   424  
   425  To answer the second question, we can look at the source control history.
   426  It turns out that to aid the conversion, we
   427  [[https://go.googlesource.com/go/+/47f4bf763dcb120d3b005974fec848eefe0858f0][added a temporary hack to the compiler]]
   428  to make code written using `os.Error` be interpreted as if it had written `error` instead.
   429  
   430  *** Kubernetes
   431  
   432  This problem with moving types is not limited to fundamental changes like `os.Error`,
   433  nor is it limited to the Go repository.
   434  Here’s a change from the [[https://kubernetes.io/][Kubernetes project]].
   435  Kubernetes has a package util, and at some point the developers
   436  decided to split out that package’s `IntOrString` type into its own
   437  [[https://pkg.go.dev/k8s.io/kubernetes/pkg/util/intstr][package intstr]].
   438  
   439  Applying the pattern for a gradual code repair,
   440  the first stage is to establish a way for the two types to be interchangeable.
   441  We can’t do that,
   442  because the `IntOrString` type is used in struct fields,
   443  and code can’t assign to that field unless the value being
   444  assigned has the correct type:
   445  
   446  	package util
   447  	type IntOrString intstr.IntOrString
   448  
   449  	// Not good enough for:
   450  
   451  	// IngressBackend describes ...
   452  	type IngressBackend struct {
   453  	    ServiceName string             `json:"serviceName"`
   454  	    ServicePort intstr.IntOrString `json:"servicePort"`
   455  	}
   456  
   457  If this use were the only problem, then you could imagine
   458  writing a getter and setter using the old type
   459  and doing a gradual code repair to change all existing code
   460  to use the getter and setter,
   461  then modifying the field to use the new type
   462  and doing a gradual code repair to change all existing code
   463  to access the field directly using the new type,
   464  then finally deleting the getter and setter that mention the old type.
   465  That required two gradual code repairs instead of one,
   466  and there are many uses of the type other than this one struct field.
   467  
   468  In practice, the only option here is an atomic code repair,
   469  or else breaking all code using `IntOrString`.
   470  
   471  *** Docker
   472  
   473  As another example,
   474  here’s a change from the [[https://www.docker.com/][Docker project]].
   475  Docker has a package utils, and at some point the developers
   476  decided to split out that package’s `JSONError` type into a separate
   477  [[https://pkg.go.dev/github.com/docker/docker/pkg/jsonmessage#JSONError][jsonmessage package]].
   478  
   479  Again we have the problem that the old and new types are not interchangeable,
   480  but it shows up in a different way, namely [[/ref/spec#Type_assertions][type assertions]]:
   481  
   482  	package utils
   483  	type JSONError jsonmessage.JSONError
   484  
   485  	// Not good enough for:
   486  
   487  	jsonError, ok := err.(*jsonmessage.JSONError)
   488  	if !ok {
   489  		jsonError = &jsonmessage.JSONError{
   490  			Message: err.Error(),
   491  		}
   492  	}
   493  
   494  If the error `err` not already a `JSONError`, this code wraps it in one,
   495  but during a gradual repair, this code handles `utils.JSONError` and `jsonmessage.JSONError` differently.
   496  The two types are not interchangeable.
   497  (A [[/ref/spec#Type_switches][type switch]] would expose the same problem.)
   498  
   499  If this line were the only problem, then you could imagine
   500  adding a type assertion for `*utils.JSONError`,
   501  then doing a gradual code repair to remove other uses of `utils.JSONError`,
   502  and finally removing the additional type guard just before removing the old type.
   503  But this line is not the only problem.
   504  The type is also used elsewhere in the API and has all the
   505  problems of the Kubernetes example.
   506  
   507  In practice, again the only option here is an atomic code repair
   508  or else breaking all code using `JSONError`.
   509  
   510  * Solutions?
   511  
   512  We’ve now seen examples of how we can and cannot move
   513  constants, functions, variables, and types from one package to another.
   514  The patterns for establishing interchangeable old and new API are:
   515  
   516  	const OldAPI = NewPackage.API
   517  
   518  	func OldAPI() { NewPackage.API() }
   519  
   520  	var OldAPI = NewPackage.API
   521  
   522  	type OldAPI ... ??? modify compiler or ... ???
   523  
   524  For constants and functions, the setup for a gradual code repair is trivial.
   525  For variables, the trivial setup is incomplete but only in ways that are not likely to arise often in practice.
   526  
   527  For types, there is no way to set up a gradual code repair in essentially any real example.
   528  The most common option is to force an atomic code repair,
   529  or else to break all code using the moved type and leave clients
   530  to fix their code at the next update.
   531  In the case of moving os.Error, we resorted to modifying the compiler.
   532  None of these options is reasonable.
   533  Developers should be able to do refactorings
   534  that involve moving a type from one package to another
   535  without needing an atomic code repair,
   536  without resorting to intermediate code and multiple rounds of repair,
   537  without forcing all client packages to update their own code immediately,
   538  and without even thinking about modifying the compiler.
   539  
   540  But how? What should these refactorings look like tomorrow?
   541  
   542  We don’t know.
   543  The goal of this article is to define the problem well enough
   544  to discuss the possible answers.
   545  
   546  ** Aliases
   547  
   548  As explained above, the fundamental problem with moving types is that
   549  while Go provides ways to create an alternate name
   550  for a constant or a function or (most of the time) a variable,
   551  there is no way to create an alternate name for a type.
   552  
   553  For Go 1.8 we experimented with introducing first-class support
   554  for these alternate names, called [[/design/16339-alias-decls][_aliases_]].
   555  A new declaration syntax, the alias form, would have provided a uniform way
   556  to create an alternate name for any kind of identifier:
   557  
   558  	const OldAPI => NewPackage.API
   559  	func  OldAPI => NewPackage.API
   560  	var   OldAPI => NewPackage.API
   561  	type  OldAPI => NewPackage.API
   562  
   563  Instead of four different mechanisms, the refactoring of package os we considered above
   564  would have used a single mechanism:
   565  
   566  	package os
   567  	const SEEK_SET => io.SeekStart
   568  	func  NewError => errors.New
   569  	var   EOF      => io.EOF
   570  	type  Error    => error
   571  
   572  During the Go 1.8 release freeze, we found two small but important unresolved technical details
   573  in the alias support (issues [[/issue/17746][17746]] and [[/issue/17784][17784]]),
   574  and we decided that it was not possible to resolve them confidently
   575  in the time remaining before the Go 1.8 release,
   576  so we held aliases back from Go 1.8.
   577  
   578  ** Versioning
   579  
   580  An obvious question is whether to rely on versioning and
   581  dependency management for code repair,
   582  instead of focusing on strategies that enable gradual code repair.
   583  
   584  Versioning and gradual code repair strategies are complementary.
   585  A versioning system’s job is to identify a compatible set of
   586  versions of all the packages needed in a program, or else to
   587  explain why no such set can be constructed.
   588  Gradual code repair creates additional compatible combinations,
   589  making it more likely that a versioning system can find a way
   590  to build a particular program.
   591  
   592  Consider again the various updates to Go’s standard library
   593  that we discussed above.
   594  Suppose that the old API
   595  corresponded in a versioning system
   596  to standard library version 5.1.3.
   597  In the usual atomic code repair approach,
   598  the new API would be introduced and the old API removed at the same time,
   599  resulting in version 6.0.0;
   600  following [[http://semver.org/][semantic versioning]],
   601  the major version number is incremented to indicate the incompatibility
   602  caused by removing the old API.
   603  
   604  Now suppose that your larger program depends on two packages, Foo and Bar.
   605  Foo still uses the old standard library API.
   606  Bar has been updated to use the new standard library API,
   607  and there have been important changes since then that your
   608  program needs: you can’t use an older version of Bar from
   609  before the standard library changes.
   610  
   611  .html refactor/version1.html
   612  
   613  There is no compatible set of libraries to build your program:
   614  you want the latest version of Bar, which requires
   615  standard library 6.0.0,
   616  but you also need Foo, which is incompatible with standard library 6.0.0.
   617  The best a versioning system can do in this case is report the failure clearly.
   618  (If you are sufficiently motivated, you might then resort to updating your own copy of Foo.)
   619  
   620  In contrast, with better support for gradual code repair,
   621  we can add the new, interchangeable API in version 5.2.0,
   622  and then remove the old API in version 6.0.0.
   623  
   624  .html refactor/version2.html
   625  
   626  The intermediate version 5.2.0 is backwards compatible with 5.1.3,
   627  indicated by the shared major version number 5.
   628  However, because the change from 5.2.0 to 6.0.0 only removed API,
   629  5.2.0 is also, perhaps surprisingly, backwards compatible with 6.0.0.
   630  Assuming that Bar declares its requirements precisely—it is
   631  compatible with both 5.2.0 and 6.0.0—a version system can see that
   632  both Foo and Bar are compatible with 5.2.0 and use that version
   633  of the standard library to build the program.
   634  
   635  Good support for and adoption of gradual code repair reduces incompatibility,
   636  giving versioning systems a better chance to find a way to build your program.
   637  
   638  ** Type aliases
   639  
   640  To enable gradual code repair during codebase refactorings,
   641  it must be possible to create alternate names for a
   642  constant, function, variable, or type.
   643  Go already allows introducing alternate names for
   644  all constants, all functions, and nearly all variables, but no types.
   645  Put another way,
   646  the general alias form is never necessary for constants,
   647  never necessary for functions,
   648  only rarely necessary for variables,
   649  but always necessary for types.
   650  
   651  The relative importance to the specific declarations
   652  suggests that perhaps the Go 1.8 aliases were an overgeneralization,
   653  and that we should instead focus on a solution limited to types.
   654  The obvious solution is type-only aliases,
   655  for which no new operator is required.
   656  Following
   657  [[http://www.freepascal.org/docs-html/ref/refse19.html][Pascal]]
   658  (or, if you prefer, [[https://doc.rust-lang.org/book/type-aliases.html][Rust]]),
   659  a Go program could introduce a type alias using the assignment operator:
   660  
   661  	type OldAPI = NewPackage.API
   662  
   663  The idea of limiting aliases to types was
   664  [[/issue/16339#issuecomment-233644777][raised during the Go 1.8 alias discussion]],
   665  but it seemed worth trying the more general approach, which we did, unsuccessfully.
   666  In retrospect, the fact that `=` and `=>` have identical meanings for constants
   667  while they have nearly identical but subtly different meanings for variables
   668  suggests that the general approach is not worth its complications.
   669  
   670  In fact, the idea of adding Pascal-style type aliases
   671  was [[/issue/16339#issuecomment-233759255][considered in the early design of Go]],
   672  but until now we didn’t have a strong use case for them.
   673  
   674  Type aliases seem like a promising approach to explore,
   675  but, at least to me, generalized aliases seemed equally promising
   676  before the discussion and experimentation during the Go 1.8 cycle.
   677  Rather than prejudge the outcome, the goal of this article is to
   678  explain the problem in detail and examine a few possible solutions,
   679  to enable a productive discussion and evaluation of ideas for next time.
   680  
   681  * Challenge
   682  
   683  Go aims to be ideal for large codebases.
   684  
   685  In large codebases, it’s important to be able to refactor codebase structure,
   686  which means moving APIs between packages and updating client code.
   687  
   688  In such large refactorings, it’s important to be able to use a gradual transition from the old API to the new API.
   689  
   690  Go does not support the specific case of gradual code repair when moving types between packages at all. It should.
   691  
   692  I hope we the Go community can fix this together in Go 1.9. Maybe type aliases are a good starting point. Maybe not. Time will tell.
   693  
   694  * Acknowledgements
   695  
   696  Thanks to the many people who helped us [[/issue/16339][think through the design questions]]
   697  that got us this far and led to the alias trial during Go 1.8 development.
   698  I look forward to the Go community helping us again when we revisit this problem for Go 1.9.
   699  If you’d like to contribute, please see [[/issue/18130][issue 18130]].
   700  

View as plain text