My Python Rumspringa, or There and Back Again

tl;dr In 2014 I started an unexpected journey into a foreign land fraught with peril.  Now I'm back and have a thing or two to share about both Go and Python.

    The Journey
        Building A Home: My Python History

        A New Adventure: Go
        Coming Home

    The Spoils (What I Brought Back)
        The Good

        The Bad

        The Ugly

    Summary
        What Python Can Learn from Go
    Appendix: Raw Go Observations

(Note: I started writing this post nearly a year ago, not long after I left Juju.)

The Journey

Building A Home: My Python History

In 2006 I started my first programming job, working on a system that migrated accounts between web hosting platforms.  It was written in Python (for which I have Van Lindberg to thank).  Over time, my curious nature led me to delve deep into the language.  Within a couple years I started following the Python mailing lists and bug tracker.  In 2008 I went to my first PyCon, but only attended the talks.

Everything accelerated in 2011.  I was more confident about participating and had even contributed both ideas and feedback.  At PyCon that year I somehow managed to get a seat at the language summit and most nights hung out with the core devs who I knew from the lists but had never met in person.  In a few short days my Python world was shifting.  Most importantly, I stayed for the sprints.

On the first day of the sprints I actively participated in the in-person discussions with Guido et al. that resolved the fate of namespace packages.  There were two competing PEPs.  In the months prior I had been involved in email threads about those PEPs on the import-sig list.  In the discussion at the sprints I was able to have a concrete impact on the decisions being made.  It was amazing!  That marked my serious commitment to the project and the language.

A New Adventure: Go

In 2014, right after PyCon, I started looking for a new job.  Until then I had been lucky enough to have worked in Python full-time.  However, deep down I knew that I needed to expose myself to new things and different perspectives.  In order to keep growing I needed to broaden my view.  In the midst of the decision on which opportunity to pursue, I chose a job at Canonical working on a project called Juju.  Among other new things (to me), I'd be working exclusively in the Go language.

Typically it is best to know what you are getting yourself into.  Consequently I took some time while preparing for interviews to get a feel for Go.  Within a few hours I had a pretty good sense of the language.  To be honest, in 2+ years my initial impression has stayed relatively true:  Go is a fine language with a few things that bug me.  I knew it would be good for me and I could tell it wasn't what I'd want in the long term.  I even promised myself that I'd stay with Juju for at least two years, just in case I started having doubts before I'd truly benefited from the experience.

For two years I enjoyed my work.  Juju is a great project with a unique and effective approach to the hard problem of managing many interoperating services in the cloud.  The folks I worked with were solid.  The work was both challenging and rewarding.  Above all, I got what I was after.  After two years my perspective on writing software had broadened significantly, particularly thanks to Go.  At the same time I was longing for my metaphorical home.

Coming Home

I came to realize something at that point; I was in a figurative long-distance relationship with Python.  At first I would visit often.  I'd write.  I'd call.  However, without even realizing it at first, I'd begun to be less involved.  In fact, after the first year of working in Go I finally noticed that I was contributing less to CPython.  I was on the mailing lists less.  I wasn't on the issue tracker as much.  I'd started to build up a backlog of things I needed to fix.  As hard as it was for me to believe, I was becoming less excited about working on Python!

As with nearly all long-distance relationships, I had to make a decision: to move back or to end the relationship.  It wasn't hard for me.  In fact, I has already been looking into an opportunity with some Python friends (which ultimately didn't pan out).  My two years was almost up.  An unexpected opportunity appeared (which ultimately *did* pan out).  PyCon came around in 2016 and I started putting out feelers.

In the end I chose to get back to Python.  I'm now working on the Landscape team at Canonical.  I'm still in that awkward phase where Python and I are adjusting to being around one another again, but I can feel it all returning.  Was it worth it?  Absolutely!  The experience taught me a lot and I'm glad I did it.

The Spoils (What I Brought Back)

Here are some observations rooted in my experience working full-time in Go for two years. Keep in mind that my work on Juju specifically has heavily influenced my point of view here. Some observations are about Go, some about Python, and some about software in general. Note that I'm not going to talk much about things that aren't more than a different flavor (e.g. curly braces, defer).  At the bottom of this post I've included an appendix that outlines my observations a bit more in depth but without elaboration.

I'll take some time in later blog posts to elaborate on some of the points.  I'll also update this post if I think of any other points or with more detail.

The Good

These are things that I appreciated about Go.

Go's Concurrency Story

• based in CSP
• dedicated syntax (go, select, <- [channel send, receive])
• built-in primitives (goroutines, channels)

Parts of Go's Type System

• interfaces (i.e. declarative duck typing)
• composition instead of inheritance
• embedding
• explicit collision resolution
• reverse template pattern
• structs
• fixed set of fields
• methods
• slices
• (sort of) first-class functions
• function param structs (in lieu of keyword arguments)

"Anonymous" Type Support

• in-line types and functions (i.e. literals)
• multi-line function literals
• anonymous blocks (scoping)

Some of Go's Syntax

• type-after-name variable declarations
• switch, type switch
• struct field tags
• multiple return values (vs. C)
• "ok" returns from maps, etc.

Tooling

• go * (esp. fmt)

The Bad

These are things that I wished were different/better, but could live with.  Mostly these actually either aren't that bad or aren't a big deal in practice.  Some of these may improve in time.

Parts of Go's Type System

• type incompatibility
• converting []interface{} to []Spam
• compatible signatures
• the need for compatibility shims
• pointers
• builtin types vs. user-defined types
• types are not first-class
• packages are not first-class
• no generics (yet?)
• no enums
• no "magic" methods
• inference (weak for a "modern" language)
• nil, interface{}
• nil vs. underlying nil
• namespaces
• exported vs. unexported (incl. internal packages)
• function comparison
• no keyword arguments
• no function argument (or struct field) defaults
• no anonymous tuples

Parts of Go's Memory Model

• nil
• immutability
• value ownership

Some of Go's Syntax

• methods harder to visually associate with struct

Tooling

• go * (esp. fmt)
• dependency versioning

Standard Library

• stdlib hard to customize

Philosophy

• minimalist (reminds me a little of Lua)
• few stdlib helpers in most areas
• designed to address Google's problems, not necessarily everyone else's
• trying hard to be a systems programming language (feels like the 70s)

The Ugly

These are things that constantly either bothered me or significantly impacted my productivity.

Multi-file Packages

• makes it a pain to figure out which source file a package member is in
• too easy to split up a struct's methods across multiple files

Error Returns (vs. Exceptions)

• a lot has been said about this already elsewhere on the internet -- I'm on the side of exceptions (the Python kind anyway)
• a step up from C, but that's about it
• introduce a ton of superfluous boilerplate (if you're using them properly)

Boilerplate

• no generics
• error returns
• type/signature compatibility shims

Testing

• full-stack
• white-box testing
• immature

Summary

My experience on Juju was a valuable one.  I'm glad I did it.  Go is a fine language, but not one I'd want to use full-time, long-term.  At one point I built a spreadsheet comparing Python, Go, and Rust.  Perhaps I'll write up a post about that comparison some day.

What Python Can Learn from Go

Everything above surely betrays my bias toward Python.  That said, I think there are quite a few lessons Python can take from Go.  That includes both how Python can improve and how Python gets many things right.  Most notably, Go's concurrency story is a step up from Python's, which I'm working to improve.  Here are a few other things I'd like to see in Python:

• anonymous code blocks
• multi-line anonymous functions
• a switch statement
• a clear, simple approach to declare simple classes
• namedtuple and types.SimpleNamespace are a weaker alternative
• class declaration syntax is already so overloaded in roles it plays
• make class composition a more obvious option

Appendix: Raw Go Observations

Type System

• statically-typed (multi-pass)
• compile-time resolution
• types are not first-class (effectively compiler instructions)
• all data mutable
• exported vs. unexported
• user-defined types
• functions
• sort of first class
• are pointer types
• can only be compared with nil
• structs
• sometimes used to hold keyword args
• default serialization
• sometimes used as "data transfer object" or "value object"
• equality unsupported in some cases (has function/map fields)
• copy vs. deepcopy
• methods
• defined separately in same package (usually same file) as struct
• harder to distinguish when reading code
• receiver type matches struct (incl. at run-time)
• receiver is passed to local scope as implicit call argument
• may be called from type, but receiver is passed explicitly
• value receivers are copies
• pointer receivers are required for mutation
• struct initializers
• struct literals
• fields default to zero values
• interfaces
• nil vs. underlying nil value
• methods-only
• supports polymorphism
• statically-typed ducktyping
• everything matches the zero interface ("interface{}")
• use type assertion to get underlying value
• aliasing
• unlike embedding, an aliased type loses access to methods
• (generics)
• lack of generics leads to rampant duplication of code
• may happen eventually
• (enums)
• not part of the language
• rough equivalent through type alias (or embedding)
• easy to accidentally expose aliased/embedded type
• supported value range not enforced by type system
• explicit run-time checking required
• (namespaces)
• no simple way to declare a namespace
• embedding (incl. -> composition)
• structs may embed interfaces structs and interfaces
• interfaces may embed only interfaces
• refer to embedded type as attribute with that name
• multiple embedding supported
• manually resolve calling embedded methods when overridden
• builtin types
• numeric (int, float)
• string/[]byte, rune
• containers (map, slice, array, channel)
• are pointer types
• equality unsupported for maps
• nil and empty not the same, though len() is
• no set type
• slice vs. array
• almost never use array
• slices auto-resize up to max
• type-related builtin functions
• slice-only
• append()
• may or may not return a new slice
• copy()
• channel-only
• close()
• map-only
• delete()
• builtin-container-only
• len()
• make()
• cap()
• new()
• value vs. pointer
• nil pointers
• auto-dereferencing in function calls
• type compatibility
• matching struct values may be used for interface variables
• types in signatures must match exactly for functions to match
• ...even if compatible
• applies to function types and interfaces
• this strict compatibility forces use of adapter types
• methods may be used for vars of a matching function type/signature
• no implicit coercion (except for type alias args and interface vars)
• explicit coercion through "type conversion" (i.e. type casting)
• inference
• only in function bodies
• only through RHS of :=
• signature/struct type declaration
• exact types for fields/params

Concurrency

• CSP-inspired model
• goroutines
• created by "go func()" syntax
• not directly exposed as values in run-time
• channels
• select
• simplifies concurrency but inherent challenges remain
• deadlocks
• leaks
• still shared-memory (no "process"-isolation like CSP)
• locks and other tools in stdlib
• no direct thread management (concept is hidden away)
• multi-processing
• no process forking
• exec.Command for starting a new process
• panic() and recover()

Memory Model

• nil
• leads to segfaults
• new vs. make
• new() returns a pointer to a newly allocated value of any type
• make() for builtin container types only
• mutability (incl. default)
• all data is mutable (no const, etc.)
• hide data access by not exporting it
• auto garbage collection
• no constructors/finalizers
• auto-promotion from stack to heap at compile-time

Syntax

• declarations: types, functions, vars, consts
• may be in any scope, not just top-level
• all but const may be in function signature
• var declaration: type after name
• easier to read/write than C-style (type before name)
• switch
• implicit break at end of each case
• type switch
• sugar for a switch with a bunch of type assertions
• no semi-colons
• multi-line statements (e.g. signatures)
• curly braces
• struct field tags
• multi-line function literals (i.e. lambdas)
• may be in-lined
• in-lined struct/interface definitions
• const only for simple literals (e.g. string, numeric)
• compiler instructions vs. run-time code

Standard Library

• fairly well populated
• often hard to customize

Testing

• still a mess

Other

• block scope
• :( storing a pointer to the loop variable each iteration
• defer
• error returns/(exceptions)
• optional "ok" return for maps, channels, coercion
• encourages copy/paste-then-adjust
• multi-file packages
• internal packages
• per-file init()