Thursday, December 19, 2013

Goals for Python 3.5

Python 3.4 is in its first beta and the release is just around the corner.  Hurray!  That means we are in feature freeze.  While I am finishing up odds and ends for the PEP 451 (module specs) implementation, I'm also starting to think seriously about what I hope to accomplish for the next Python version, 3.5.

The following are lists of things I'd like to make happen for Python 3.5.  Most of them are things I wanted to do for 3.4 but ran out of time (go figure).  I'll be adding to this list as I run across things I missed.  I'm sure I've missed a thing or three that I promised myself or someone else that I'd work on.

Inspired in part by Nick Coghlan, I've categorized this 3.5 wishlist in descending order of likelihood.

[WARNING: Realistically, a bunch of this will not happen.]

Must-haves


These things are my main priorities.  Barring rejection in python-dev, I will work to push these through.

1. a C implementation of OrderedDict (basically done)
2. OrderedDict as the default class definition namespace [guido]
3. a __definition_order__ attribute on classes that preserves the order of keys (or at least the keys) [me]
4. a __kwargs_order__ attribute in function locals that preserves the order of keys from **kwargs in calls (in lieu of making kwargs an OrderedDict) [link]
5. a new type that abstracts the full import system (similar to PEP 406)
    - incl. exposing suffix->loader mapping
    - context manager support
6. add statement local namespaces [PEP 3150][PEP 403][PEP 340][PEP 359][Nick]

#1-4 are all related, though 4 probably won't involve OrderedDict when all is said and done.  They shouldn't require all that much work to take across the finish line.

Helping Out


These are things that I expect others will be targeting for 3.5 and with which I plan on helping.

* refactor pythonrun.c [PEP 432]
* refactor the C extension module API to take advantage of the PEP 451 loader APIs [import-sig link][issue16421][issue13429][issue16392?]
    - incl. implement exec_module() for BuiltinImporter and ExtensionFileLoader
* a successor to PEP 395
* "speed up" interpreter start-up, import [issue16101][link][link]
* [subinterpreters] (incl. using them to circumvent GIL? [link], interpreter-per-thread)
* a pure python zip importer [link]
* a lazy importer [link]
* missing *-unpacking generalizations [PEP 448]
* add --path0/--no-path0 cmdline options (also issue15716)
* add __locallookup__ method to metaclass [PEP 447]?
* address the hacky uses of function parameter defaults [link] (Jan K.)

Smaller Tasks


These are things that I hope to get done for 3.5, but won't require a lot of work.

* a new frame attribute, f_func, which is a weakref to the function object that was actually called (basically done)
* give the sys module a custom type and add descriptors [link]
* add inspect.Signature.from_callable()
* change setup.py (the main one) to not rely on imp.load_dynamic() [imp => _imp?] (may be done already)
* update pickle to use __spec__ (in the __main__ case)

Would be Nice


Stuff that I'd like to see in 3.5 but that I likely won't have time to work on.

* interpreter startup hook (via cmdline option) [link] (for coverage.py, etc.)
allow classes to be referenced in the class body
* extension/plugin framework for unittest a la nose [link]

May Be a Good Idea...


Other things that may or may not be worth doing.

* expose _SpecMethods (or something like it) publicly
* add importlib.util.load_from_spec()?
* .ref files [link]
* find a non-hacky replacement for a module replacing itself in sys.modules [link]
* re-organize the sys module with sub-namespaces along the lines of the config categories listed in PEP 432 (similar to but more extensive than PEP 3139)
* parser hooks
* tokenizer hooks
* traceback hooks
* API for excluding functions/methods from tracebacks like importlib and unittest do [issue1705520?]
* spec.data (a wrapper for the data-related API of loaders) [issue14982?]
* add a classonlymethod decorator (not necessarily as a builtin) [link]
* support positional-only parameters (via decorator?) [link][PEP 457]
* add a listproxy type to parallel dictproxy?
* improve re-import of builtin modules, extension modules (see above in Helping Out) [issue14715?]
* generic context API (incl. Context, ThreadLocalContext), a la decimal (email policy?) (asyncio concerns [link])
* post-import hooks (a la PEP 369)
* abc.implements(obj, *<ABC>) that ensures the object has all the appropriate attributes
* ABCMeta.implements(decorated) that registers the decorated class on an ABC and wraps the decorated class's __init__() with an abc.implements() call after it runs [issue12128]
* some mechanism to inherit docstrings [issue15582][issue15731]
* expose PyArg_ParseTuple, etc. in Python
* function to resolve __qualname__ (or provide func.__origin__)

More Radical Ideas


* always import from <module.py> and set __path__ if there is a directory by the same name (obviates __init__.py) [link] (related to PEP 402 discussions) -- at least provide an import hook
* provide a safe mode for ImportSystem (see #5 above) that prevents leaking import side effects outside a context
* replace "if __name__ == '__main__'" [link][PEP 299][PEP 3122] (__spec__.is_main + builtin is_main?)
* a reverse name binding protocol (the object being bound/unbound to the name is notified) [link]
* expose key metaclass operations via special class methods (a la PEP 422)
* attribute docstrings [link]
* a hook to process a docstring after its corresponding object is created (but before decorators are applied)
* revisit PEP 224PEP 329PEP 351PEP 363PEP 377 (Kristjan)

Random Off-the-cuff Ideas


* a decorator for type hints (optional static typing)
* a decorator that supports all sorts of *args/**kwargs handling
* define a standard API for object (modules included) to "install" themselves; provide installer helpers (a la install_importhook() or an Installer base class + ImportHookInstaller base subclass)
* generic Registry class (register() method, etc.)
* spec.open() for locatable specs
* expose an easy way to make some module-global read-only (make use of loader.create_module()?) [issue15031]
* break or return in global module code to stop early (instead of delegating skipped code to submodule)
* ...

This list is as endless as the sea! :-)

Wednesday, June 19, 2013

PyCon 2014 Talks Wishlist


Last year I posted a list of talks I wished someone would give at PyCon 2013.  This is the PyCon 2014 version.  From my wishlist last year 2 of the talks actually happened.  The rest I'd still like to see so I'll include them here.  Feel free to take any of the ideas and run with them.  I'll gladly help anyone write a proposal, whether from my list or not, and see it through the process--I'm on the program selection committee.  FYI, the CFP isn't that far away!

Here are the repeats from last year:
  • Python in Chrome's NativeClient (PyPy anyone?)
  • Python in Android (and I don't mean SL4A) (kivy?)
  • Using Salt States for dev ops
  • Using Salt to control a host of hosts
  • Python internals (take your pick)
  • "Java (hordes of libraries) + Python (duh) = Jython (awesome)"
  • something similar for IronPython and .NET
  • something about Cython's "Pure Python Mode"
  • security: what Python does and doesn't do well
  • threads vs. processes: how to decide between them for the job at hand
And the new ones:
  • CPython's dict and set type implementations
  • CPython's list and tuple type implementations
  • CPython's memory management implementation
  • why the future looks bright for Python packaging
  • contributing to Python--it's not as hard as you think
I'll try to keep this updated at time marches on. Feel free to make suggestions in the comments.

Also, here are PyCon 2014 wishlists from others:

<none yet>

Resources:

https://us.pycon.org/2014/speaking/cfp/ (not active yet)