PIP your first Python package

fat_python

Python is simple, beautiful, explicit. When you break Python’s heart it doesn’t go sulk in the corner. It tells you exactly what’s wrong.  So you can make it right. You would expect that, with such a beautiful language, its primary package index (pip) would also be the kind of friend who never leaves you out in the cold.

But you would be wrong.

Publishing Python packages for the first time is painful. The documentation is exhaustive and yet it somehow misses the key points that, well, you’ll just have to figure out on your own. But you can’t complain because Python is largely glued together with duct tape, bailing wire and volunteers.  And you’re one of those volunteers now.  So make it better. These docs aren’t going to write themselves people!

So here’s the documentation I wish I’d read the first time round.  Keep in mind that this is hard for you so that it’ll be easy for the thousands of developers who use your module.

Step 1: Test your code

Don’t worry about all those other users on other Python versions.  Just get it working for you.  I’ll explain later.  For now, just test it and make sure it works.  The best way to do this is to save your code in one of the package paths.  If you don’t know a package path, just open up the python IDLE and type:

>>> import sys
>>> sys.path

This should return a bunch of paths.  Pick one.  Stick your module in it.  Now go to another folder and open python and see if you can import your module and get it working.  Now try another folder you’ve never been in before and try to run it from there.  If it works, you’re done testing.

Step 2: Make your code compatible with other versions of Python

Here’s one step that is way simpler than it ought to be. From your terminal, go back to the package path where you stored the module and type:

# pasteurize -w mypy3module.py

…where mypymodule is the name of your module that you got working in Python 3.x. If you are coding in some ancient language such as Latin, Python 2.x or Sanskrit, DON’T WORRY! Just use futurize in place of pasteurize. Isn’t that cute?

Now, when you go back into your module, you’ll notice that a bunch of your code has been rewritten by a robot. But it works, so who cares?

One small caveat: future has no way of updating your package dependencies, so you’ll still need to test your futurized code on all versions.  If it breaks after futurize, you can probably solve this problem by managing the dependencies in step 4 (below)

Step 3: Rename your code and stick it in a folder…and then another folder

Up until this point, your code has probably just been one file named something like mypy3module.py. That’s fine! Don’t change a thing about it. Just rename the file __init__.py and stick it in a folder called mypy3module (or whatever you want the name of your module to be). At this point it might be a good time to search for your module name at pypi.python.org and make sure no one else has nabbed it already. If they have, find another name. Now stick that folder in another folder with the same name.  Your eventual file structure will look like:

mypy3module

|_ setup.py

|_ setup.cfg

|_ README.rft (optional)

|_ mypy3module

     |_  __init__.py

Step 4: Build your setup.py file

This is required, and it’ll take a bit of time to write, but you can mostly copy mine.  There are a couple lines that bear explaining:

version='0.1.2',

Your versioning scheme should be deliberate. The first number shouldn’t change unless you’ve made a change to your code that might break code built on previous versions. The second number changes to indicate a feature addition that won’t break dependent code. The third number changes for maintenance or any minor change.  I’m going to skip past the list of all the compatible Python versions, which is pretty self-explanatory, to the topic:

'Topic :: Internet :: WWW/HTTP',

The best way to classify your code is to search for similar code at pypi.python.org and see what topic they adopted.  Next, the module:

py_modules=['npr'],

This can just be the name of your folder.  Finally, the required files:

install_requires=[
 'requests','future','requests[security];python_version<"2.9"',
 ],

OK. This one is hella important. If you screw this up you will want to stab your eyes out with a hammer. Don’t list everything you stuck in the import at the top. Use ‘future’ (cause that’s what you used to make it cross-version compatible), and whatever non-standard library packages you installed. If you can’t remember which ones you had to install (or if you’re using something like Anaconda, which may have installed them for you), just go to the python standard library and search for each of your imports.  Only list the ones you can’t find!  If you list something from the standard library, pip will break whenever someone tries to install it.  If you fail to list a required package and your user doesn’t have it, your code will fail the first time they try to use it.  Also notice the special package for pre-python2.9.  As you’re testing your code on different versions of python, take special note of what adjustments you have to make.

Step 5: Write setup.cfg

This is easy. This file just needs to say:

[bdist_wheel]
 universal=1

This tells the package manager that your code is cross-Python compatible. Which, thanks to step 2, it is.

Step 6: Ship it!

So you may want to create a README.rst file before you do this step, but it’s not required. And hey, who needs documentation anyway?  Not Chuck Norris.  So let’s get this package into the cloud!  If you haven’t already, you’ll want to register at PyPi.  Remember those credentials.  You might need them again someday.  Like now.  Use cd to navigate into the outer wrapper of your package from shell.  From there, type:

pip install twine
python setup.py sdist bdist_wheel

At this point, it’s a good idea to check the /dist folder.  If all went well, you should see a tar.gz file and a wheel npr-1.1.11-py2.py3-none-any.whl  If you see a folder that has your os name in it (like mac os or linux os), bdist failed and tried to create a package out of some bullshit.  One possible remedy is to roll back to a previous version.  You can also try looking in the build/lib file and checking to see that your __init__.py folder is there.  If not, try adding it manually.  If you see multiple versions of these two files, delete the prior versions before running:

twine upload dist/* --skip-existing

This last command will prompt you for that user name and password from your PyPi registration and, after you enter it, your code should upload properly or tell you what’s wrong.  The –skip-existing isn’t necessary if you’ve deleted the prior versions in your dist file.

Step 7: Test your package

Now from your terminal go into your python package path and delete your original module. Open up the python IDLE and try to import your module. It should give you an error to prove that it’s no longer installed. Now exit() out of the IDLE and try to install your package:

pip install yourpackagename

If this works perfectly for you the first time, you make me angry.

But if, after entering the above, you get error messages, don’t lose heart. Just edit your setup.py file to try to address the error messages and, this is key: increment your version number. Otherwise, PyPi will ignore your silly little attempts to update your package, and your package will be broken forever. Once you’ve updated your setup.py, re-run Step 6 and 7 until you get it working.  Congratulations!

It’ll be easier the second time, right?

Stop worrying about millennials: generation Z is way scarier

Everywhere you look, you can find signs of how millennials are ruining the world.  The real-estate market is going to crash because they don’t own houses.  The manufacturing industry will collapse because they refuse to buy cars and because they subscribe to something called the sharing economy.  And these trophy kids are impossible to work with because — unlike the rest of us — they dislike “canned career advice or the same old, same old management-training tips.”  Also they spend almost as much time looking at their cellphones as their parents spent … watching television.

But there’s another generation coming and, believe me, it’s way worse.  I know this because I have two of them living in my home and they are THE. WORST. ROOMMATES. EVER.

Continue reading

Using JSON dot notation in php

Sometimes it’s hard…to say goodbye…to php.

–The Bleatles

If you’re making the transition to node.js and schema-less databases, you’re probably blowing a mind-fuse every day as you watch your code just work instead of spending hours constructing mindless php workarounds.  So in the spirit of nostalgia (for those of us who just can’t let go), here’s … another mindless php workaround.   Continue reading

Investment advice from a blowhard

Screen Shot 2015-05-20 at 10.21.23 PM
People sometimes make the mistake of asking me for investment advice.  They quickly realize their mistake when they receive a blast from Captain Achilles Von Humboldt Blowhard, but by then it’s too late.  Maybe by putting it on the internet (where the whole world can see it!), I won’t be so tempted to impart my wisdom at the next cocktail party. Continue reading

How is a stock like an equity bond?

Warren Buffett described his philosophy of viewing stocks as ‘equity bonds’ in a 1977 Forbes magazine article and subsequently in a speech at Columbia Business School.  In her book, Warren Buffett and the Interpretation of Financial Statements, Mary Buffett attributes the concept to Buffett, which is interesting considering that the title of her book is a nod to an earlier work by the true father of that philosophy.

Continue reading

Fiction: Ten Fifty on Rodeo

The boys looked up from behind the bush when the cruiser rounded the corner onto Rodeo Drive. Manuel and Chico froze while Chad lazily turned his shoulder towards the brick wall, grabbing the pipe out of Chico’s hand in a single, steady movement. He held it a moment as the officers, a man and a woman, gazed at him. Chad smiled at them and jerked his chin up subversively, with military precision.

Continue reading

Fiction: Eppur Si Mouve

In the days before they were a couple, but after they were an item, Thomas and Hannah spent their Saturday evenings on Capitol Hill in the apartments of the same young policy wonks with whom Hannah spent her workdays. The idea that Washington is Hollywood for ugly people isn’t true. Hollywood has been getting uglier every year since the 1940s, and Hannah’s Washingtonians were all beautiful. Some of them were even moderately successful, though many of them lived indifferently in mildewed basements surrounded by old books. All of them were painfully clever and without ambition.

Continue reading

Proxy surf on a mac for free

There are many uses for this, but the one I’m most familiar with is the need to watch the BBC and ITV from outside the UK. The BBC derives a significant portion of its operating budget from a tax on all british households with a TV (I think it’s something like 11 pounds per month). The British would be bloody well narked if us non-paying colonists could just nip into their website and watch the Beeb whenever we wanted. At the same time, if the BBC put up any sort of serious security on their site (such as a registration wall that asked you to prove you paid the tax), the British people would probably burn down their building. So the BBC digital media team long ago settled on the idea of waving through anyone coming in through a UK server, and keeping out everyone else. It’s a pretty good system: no one has burned down the building yet, and it keeps the foreign wankers out…mostly.

Continue reading