Python is simple, beautiful, explicit. When you break Python’s heart it doesn’t go sulk in the corner. It tells you exactly what’s wrong. So you can make it right. You would expect that, with such a beautiful language, its primary package index (pip) would also be the kind of friend who never leaves you out in the cold.
But you would be wrong.
Publishing Python packages for the first time is painful. The documentation is exhaustive and yet it somehow misses the key points that, well, you’ll just have to figure out on your own. But you can’t complain because Python is largely glued together with duct tape, bailing wire and volunteers. And you’re one of those volunteers now. So make it better. These docs aren’t going to write themselves people!
So here’s the documentation I wish I’d read the first time round. Keep in mind that this is hard for you so that it’ll be easy for the thousands of developers who use your module.
Step 1: Test your code
Don’t worry about all those other users on other Python versions. Just get it working for you. I’ll explain later. For now, just test it and make sure it works. The best way to do this is to save your code in one of the package paths. If you don’t know a package path, just open up the python IDLE and type:
>>> import sys >>> sys.path
This should return a bunch of paths. Pick one. Stick your module in it. Now go to another folder and open python and see if you can import your module and get it working. Now try another folder you’ve never been in before and try to run it from there. If it works, you’re done testing.
Step 2: Make your code compatible with other versions of Python
Here’s one step that is way simpler than it ought to be. From your terminal, go back to the package path where you stored the module and type:
# pasteurize -w mypy3module.py
…where mypymodule is the name of your module that you got working in Python 3.x. If you are coding in some ancient language such as Latin, Python 2.x or Sanskrit, DON’T WORRY! Just use futurize in place of pasteurize. Isn’t that cute?
Now, when you go back into your module, you’ll notice that a bunch of your code has been rewritten by a robot. But it works, so who cares?
One small caveat: future has no way of updating your package dependencies, so you’ll still need to test your futurized code on all versions. If it breaks after futurize, you can probably solve this problem by managing the dependencies in step 4 (below)
Step 3: Rename your code and stick it in a folder…and then another folder
Up until this point, your code has probably just been one file named something like mypy3module.py. That’s fine! Don’t change a thing about it. Just rename the file __init__.py and stick it in a folder called mypy3module (or whatever you want the name of your module to be). At this point it might be a good time to search for your module name at pypi.python.org and make sure no one else has nabbed it already. If they have, find another name. Now stick that folder in another folder with the same name. Your eventual file structure will look like:
|_ README.rft (optional)
Step 4: Build your setup.py file
This is required, and it’ll take a bit of time to write, but you can mostly copy mine. There are a couple lines that bear explaining:
Your versioning scheme should be deliberate. The first number shouldn’t change unless you’ve made a change to your code that might break code built on previous versions. The second number changes to indicate a feature addition that won’t break dependent code. The third number changes for maintenance or any minor change. I’m going to skip past the list of all the compatible Python versions, which is pretty self-explanatory, to the topic:
'Topic :: Internet :: WWW/HTTP',
The best way to classify your code is to search for similar code at pypi.python.org and see what topic they adopted. Next, the module:
This can just be the name of your folder. Finally, the required files:
install_requires=[ 'requests','future','requests[security];python_version<"2.9"', ],
OK. This one is hella important. If you screw this up you will want to stab your eyes out with a hammer. Don’t list everything you stuck in the import at the top. Use ‘future’ (cause that’s what you used to make it cross-version compatible), and whatever non-standard library packages you installed. If you can’t remember which ones you had to install (or if you’re using something like Anaconda, which may have installed them for you), just go to the python standard library and search for each of your imports. Only list the ones you can’t find! If you list something from the standard library, pip will break whenever someone tries to install it. If you fail to list a required package and your user doesn’t have it, your code will fail the first time they try to use it. Also notice the special package for pre-python2.9. As you’re testing your code on different versions of python, take special note of what adjustments you have to make.
Step 5: Write setup.cfg
This is easy. This file just needs to say:
This tells the package manager that your code is cross-Python compatible. Which, thanks to step 2, it is.
Step 6: Ship it!
So you may want to create a README.rst file before you do this step, but it’s not required. And hey, who needs documentation anyway? Not Chuck Norris. So let’s get this package into the cloud! If you haven’t already, you’ll want to register at PyPi. Remember those credentials. You might need them again someday. Like now. Use cd to navigate into the outer wrapper of your package from shell. From there, type:
pip install twine python setup.py sdist bdist_wheel
At this point, it’s a good idea to check the /dist folder. If all went well, you should see a tar.gz file and a wheel npr-1.1.11-py2.py3-none-any.whl If you see a folder that has your os name in it (like mac os or linux os), bdist failed and tried to create a package out of some bullshit. One possible remedy is to roll back to a previous version. You can also try looking in the build/lib file and checking to see that your __init__.py folder is there. If not, try adding it manually. If you see multiple versions of these two files, delete the prior versions before running:
twine upload dist/* --skip-existing
This last command will prompt you for that user name and password from your PyPi registration and, after you enter it, your code should upload properly or tell you what’s wrong. The –skip-existing isn’t necessary if you’ve deleted the prior versions in your dist file.
Step 7: Test your package
Now from your terminal go into your python package path and delete your original module. Open up the python IDLE and try to import your module. It should give you an error to prove that it’s no longer installed. Now exit() out of the IDLE and try to install your package:
pip install yourpackagename
If this works perfectly for you the first time, you make me angry.
But if, after entering the above, you get error messages, don’t lose heart. Just edit your setup.py file to try to address the error messages and, this is key: increment your version number. Otherwise, PyPi will ignore your silly little attempts to update your package, and your package will be broken forever. Once you’ve updated your setup.py, re-run Step 6 and 7 until you get it working. Congratulations!
It’ll be easier the second time, right?