pycares is a Python module which provides an interface to c-ares. c-ares is a C library that performs DNS requests and name resolutions asynchronously.
Finally, after a long time, I managed to make a new pycares release. Introducing pycares 4.0.0. This is an important one (note the major number change!) here are the most important features from this release:
- Dropped support for Python <= 3.5, the supported Python versions are 3.6-3.9
- Added getaddrinfo()
- Added support for CAA queries
- Updated c-ares to the latest version
- Dropped custom patches from the embedded c-ares library
- Added ability to use the system provided c-ares library
- Refactor automation of the release process and wheel generation and upload to PyPI
I’m looking for help maintaining this module further, if you are interested, please comment here.
While working on this release I had some thoughts and reflections, if you are interested in those come with me in a trip down memory lane.
Inception
It has been 9 years since I published the first version of pycares. The pycares project was actually not the first incarnation of a c-ares Python wrapper I wrote, it used to be part of libuv, as such part of my wrapper pyuv.
I started pyuv 10 years ago with a somewhat open goal in mind: improve my C skills and Python internals knowledge. I didn’t know the rabbit hole was that deep though…
Mistakes were made
To make the build easier and self-contained, I chose to embed the c-ares source code. That in and of itself is not a bad thing. However, that younger Saúl thought it was a great idea to pull a patch that added TTL support to extra records off the mailing list and apply it to the embedded version. “More features for my users!” I thought.
I have come to regret that decision. Keeping an out of tree patch which broke the API and ABI means I (as the pycares author) was the responsible entity for keeping my “fork” up to date with security updates and such. Every update needed the patch reapplied, which made that process messy.
In addition, it made using pycares with a system wide installation of c-ares impossible. Not my finest hour.
asyncio
I jumped early on the asyncio bandwagon. Since c-ares is designed for integration with an asynchronous model, pycares was too, so I quickly built aiodns, which uses pycares under the hood.
A popular asyncio library (aiohttp) picked up aiodns as a dependency and as a result pycares got downloaded over 800 thousand times last month. Whaaaaaat!
Maturity and maintenance burden
I consider the project to be mature at this point. It started out as a pure C Python extension for Python 2 and 3. Then a CFFI based core was introduced so it also worked on PyPy. The CFFI core ended up replacing the pure C core, thus simplifying things quite a bit. Dropping old versions, Python 2 specially, also simplified things.
Even when things are simple(r) everything takes time.
This project never had a business use case. I built it to learn things and because I like Internet technologies in general and DNS in particular, and I just stuck with it.
Earlier this year, it got to my attention the package was in need of some love. But I couldn’t handle it, so I sought help.
I got enough help to fix some important problems and that gave me enough motivation to work on what is now pycares 4.0.
Mistakes were fixed
The complexity in the maintenance of this package was self-inflicted. Hindsight is 20-20, so now, 9 years later and with the wisdom of hindsight it’s time to fix those mistakes of the past.
pycares 4.0 no longer ships with a patched c-ares. Yes this means we lose TTLs on non A/AAA records, but we gain in ease of updates, which is crucial for security and future-proofing the maintenance of the package. At any rate, looks like TTLs are coming to c-ares!
This is just a small package to perform DNS queries from Python, but I’ve learned so much while making it, I can certainly say it has made me a better software engineer.