pyuv 1.4.0 released

Today I finally found some time to put together a new pyuv release! Oh, but what is pyuv? I’m glad you asked!

pyuv is a Python module which provides an interface to libuv. libuv is a high performance asynchronous networking and platform abstraction library.

This a big one. Here are some of the highlights:

  • PyPy support (finally!)
  • CPython 3.6 support
  • Overhauled build system (we no longer rely on the libuv build system)
  • Linux and Windows wheels published on PyPI
  • New co-maintainer! Say hi to Marc Schlaich!

See the full changelog here.

I just realized the last release was over a year ago. Time flies! At any rate, here it is now. Enjoy!

How to get AppRTCMobile for Android

More often than not, when reporting issues in the WebRTC bug tracker, you’ll be asked to reproduce it with AppRTCMobile (the “reference” or “demo” app of sorts). This may be a hassle because you might not be able to build it yourself.

Worry not! Google archives all the apk builds from their CI system and you can download it from there. I’m not sure if this publicly available information, but I couldn’t find it anywhere, so here we go!

  • Go to this site and pick the Android builder you desire
  • Then click on the latest successful build (this one for example)
  • Next, locate the “gsutil upload” step (it’s 22 at the time of this writing)
  • There should be a “gsutil.upload” link below it (this one for example)
  • Click it to download the file!

The downloaded file is a zip file containing the apk. Installing it is simple, connect your Android device and do:

Done!

Winds of change

I’ve tried to mentally write this blog post for weeks, but there are just too many things to say, so I’m just going to get on with it and see what happens.

It’s with a weird feeling in my stomach that I say goodbye to AG Projects.  Starting in January I’ll be an Atlassian employee, joining Emil Ivov and the Jitsi team taking on new challenges.

This was by far the toughest decision I’ve ever made in my carrier. Tougher than leaving my home town 7 years ago when I came to Amsterdam.

“Relax, it’s only work” or so the saying goes. For me work is never just work. For better or worse I pour part of my soul into everything I do, otherwise I wouldn’t be able to do it. This means making choices is harder because feelings arise. Moreover, in this case it’s definitely not just work. Moving to Amsterdam changed my life. I’ve met people I wouldn’t have met otherwise, and most importantly, my wife found the motivating job she longed.

These past 7 years have been amazing. I’ve been able to travel the world and attend conferences, write an RFC, meet some amazing people and write the code I wanted, the way I wanted. 7 years is equivalent to eternity at Internet scale, but Adrian always found something awesome for us to work on next.

I consider myself very fortunate to have found great friends, coworkers and mentors throughout my journey in life. They all taught me things I didn’t know and I like to think we learned some others together.  That’s why it’s so damn hard to say goodbye.

From the bottom of my heart, thank you. Best wishes for the future and keep on rocking!

As for my future at Atlassian, I’m really excited for what’s to come! I’m joining a team of extremely talented individuals which have a track record of creating top notch software and I’m glad to be a part of that.

The best things in life are often waiting for you at the exit ramp of your comfort zone.

Here is to the future! We’ll always have Amsterdam.

PS: Hey, they have an opening right now, so if you want to work with these great people and have a chance to criticize my code and design choices you should definitely apply!

Liverpool Summer Camp 2016

These are my last few days here in Liverpool, and as I sit in a half-empty house, looking at the Mersey through the window, all I can think of is what a ride it’s been.

We came here earlier this year because my wife was working on a project and I tagged along.  Since she had to work long hours we looked for a coworking place where I could work from, to force me out of the house and interact with the autochthonous species.

The place we found forever changed me, in ways I’d never foresee. Allow me to introduce you to DoES Liverpool. For many, Liverpool is home of The Beatles and a place where you’ll never walk alone. Not for me. For me Liverpool will always be home of DoES, where I developed my dormant maker skills and brought them to a whole new level.

Browsing through the website I discovered there were laser cutters and 3D printers available, that got me hooked on the spot. Bu then I came across the about page, which outlines their mission, so I fell in love.

2016-10-06-20-52-20

The people there are incredible. Everyone is ready to teach you how things work, have a chat about what they are working on, and help you out when something doesn’t work as expected. At first I wasn’t sure if it’s the maker spirit that makes everyone nice around here or something about the wind on the Merseyside, but be that as it may, Scousers are a great bunch.

Being happy and having a sense of accomplishment are key to producing good work, and thanks to the positive and makery environment, these past few months have been really productive for me. I’m really going to miss this.

If you find yourself in Liverpool, visit DoES, your first day will be free if you bring cake, what’s there not to like?

I guess this is goodbye. Thanks for all the tips and tricks, for maker nights and maker days, and for bearing with me and my million questions 🙂 I hope you folks find the right building to grow DoES and make it even greater. Hopefully I can visit one day.

So long Liverpool, I’ll cherish the memories; and godspeed, DoES, godspeed!

2016-10-07-09-09-08

Skookum JS 0.4.0 and the plan forward

Hey there! Another Skookum JS release is out and about! It’s 0.4.0 this time, slowly making progress.

This time around the release focus was on getting the module system mode Common JS compliant and similar to Node’s. This also brought __filename and __dirname, which simplified a few tests.

There are also some improvements like simple tab completion for the CLI and some new modules, check the full changelog for details.

When I started sjs I had this idea about a JS interpreter with a more “traditional” look, rather than the inherent async model Node provides. Just for the sake of exploring. Well, live and learn, I had no idea Common JS defined more stuff than modules! Somewhat randomly I also ran into Ringo JS, which looks pretty much like what I wanted to do, but built on the JVM. So I’ve decided to follow some of the Common JS specs (open issue here), and let’s see how deep the rabbit hole goes.

commonjs

Skookum JS 0.3.0 released!

Roughly three weeks after the last release, today I’m happy to announce Skookum JS 0.3.0!

What is Sookum JS?

Skookum JS, or sjs for short, is a JavaScript runtime focused on providing comprehensive POSIX APIs.

The motivation for this project comes from answering the question “how would a JavaScript runtime look like if there were no browsers?”.

This new release contains a few new modules: random, system (now non-builtin), process, pwd and uuid.

I’m specially happy about two of those modules: random and process, so let’s explore them a bit:

random

The random module implements a PRNG based on the well known Mersenne Twister PRNG. It also implements a CSPRNG by reading from the best random source available on the system: getrandom on Linux and arc4random_buf on OSX, falling back to reading from /dev/urandom.

If you are an expert in this field and have some comments about the implementation, please reach out!

process

The process module is by far what took the longest to complete for this release. It implements just two functions: daemonize and spawn, to daemonize the current process and to spawn new child processes respectively.

The challenge was that in order to implement those in JavaScript I had to add tons of new APIs to the os module (plus tests and documentation!). I can’t be happier about the result!


There are tons of other additions abd bugfixes, check the full changelog for details and the documentation for all APIs.

Last, I found some inspiration and made a new logo, what do you think?

sjs-logo-icon

Enjoy!

pyuv 1.3.0 released!

Looks like I’m on a release spree! Today I’m releasing pyuv (libuv bindings for Python) version1.3.0.

This release focused on 2 main things:

  • Update to latest libuv release: 1.9.1
  • Add Python 3.5 support

There are other (small) bugfixes too, check the full changelog for details. One of the things I’m most happy about is that thanks to AppVeyor I’m able to provide Wheels for Python 2.7, 3.3, 3.4 and 3.5 on Windows (32 and 64 bits) so you don’t have to compile pyuv, which is sadly not straightforward on Windows; checkout the PyPI page.

Enjoy!

pycares 2.0.0 released!

Tonight I’m happy to announce that pycares (the Python bindings for c-ares, an asynchronous DNS resolver) has reached version 2.0.0.

This release contains a few important features:

  • CFFI port for PyPy (it can optionally also be used in CPython)
  • Python 3.5 support
  • c-ares updated to version 1.11.1

Plus some minor bugfixes. I’d like to thank Jesse (@boytm) for the CFFI patch, that was a massive contribution, thank you so much!

Binary wheels are available for Python 2.7, 3.3, 3.4 and 3.5 on Windows (both 32 and 64 bits), checkout the PyPI page.

Enjoy!

libuv internals: the OSX select(2) trick

In case you didn’t know, libuv is an asynchronous platform abstraction library which you should totally check out.

libuv does a lot more than abstract epoll, kequeue and friends, and today we’re going to take a look at one of the many tricks libuv provides.

On OSX libuv uses the kqueue interface for polling sockets. This is currently the most efficient way to do so. For file i/o, however, libuv uses a thread pool, but that’s probably a topic for another post 🙂

Back to kqueue. While it works perfectly fine for sockets, we use it for other types of file descriptors too: pipes (with uv_pipe_t) and ttys (with uv_tty_t). On Unix systems, uv_pipe_t can also be used to open an arbitrary file descriptor and treat it as a libuv stream, trhough uv_pipe_open.

When a tty is opened with uv_tty_init, libuv opens /dev/tty instead (it’s a bit more complicated now, but let’s assume that, for the sake of simplicity) in order to be able to put stdin/out/err in non-blocking mode, without affecting processes which share them. There is a problem, however: file descriptors pointing to /dev/tty don’t work with kqueue. Ouch. You can verify that with this little Python script, which is a port of the test libuv does:

 

So, what do we do now? As it turns out, those file descriptors don’t work with poll(2) either… but they do work with select(2)! This means that while we use kqueue for most file descriptors, we can use select(2) when that doesn’t work.

Enter The OSX select(2) Trick (TM) by our OSX expert resident Fedor Indutny. The trick is to spawn an auxiliary thread which will use select(2) on a file descritor which doesn’t work with kqueue and report POLLIN and POLLOUT events to the loop thread, where the read and write operations are performed. Have a look here for the first implementation.

The avid reader might be wondering: “what if we have more than 1024 file descriptors? select is not going to work!” You’re right! This was a problem, so let’s enter The OSX select(2) Trick II: _DARWIN_UNLIMITED_SELECT.

This little gem hidden in the manual page tells us that if _DARWIN_UNLIMITED_SELECT is defined at compilation time, we are allowed to go beyond the FD_SETSIZE limit! We cannot create the fd_set as usual nor use FD_ZERO, we’ll need to manually allocate it and zero it with memset: 1, 2, 3.

So, there you go, this is how libuv is able to use file descrioptors that don’t work with kqueue seamlessly on OSX.

Liked the article? Want me to write more about libuv internals? Do let me know!

Skookum JS 0.2.0 released!

Hey there!

Skookum JS, the JavaScript runtime all your friends are talking about just released its 0.2.0 version.

The initial release was a couple of weeks ago, but there are improvement all across the board:

  • Better CLI experience and ability to toggle strict mode
  • Multiple fixes to the build system (I’m still learning CMake)
  • Fix building proper strack traces
  • New modules: os and refactored io
  • Buffer support for i/o operations

These are just the tip of the iceberg, check the changelog for all details.

For those wondering why sjs is so skookum, see the initial anoouncement and the design documentation.

In the next release I’ll be primarily focusing on child process support and maybe experimenting with multi-threading too. Stay tuned!