Over the past few years I’ve increasingly tried to make small contributions to open source projects that I use. I’m not on the core team of any one project, so usually my contributions are very small. That’s why I was very surprised when I got an email from Google’s Open Source Peer Bonus program, letting me know that I had been nominated!

I recently discovered that Pandas has a function to propagate time series events forward (or backward) in time across a DataFrame. Here’s how it works.

Since I’ve started using Apache Spark, one of the frequent annoyances I’ve come up against is having an idea that would be very easy to implement in Pandas, but turns out to require a really verbose workaround in Spark. A recent example of this is doing a forward fill (filling null values with the last known non-null value).

Pandas has a useful feature that I didn’t appreciate enough when I first started using it: groupbys without aggregation. What do I mean by that? Let’s look at an example.

This is a cute trick I discovered the other day for quickly computing the time since an event on regularly spaced time series data (like monthly reporting), without looping over the data.