Posts by Jeff Maggio

A Beginners Guide to Pipelines

What is a Pipeline???

In short, its everything that happens from your raw data to your results. All scripts and algorithms can be represented as a pipeline. If your background is computer science, you may have heard this concept referred to as a “graph”. ImagePypelines at it’s core is just a library to construct pipelines. Let’s teach by example.

Read more ...

Let’s be honest, code in science isn’t portable

I recently read an excellent blog post by Ishan Mishra on Astrobites. He lays out a compelling case for changing the way we code in science.

As a student, and a research engineer, I frequently encountered the “messy monster codes” he describes. I’ve written plenty of them myself. The competitive and sometimes snotty nature of academia prioritizes results. We focus on how we can get our code to run now. We interlace our code with print statement. We run our code again and again after each incremental change. Too often our comments are out of date or confusing. Ishan points out four ways we can make these codes better:

XKCD data pipeline

Read more ...