🤝 Sponsorship

On Deck Engineering - Where engineers go to become leaders

Impactful engineering leadership requires a distinct skill set from being a great engineer.

Scaling teams and projects can be difficult to navigate without any support.

The On Deck Engineering Fellowship (ODE) is solving this problem. 

ODE is a curated community designed to help aspiring engineering leaders and technical managers build a predictable path to leadership. It’s a safe space to build managerial confidence, learn from industry leaders and cultivate a personal advisory board.

Applications close soon & there are only 80 spots available!

🏗️ Projects

Upptime

Upptime is the open-source uptime monitor and status page, powered entirely by GitHub Actions, Issues, and Pages.

language: Typescript, stars: 9509, forks: 519, issues: 47 last commit: April 16, 2022
repo: github.com/upptime/upptime
site: demo.upptime.js.org/

NextUI

NextUI is a beautiful, fast, and modern React UI library that allows you to make beautiful, modern, and fast websites/applications regardless of your design experience, created with React.js and Stitches, based on React Aria, and inspired by Vuesax.

language: TypeScript, stars: 5785, forks: 249, issues: 18, last commit: April 15, 2022
repo: github.com/nextui-org/nextui
site: nextui.org/

DataStation

DataStation is an open-source data IDE for developers. It allows you to easily build graphs and tables with data pulled from SQL databases, logging databases, metrics databases, HTTP servers, and all kinds of text and binary files.

language: Typescript, stars: 1325, forks: 45, issues: last commit: April 15, 2022
repo: github.com/multiprocessio/datastation
site: datastation.multiprocess.io/

🎤 Interview With Phil of DataStation

Hey Phil! Thanks for joining us! Let’s start with your background. Where have you worked in the past, where are you from, how did you learn how to program, and what languages or frameworks do you like?

I started programming in middle school because I wanted to build Jarvis (from Iron Man). That didn’t work out but I ended up getting a couple of programming internships in high school, did a year of college, dropped out when I couldn’t afford it and have been programming professionally ever since. Most recently I was an engineering manager at Oracle, but I left last June to work on my own projects.

When I was in college I got super into programming language research so for a few years after dropping out I spent time with OCaml, Standard ML, and Scheme. I’m still the primary moderator of /r/sml. But at work, I mostly use Go, Python, and JavaScript.

What made you want to leave Oracle?

Combination of difficult org changes around that time at Oracle and that my wife had just started a good job so it gave me some breathing room.

It was just a good time to start something new.

What's the most interesting thing you learned about programming languages during that time?

Well, I got really good at functional programming! :) I didn't even learn how to iterate over a tree using a stack (i.e. non-recursively) until this year. This is of course not a great thing since pure recursion is not a great fit for dynamic input like parsing because you can blow the stack if the user gives you data that is too nested.

Who or what are your biggest influences as a developer?

I’m not sure I’ve had big influences other than just a desire to improve on every aspect of programming because of all the incredibly talented people I saw around me. In the last few years though Anders Hejlsberg has definitely come to be one of the developers I look most up to because of his excellence and pragmatism as a lead of Delphi, C#, and now TypeScript.

What's an opinion you have that most people don't agree with?

JavaScript is a fantastic language. It’s one of the most expressive languages in real use today and also has some of the fastest implementations of any high-level language. Furthermore, I think it (and web development in general) has offered a pathway for a lot of people like myself to enter the (upper-)middle class.

What’s your most controversial programming opinion?

Python, as a language, is awful. It doesn’t have a very uniform syntax like Scheme or JavaScript, some things are pointlessly not values (like class declarations and function declarations), the way it handles default argument values is messy, and whitespace sensitivity plus dynamic typing makes it so easy to introduce bugs during refactoring.

That said, I care more about products and people than programming languages when it comes to how I choose my work. Nor do I think worse of anyone who likes Python. Even in my own projects (i.e. when I have other choices), I’ll still choose Python when it comes to building scripts. But programming in Python makes me grimace.

What is your favorite software tool?

Chrome. Browsers more generally but Chrome is the nicest one. It's fast, beautiful, works on every platform (except iOS maybe, I don't know), has the best JavaScript implementation, has great devtools. I've been able to run Windows, Linux, FreeBSD, macOS on my desktops and laptops (at different times) and do real work on them all because Chrome (or Chromium) is guaranteed to work. Firefox might similarly work on all those platforms but it has always seemed slower and more visually clunky to me.

What makes Chrome so useful?

It's fast, beautiful, works on every platform (except iOS maybe, I don't know), has the best JavaScript implementation, has great devtools. I've been able to run Windows, Linux, FreeBSD, macOS on my desktops and laptops (at different times) and do real work on them all because Chrome (or Chromium) is guaranteed to work. Firefox might similarly work on all those platforms but it has always seemed slower and more visually clunky to me.

If I gave you $100 million to invest in one thing right now, where would you put it?

Education. I don’t think we can spend too much on education. I think underinvestment in education is the root of a lot of Western social and economic problems.

What have you been listening to lately?

I don’t normally like American country or folk music but I stumbled onto Johnny Cash’s 2002 American IV album recently and really like it.

Why was Datastation started?

I’ve been a tech lead or manager for the last 5 years. There have been a number of different reports I wanted to run that varied widely in the system they had to access. Customer data in PostgreSQL, historic API usage info (i.e. request logs) in Elasticsearch, sales data in Salesforce, random CSVs from product managers. I’ve spent a lot of time trying to hack Grafana or DBeaver to help me out but mostly I ended up falling back to Python scripts that take a while to write. Every time I’d want a new report I’d have to remember how to connect to PostgreSQL or Elasticsearch or whatever and it just took a long time even for simple stuff. And even once you have the report you still need to figure out how to distribute it and how often to run it and whatnot.

So when I quit Oracle last June I decided to build an app that takes away a lot of the ancillary issues around reporting on diverse data. I wanted it to have built-in support for connecting to every major data system so you don’t need to remember that glue code, just your credentials, and your query. And I wanted it to come with support for hosting the resulting data as a dashboard or recurring email export.

What trade-offs have been made in Datastation as a consequence of your design goals?

It has to be easy to adopt. In corporate environments like Oracle, it can be really hard to try out new software. So while SaaS may be the easiest way to monetize or get funding for a project, I decided to go desktop first because I wanted this to be something developers actually use. They shouldn’t need to go through the procurement or security circus just to help themselves out.

Since it’s a desktop app it stays on your laptop. The data stays on your laptop. You control it completely.

What is the most challenging problem that’s been solved in Datastation, so far?

It’s kind of on a tangent but for the first time in my career, I decided to really dedicate time to benchmarking as I’m making decisions. I’ve been able to publish a number of these benchmarks and their code. Like Benchmarking esbuild, swc, tsc, and babel for React/JSX projects, Speeding up Go's builtin JSON encoder up to 55% for large arrays of objects, and Experimenting with column- and row-oriented datastructures.

Are there any competitors or projects similar to Datastation? If so, what were they lacking that made you consider building something new?

There are TONS of data tools but their focus is almost solely on product managers, data scientists, business analysts, and not developers. Maybe the only BI tool that has any traction among developers is Metabase. The big problem with BI tools is that they’re 1) mostly focused on SQL and 2) they don’t really provide advanced escape hatches like the ability to script data. In contrast, DataStation can talk to any kind of database (SQL like PostgreSQL but also No-SQL like Prometheus, Influx, and Airtable). And DataStation allows you to script in JavaScript, Ruby, Python, Julia, or R.

And the only data-code tool that might have had a hope for traction among developers is Jupyter. But Jupyter is focused almost solely on data scientists; I’ve never seen a non-data scientist developer use it. Furthermore, Jupyter doesn’t come with the same high-level UI support for querying databases or accessing files on servers that DataStation does.

DataStation is focused on (mostly backend) developers, tech leads, and managers who write a little code and have databases and APIs in production they want to report on.

What is the release process like for Datastation?

Recently I’ve fallen into the rhythm of having one primary feature that gets released each month and then a few smaller ones. So I spend something like two weeks working on the big feature and then a week doing cleanup and testing before a release. Then after the release I take a break and focus on writing a cool blog post and other marketing or documentation.

How do you intend to monetize Datastation?

I’m working on releasing a paid Professional Edition of the desktop app and longer-term I’d like to offer a SaaS version where it’s not just you building reports but your whole team being able to work on reports together, whether it’s editing or just viewing. Anyone interested in following along with that process should subscribe where I share monthly updates on features and the community.

What is the best way for a new developer to contribute to Datastation?

Join the #dev channel on our Discord and say hi! There are a ton of projects ready for beginners to pick up. The only expectation is that you have some experience with Go or JavaScript. But there are way more potential tasks in Go. Since publishing this good first projects list, I've now had two first-time OSS contributors merge features from this list. And I've had a few more folks show up in the #dev channel beginning to explore the code

What motivates you to continue contributing to Datastation?

I’d like to run a successful company. We’ll see if that happens!

Are there any other projects besides Datastation that you’re working on?

5 years ago I decided to write one blog post per month either about a code project or about management or some other work-adjacent topic. In December I did a Lua VM in Rust, last month I did an SMTP server in Go. Stuff like that.

Do you have any other project ideas that you haven’t started?

There’s a huge list!

How about the top three?

Sure, three random ones include "How does regex speed compare to a handwritten lexer?", "An overview of bytecode operations across major VMs" (like JVM, CPython, CLR, etc.) and a long-term goal of mine is to write a series on implementing TCP/IP in userland. But I've tried that one a few times and failed. Maybe one day.

Do you have any suggestions for someone trying to make their first contribution to an open-source project?

Whether or not you’re interested in DataStation, join our Discord! Aside from DataStation stuff, there’s a growing community of developers interested in hacking on databases, compilers, emulators, browsers, graphics, any tough technical challenge you name it.

More directly though I’d say it’s easiest to get started by using software and paying attention to bad docs or bad error messages or flat-out bugs. Report the bugs, make docs suggestion fixes, and improve error messages. These get you familiar with the outer perimeter of projects and as you do more of them you’ll see more ways in if you want to.

And of course, I’ll suggest you come hack on DataStation where I can help you out!

Want to join the conversation about one of the projects featured this week? Drop a comment, or see what others are saying!

Leave a comment

Interested in sponsoring the newsletter or know of any cool projects or interesting developers you want us to interview? Reach out at console.substack@gmail.com or mention us @ConsoleWeekly!