Sponsorship

If you, or someone you know, is interested in sponsoring the newsletter, please reach out at console.substack@gmail.com

Not subscribed to Console? Subscribe now to get a list of new open-source projects curated by an Amazon engineer in your email every week.

Already subscribed? Why not spread the word by forwarding Console to your friends and colleagues?

Projects

scylla

Scylla is the real-time big data database that is API-compatible with Apache Cassandra and Amazon DynamoDB.

language: C++, stars: 7107, watchers: 345, forks: 832, issues: 1772

last commit: August 29, 2021, first commit: August 9, 2014

https://twitter.com/ScyllaDB

noise

noise is an opinionated, easy-to-use P2P network stack for decentralized applications, and cryptographic protocols written in Go.

language: Go, stars: 1643, watchers: 74, forks: 188, issues: 22

last commit: March 30, 2020, first commit: June 13, 2018

https://twitter.com/PerlinNetwork

diffsitter

Diffsitter is a tree-sitter based AST difftool to get meaningful semantic diffs.

language: Rust, stars: 881, watchers: 9, forks: 15, issues: 12

last commit: September 13, 2021, first commit: June 26, 2020

https://afnan.io/

🎉 Shout out to Dragos for becoming a Gold Tier Patreon subscriber this week! Thank you for your support Dragos! 🎉

Console is powered by donations. We use your donations to grow the newsletter via advertisement. If you’d like to see the newsletter reach more people, or would just like to show your appreciation for the projects featured in the newsletter, please consider a donation 😊

Donate Today

An Interview With Botond Dénes of Scylla

Hey Botond! Thanks for joining us! Let’s start with your background. Where have you worked in the past, where are you from, how did you learn how to program, what languages or frameworks do you like?

I’m from a small village in Northern Transylvania, Romania. I was fascinated by computers as a child, although I mostly just played games on them. I was introduced to programming in high-school and liked it so much that I decided I want to become a programmer.

I started my career as a web-programmer but I switched to C++ as soon as I could. I stuck to it ever since (with a minor Python de-tour).

I like to work on Linux, do backend work, and to be exposed to a wide variety of challenges, so ScyllaDB is a perfect fit for me.

What is one app on your phone that you can’t live without that you think others should know about?

I mostly use my phone for phone stuff: communicating (calls, text, chat, email). Lately I started listening to YouTube videos while doing things. I find this a very useful way to turn something dull like mowing the lawn into an opportunity to learn something or just be entertained.

Have you looked into any podcasts for this?

Not yet, so far I have only subscribed to podcast versions of some YouTube channels that I already listen to. I have YouTube Premium, so I can listen to videos in the background, and hence I didn't have the motivation to look for podcasts yet.

If you could teach every 12 year old in the world one thing, what would it be and why?

Two things come to mind that I think are sorely missing from general education: how to handle money and the local equivalent of the highway code. Even if you never drive a car, knowing how the roads “work” makes traveling that much safer for you.

What are you currently learning?

C++. I feel like even after 10 years of using it there is still much to learn about it. Especially with the new standard release cadence, which we usually adopt as soon as mainstream compilers start supporting it.

What have you been listening to lately?

I have a broad range of interests: history, cars, economy, science, space, gaming, etc. I try to give everything a slice of my limited listening/free time.

Are there any overarching goals of Scylla that drive design or implementation?

Make efficient use of modern multi-core CPUs and fast SSDs. This is what dictated the thread-per-core architecture of Scylla as well as the exclusive use of asynchronous I/O and subsequently cooperative multitasking. All of this is provided by the open-source framework Scylla is built on: seastar.

Another important goal is self-tuning, to this end Scylla employs classic control theory to self-tune things like the scheduling and I/O priority of background tasks vs. foreground tasks.

What trade-offs have been made in Scylla as a consequence of these goals?

While I don’t feel we had to make any big trade-offs, for sure we have given up comfort when opting to write Scylla in C++ using seastar’s very awkward continuation API. C++ just gained native coroutine support, so things are getting considerably better on this front.

Are there any competitors or projects similar to Scylla? If so, what were they lacking that made you consider building something new?

Our main competitor is Apache Cassandra. Scylla was in fact started as a rewrite of Apache Cassandra in C++, with the goal to provide a vastly faster and more efficient version of it, which can also scale vertically to huge machines, as well as requiring little to no tuning cycles.

At what point did Scylla stop becoming a rewrite of Cassandra and start to become its own project? I'm guessing there was a point in time when a decision was made to start providing different behaviors than Cassandra supports, so I'm curious about that point in time and what that behavior was. Or, would you still consider Scylla a rewrite?

Although started as a rewrite, providing a faster and more efficient Apache Cassandra was gradually shifted out as a goal as we approached feature parity and we started aiming higher. We developed Alternator, a DynamoDB Compatible API to provide a more performant and lower cost alternative to DynamoDB users, we developed our own CDC API because we think the one Apache Cassandra has is cumbersome to use and we are working on making cluster operations strongly consistent by making use of RAFT, just to mention a few examples.

What was the most surprising thing you learned while working on Scylla?

Just how slow it can be to compile complex C++ code. I think when looking at compile times, Scylla is very competitive with notoriously slow to compile monsters like Chromium and Firefox. This is “thanks” to our heavy use of templates and inlining to maximise performance.

What is your typical approach to debugging issues filed in the Scylla repo?

Usually somebody first triages the issue, to make sure it is a legit bug indeed. Then it is picked up by the team-lead of the relevant team and it is assigned to an engineer. But the process is not written in stone. Sometimes engineers jump on issues that are found in the code they wrote on their own initiative.

How do you balance your work on open-source with your day job and other responsibilities?

Working on the Scylla open-source project is my day job, which is a dream come true for me. I occasionally work on proprietary projects as well (like Scylla-enterprise) but the vast majority of my time is spent on Scylla open-source.

What is the best way for a new developer to contribute to Scylla?

You can look for issues on the github issue tracker labeled with the label n00b. Be sure to check out CONTRIBUTING.md first and of course familiarizing yourself with the basics doesn’t hurt either.

What motivates you to continue contributing to Scylla?

Databases are in an ever greater demand thanks to the ever increasing amount and value of data. It is great to work on something that is such a critical piece of the infrastructure of almost every company. I feel like I’m doing something meaningful. Also contributing to ScyllaDB is simply a lot of fun for me, I love the kind of challenges I’m facing, I’m never bored.