🤝 Sponsor

This space is reserved for sponsors that support us to keep the newsletter going! Want to support Console? Send us a note at osh@codesee.io

🏗️ Projects

Browse through open source projects on OpenSourceHub.io, add your project to get more exposure and connect with other maintainers and contributors!

💸 Actual

Actual is a self-hosted personal finance system written in NodeJS. It has a synchronization element so that all your changes can move between devices without any heavy lifting.

language: JS stars: 6064 last commit: today
repo: github.com/actualbudget/actual
site: actualbudget.com

🤖 Visual ChatGPT

Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models by Microsoft.

language: Python stars: 13651 last commit: 2 days
repo: github.com/microsoft/visual-chatgpt
paper: arxiv.org/abs/2303.04671

🦀 feroxbuster

A fast, simple, recursive content discovery tool written in Rust.

language: Rust stars: 3827 last commit: today
repo: github.com/epi052/feroxbuster
site: epi052.github.io/feroxbuster-docs

Join thousands of other open-source enthusiasts and developers in the Open Source Hub Discord server to continue the discussion on the projects in this week's email!

🎙️ Interview With Ben of feroxbuster

Hey Ben! Thanks for joining us! Let us start with your background.

I’m currently in the United States, specifically Texas. 

Honestly, I grew up not doing much with computers at all. It wasn’t until I started working on my undergraduate degree (in Accounting 🙃), that I took a programming elective. After that class, I knew I had found something I truly enjoyed, and eventually changed my major to Computer Science.

I was able to get myself into some computer related jobs like teaching Cybersecurity and a stint doing mostly C and Python programming. All the while, on my own time, I was writing code for personal projects, taking OffSec courses, participating in Capture the Flags, hacking on bug bounty programs and Hacking The Box™. Basically, while I was working, I was still learning new things outside of work as well. The pursuit of knowledge eventually led me to getting my master’s degree in Computer Science as well.  

Nowadays, I’m fortunate enough to leverage all those hours on ctfs, certifications, and school by doing Vulnerability Research and Reverse Engineering (VR/RE). I’ve found that my heart lies in breaking things, and building things that break things faster.

What languages do you like?

My go-to languages are Rust and Python, in that order. I’m able to write a wide array of other languages as needed as well, but Rust and Python are what I reach for when the choice is up to me.

If I gave you $10 million to invest in one thing right now, where would you put it?

I’d split it into two $5 million chunks and invest one into openai or a similar AI company. I’d take the other half and invest it into a company that’s doing AI detection. Then, I could sit back and watch the arms race. That still counts as one thing, right?

What are you currently learning?

I just took a course on Binary Deobfuscation. It was a lot of fun and dealt with things like mixed boolean arithmetic, symbolic execution, and program synthesis. I haven’t taken it any farther than the course itself, but it’s certainly an interesting topic that I hope to spend more time with later.

For those interested, here’s an article written by the course author: A gentle introduction to SMT-based program analysis. It’s not specifically about binary deobfuscation, but is a related topic that comes up often when talking about deobfuscation.

Why was feroxbuster started?

I wrote a few school assignments in Rust and wanted to really learn the language, so I started feroxbuster. I had already written a recursive (Python) wrapper around gobuster and figured that writing a recursive content discovery tool in Rust would be a great way to learn and practice.

Where did the name for feroxbuster come from?

Ferox is short for Ferric Oxide. Ferric Oxide, simply put, is rust. The name rustbuster was taken, so I decided on a variation.

How did it get popular?

There were a few users/stars early on, likely due to me demonstrating it as part of some HackTheBox write-ups and tweeting releases as they happened. It really started getting a lot of traction when Stok Fredrik (@stokfredrik) featured it in some of his (Bug) Bounty Thursday videos on YouTube. After that some other popular infosec folks like 0xdf, Ippsec, and Jason Haddix publicly endorsed it. It’s been a lot of fun seeing so many people use and enjoy something I made.

Are there any overarching goals of feroxbuster that drive design or implementation?

Feroxbuster’s focus has always been scanning for content discovery. I wanted it to adhere to the old Unix philosophy: 

Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new “features.”

That’s not to say that feroxbuster hasn’t gotten a lot of new features over its life, it has. However, each one has either aided in the discovery of more content, or has made the tool more user-friendly/able to integrate into typical recon workflows.

What trade-offs have been made in feroxbuster as a consequence of these goals?

Mostly, the tradeoffs were politely declining feature requests for things that would take feroxbuster into areas that weren’t content discovery.

What is the most challenging problem that’s been solved in feroxbuster, so far (code links encouraged)?

They weren’t the most challenging problems, but I had a lot of fun working on the following: 

1. The array-backed max-heap for --auto-tune

2. The Term Frequency Inverse Document Frequency (TF-IDF) model for --collect-words

I don’t get many opportunities to build bespoke data structures, but that’s exactly what underpins --auto-tune’s rate adjusting. It’s an array-backed max-heap that I populate based on the current scan speed. When a certain number of errors or 429’s are encountered, I move down the heap, to the right child, which is a lower speed than its parent node (the left child being a higher speed). As errors come and go, I move around the tree to arrive at new rate limits. It reminded me of my data structures and algorithms classes, and was just fun to do.

The TF-IDF model allowed me to integrate some Natural Language Processing into how feroxbuster looks for interesting words while scanning. Those words deemed interesting enough are added to the wordlist that feroxbuster uses to scan the site. The hope is that by extracting words from the target’s webpage, we’re likely to find hosted content that wouldn’t be in a normal wordlist. 

What are you most proud of?

As far as feroxbuster, and open source/community contributions in general, I get a lot of enjoyment out of hearing that something I wrote helped someone and/or they found it useful. It was fun when I started my new job, and a few of the folks messaged me privately to say that they use feroxbuster all the time and to say thank you. 

What is the best way for a new developer to contribute to feroxbuster?

Just open an issue and start a conversation! I’m more than happy to guide folks as they work on an issue.

Are there any other projects besides feroxbuster that you’re working on?

Yup! The other main project I’m working on right now is FeroxFuzz (https://github.com/epi052/feroxfuzz), a structure-aware HTTP fuzzing library. The primary goal in writing FeroxFuzz was to move some core pieces out of feroxbuster and into a place where they could be generally useful for other folks. In doing so, my hope is that anyone who wants to write web tooling and/or one-off web fuzzers in Rust, can do so with minimal effort.

Want to join the conversation about one of the projects featured this week? Drop a comment, or see what others are saying!

Leave a comment

Interested in sponsoring the newsletter or know of any cool projects or interesting developers you want us to interview? Reach out at osh@codesee.io or mention us @ConsoleWeekly!