From web developer to database developer in 10 years - Notes on software development

Last month I completed my first year at EnterpriseDB. I'm on the teamthat built and maintainspglogical and who, overthe years, contributed a good chunk of the logical replicationfunctionality that exists in community Postgres. Most of my work, ourwork, is in C and Rust with tests in Perl and Python. Our focus thesedays is a descendant of pglogical called PostgresDistributed whichsupports replicating DDL, tunable consistency across the cluster, etc.

This post is about how I got here.

Black boxes

I was a web developer from 2014-2021†. I wroteJavaScript and HTML and CSS and whatever server-side language: Pythonor Go or PHP. I was a hands-on engineering manager from 2017-2021. Iwas pretty clueless about databases and indeed database knowledge wasnot a serious part of any interview I did.

Throughout that time (2014-2021) I wanted to move my career forward asquickly as possible so I spent much of my free time doing educationalprojects and writing about them on this blog (or previous incarnationsof it). I learned how to write primitive HTTP servers, how to writelittle parsers and interpreters and compilers. It was a virtuous cyclebecause the internet (Hacker News anyway) liked reading these postsand I wanted to learn how the black boxes worked.

But I shied away from data structures and algorithms (DSA) becausethey seemed complicated and useless to the work that I did. That is,until 2020 when an inbox page I built started loading more and more slowly asthe inbox grew. My coworker pointed me at Use The Index,Luke and the DSA scales fell from myeyes. I wanted to understand this new black box so I built a littlein-memory SQLdatabase withsupport for indexes.

I'm a college dropout so even while I was interested in compilers andinterpreters earlier in my career I never dreamed I could get a jobworking on them. Only geniuses and PhDs did that work and I wasneither. The idea of working on a database felt the same. However, Icould work on little database side projects like I had done before on othertopics, so Idid. Or a series ofexplorations of Raftimplementations, others' and my own.

Startups

From 2021-2023 I tried to start acompany and when thatdidn't pan out I joined TigerBeetle as a cofounder to work onmarketing and community. It was during this time I started theSoftware Internals Discord and/r/databasedevelopmentwhich have since kind of exploded in popularity among professionalsand academics in database and distributed systems.

TigerBeetle was my first job at a database company, and while Icontributed bits of code I was not a developer there. It was a wayinto thespace. Andindeed it was an incredible learning experience both on the cofounderside and on the database side. I wrote articles with King and Joranthat helped teach and affirm for myself the basics of databases andconsensus-based distributed systems.

Holding out

When I left TigerBeetle in 2023 I was still not sure if Icould get a job as an actual database developer. My network hadexploded since 2021 (when I started my own company that didn't pan out)so I had no trouble getting referrals at database companies.

But my background kept leading hiring managers to suggest putting meon cloud teams doing orchestration in Go around a database rather thanworking on the database itself.

I was unhappy with this type-castingso I held out while unemployed and continued to write posts and hostvirtual hackweeks messing withPostgres and MySQL. I started the firstincarnationof the Software Internals Book Club during this time, readingDesigning Data Intensive Applications with 5-10 other developers inBryant Park. During this time I also started the NYC Systems CoffeeClub.

Postgres

After about four months of searching I ended up with three goodoffers, all to do C and Rust development on Postgres (extensions) asan individual contributor. Working on extensions might sound like thedefinition of not-sexy, but Postgres APIs are so loosely abstractedit's really as if you're working on Postgres itself.

You can mess with almost anything in Postgres so you have to bevery aware of what you're doing. And when you can't mess withsomething in Postgres because an API doesn't yet exist, companies havethe tendency to just fork Postgres so they can. (This tendency isn'tspecific to Postgres, almost every open-source database company seems to have along-running internal fork or two of the database.)

EnterpriseDB

Two of the three offers were from early-stage startups and after morethan 3 years being part of the earliest stages of startups I was happyfor a break. But the third offer was from one of the biggestcontributorsto Postgres, a 20-year old company called EnterpriseDB. (You can probably come up withdifferent rankings of companies using different metrics so I'm onlysaying EnterpriseDB is one of the biggest contributors.)

It seemed like the best place to be to learn a lot and contributesomething meaningful.

My coworkers are a mix of Postgres veterans (people whocontributed the WAL to Postgres, who contributed MVCC to Postgres, whocontributed logical decoding and logical replication, who contributedparallel queries; the list goes on and on) but also mydeveloper-coworkers are people who started at EnterpriseDB ontechnical support, or who were previously Postgres administrators.

It's quite a mix. Relatively few geniuses or PhDs, despite what I usedto think, but they certainly work hard and have hard-earnedexperience.

Anyway, I've now been working at EnterpriseDB for over a year so Iwanted to share this retrospective. I also wanted to cover what it'slike coming from engineering management and founding companies togoing back to being an individual contributor. (Spoiler: incrediblyenjoyable.) But it has been hard enough to make myself write this much soI'm calling it a day. :)

I wrote a post about the winding path I took from web developer to database developer over 10 years. pic.twitter.com/tf8bUDRzjV

— Phil Eaton (@eatonphil) February 15, 2025

† From 2011-2014 I also did contract web development but this waspart-time while I was in school.

本文章由 flowerss 抓取自RSS，版权归源站点所有。

查看原文：From web developer to database developer in 10 years - Notes on software development