Ship It! ā€“ Episode #80

Kaizen! 24 improvements & a lot more

with Adam & Jerod

All Episodes

For our last 2022 Kaizen episode, we went all out:

  • šŸ’Ŗ @jerod outdid himself in the number of improvements shipped between Kaizens
  • šŸ•ŗ A few of our listeners contributed ā†’ prompted us to create a new contributing guide
  • šŸ—ŗ We now have a new infrastructure diagram

All of this, and a whole lot more, is captured as GitHub discussion šŸ™ changelog.com#433. If you want to see everything that we improved, that is a great companion to this episode.

Featuring

Sponsors

Sentry ā€“ Working code means happy customers. Thatā€™s exactly why teams choose Sentry. From error tracking to performance monitoring, Sentry helps teams see what actually matters, resolve problems quicker, and learn continuously about their applications - from the frontend to the backend. Use the code CHANGELOG and get the team plan free for three months.

FireHydrant ā€“ The reliability platform for every developer. Incidents impact everyone, not just SREs. FireHydrant gives teams the tools to maintain service catalogs, respond to incidents, communicate through status pages, and learn with retrospectives. Small teams up to 10 people can get started for free with all FireHydrant features included. No credit card required to sign up. Learn more at firehydrant.com/

Sourcegraph ā€“ Transform your code into a queryable database to create customizable visual dashboards in seconds. Sourcegraph recently launched Code Insights ā€” now you can track what really matters to you and your team in your codebase. See how other teams are using this awesome feature at about.sourcegraph.com/code-insights

Notes & Links

šŸ“ Edit Notes

All episode notes are in GitHub discussion changelog.com#433. Feel free to add your thoughts / questions!

Gerhard, Jerod & Adam

Chapters

1 00:00 Welcome
2 00:54 Sponsor: Sentry
3 01:35 Intro
4 08:34 24 Improvements!
5 12:31 PR 415
6 15:30 Fly & Docker
7 18:43 PR 430
8 22:18 Sponsor: FireHydrant
9 23:36 Spotify
10 25:15 You, the listener
11 27:20 PR 431
12 30:40 DCO
13 33:20 Passphrases and 1Password
14 40:24 From LastPass to 1Password
15 43:20 Sponsor: Sourcegraph
16 45:43 Discussion 433
17 51:42 Mermaid text
18 57:36 Dagger
19 1:03:24 Wrap up
20 1:07:47 Outro

Transcript

šŸ“ Edit Transcript

Changelog

Play the audio to listen along while you enjoy the transcript. šŸŽ§

Our last Kaizen, episode 70, with a single big improvement, IDv3x Elixir library, with Larsā€¦ You know it, itā€™s episode chapters. So now you can enjoy episode chapters, because of that one single improvement. This is Kaizen episode 80, with Jerod and Adamā€¦

Whatā€™s up?

All good.

All good.

But Jerod has 24 improvements in this episode. Okay, I just have to start thereā€¦

Woo-hoo!

How did you manage that, Jerod?

Kaizen. Iā€™m just kaizening it, I donā€™t knowā€¦ There are a lot of little things ā€“ everytime you do a big release, or big feature, and you drop it in, thereā€™s always a bunch of little things, that either youā€™ve been putting them off, because they havenā€™t been as important, or theyā€™re related to the thing that you shipped, because it wasnā€™t 100% bakedā€¦ As was the case with chapters. So lots of little improvements to chapters, and then once you get rolling, you know, you just keep improving all areas of the applicationā€¦ And I just ā€“ I hit my stride and made some commits to this one.

Once you pop, you canā€™t stop.

Thatā€™s right.

They do say that about Pringlesā€¦

Yeah, exactly. We have to ā€“ anyways, the point is, once you start improving, you canā€™t stop.

Do you think the word ā€œcontinuousā€ and ā€œalwaysā€ are similar, or synonymous? Continuous, alwaysā€¦ Theyā€™re kind of the same, right?

I would say theyā€™re definitely similar, but yeahā€¦

Theyā€™re pretty close, right? So we have an ABR method elsewhere; Always Be Recording. So it could be ABI, Always Be Improving.

Always Be Shipping.

Always Be Shipping. ABS. This breaks.

ABI, Always Be Improving. I like that. I think we mentioned it in a previous Kaizen. Anyways, back to the improvementsā€¦ Jerod - I cannot believeā€¦ So we started doing a few things differently for this episode. The first one - thereā€™s a GitHub discussion; weā€™ll come back to that later. Secondly, we have GitHub tags for every single Kaizen, from Kaizen oneā€¦ And now we can compare them. So itā€™s comparing from Kaizen 7 to our main branch, what are the differences? A lot. Jerod has been killing it. And you have so many things that you meant straight into production, rather than PRs, rather than anything like that. So what was your thinking there?

Why do you mean? Like, why donā€™t I open pull requests and then merge them myself?

Something like that, yes; to keep things together.

Itā€™s the same reason why I donā€™t run a Docker container on my machine while Iā€™m coding. Itā€™s unnecessary ceremony. When Iā€™ve already done all the things, we ship very fastā€¦ So if I ship a bug, I can very quickly fix it, which gives me the confidence to just go straight live and just see what happens. I donā€™t knowā€¦ Should I be doing these pull requests if itā€™s just like a bunch of little things and I know whatā€™s going on and no oneā€™s really gonna review my code except for myself? Should I be doing that anyways, for some sort of happy trails?

No, I donā€™t think so.

I think thatā€™s one of GitHubā€™s best and worst features - pull requests. Why do I say that? It forces you to create a branch, it forces you to have some code which is running in parallel to the main branchā€¦ And what that means is what do you do with it? How do you deploy? You have to set up a lot of stuff - preview environmentsā€¦ The list goes on and on. Stagingā€¦ Whatever people call it, itā€™s really complicated. What you actually want is to group commits together, and maybe have a discussion on those commits. But they shouldnā€™t be on a separate branch. That would be my ideal, where commits that are grouped together are good, because we can see how something evolves; we can have a discussion about it, and I try this bugā€¦ So how many follow-up pull requests have people done, the ones that are listening to this podcast? Iā€™m sure manyā€¦ Because you ship a pull request, you think itā€™s finishedā€¦ Ah, thereā€™s a new bug. Okay, so thereā€™s a follow-up pull request to my initial pull request, because I forgot about this thing, which you have to merge again, and so on and so forth. So I wish GitHub behaved differently when it comes to pull requests.

Should we dig into that a little bit? How would you do that then? What would be the experience for that? Because wouldnā€™t branches be the most logical way to natively get ā€“ you know, in the Git realm, not the GitHub realmā€¦ Separate your main branch, your master branch, or whatever, from other code. Just to give it its own lane, so to speak. How else would you group commits?

[06:13] Well, if you could attach a commit with some metadata to a pull request, which is just a virtual object, and youā€™re done, the pull request is finished, when maybe you flip a feature flag. And by flipping the feature flag, that feature is live. Everyone has access to it. But you keep shipping into production, you donā€™t enable it for everyone, and when it feels like itā€™s done, youā€™re done. If there are still bugs - well, guess what? You can do follow-up commits, attach it to the same pull request, and itā€™s still like the same logical grouping. I think that would make more sense.

But it still lives in the master and main branch, basically.

Yeah. Everything goes on the main branch, yes

So does the grouping only exist then on the platform? Like, it doesnā€™t exist inside of Git? Or youā€™re hoping this would be a feature of Git itself?

Maybe bothā€¦ I mean, I donā€™t know who has the appetite for this, butā€¦ Pull requests are a GitHub thing.

They donā€™t exist in Git. Branches exist in Git. So if pull requests were first-class in Git, I think that would make more sense.

Right. Well, the very first users of Git did not use historically - Iā€™m not sure what they do now - pull requests; they used patches. So the Linux Kernel, for which Linus made Git to collaborate around, theyā€™re all about ā€“ they were all about (Iā€™m not up to date), emailing patches.

So like you said, pull requests were very much an invention of GitHub, and have been copied; I mean, theyā€™re called merge requests over on GitLab, which I think is actually a more accurate nameā€¦ But sorry, pull request has the inertia, so it just doesnā€™t quite land the same way, but theyā€™re pretty much identical. Thoughts on different platforms, neither of which were a Git-native action.

Yeah. I do like the pull request representation, where you can have a discussion, you can link to other pull requests, you can link to issues, you can solve issues, you can add imagesā€¦ I use that a lot; screenshots, videosā€¦ A lot of things. So itā€™s great for a discussion. I find that richness, sharing that information and being able to refer back to it, so useful.

Right.

And itā€™s like a biography too of what happened. So going back to triage, or learn from mistakes, or learn from positive things, too. You can totally do that with a well-done issue, or a well-done PR.

So that felt like a segueā€¦ Back to the 24 improvements. Iā€™m looking at the list right now, thereā€™s so many things there.

Do you wanna talk about any of these in particular?

Well, which ones do you wanna talk about? ā€¦because theyā€™re all yoursā€¦

Right.

Which felt interesting, which felt unexpectedā€¦?

Unexpected?

Mm-hmā€¦

Iā€™m just kidding. Go ahead. Keep going.

So commit 19, when I fixed the bug, that was unexpected, because I didnā€™t expect there to be bugs in my codeā€¦ Um, so a lot of this was follow-ups to chapters, and kind of finishing or rounding out the chapters feature, allowing for the attachment of images to chapters, adding chapter support in the on-site player, which is still a first draft of the feature, but was a small enough feature that I could do it in an afternoonā€¦

So all that chapter-related stuffā€¦ I also did some kind of one-off things that Iā€™ve been waiting to do. All of our Medium posts go to Scribe now, which is just a quality of life improvement for our readers when we link to Mediumā€¦ Because Scribe is basically ā€“ remember InstaPaper, where you would take a URL and then get the InstaPaper view? Now built into mini-browsers; Safari has the reader viewā€¦ A lot of them have thatā€¦ Scribe is kind of like that for Medium posts, where you can just take the post, direct it to Scribe instead, and it gives it a nice reading experience without all the Medium cruft that we donā€™t like so muchā€¦ Because - gosh, I hated linking to Medium because the reader experience sucks, in my opinion. But there was so much good content there that it was compelling. And I didnā€™t wanna turn people away just because theyā€™re on Medium, but we used to a little bit; Iā€™d email them and say ā€œHey, write this on your own domain and Iā€™ll link to it.ā€

[10:18] Now, this is a much better answer. If the context is good, we wanna be able to link to it, and now we have just a better reading experience. So that was one that Iā€™d been waiting to do, and once chapters was done, I kind of had no excuse not toā€¦

Notably, we switched to ā€“ or we didnā€™t switch to, but we started to integrate the Open Podcast Prefix Project, OP3.dev. We had a whole show of our Backstage podcast talking to John Spurlock all about that project, and we told him at the end of that show ā€œYeah, weā€™ll give it a shot. Weā€™ll send you our data.ā€ And so I did that pretty easy change, but hopefully a profound oneā€¦ And then just a bunch of cleanup; things that we didnā€™t need.

One thing that we get a lot of is spam. And the reason we get a lot of spam is because historically podcast RSS feeds required you to have a working email address in your feed. So editors [at] changelog.com for years and years and years has existed in all of our feedsā€¦ And all these podcast spammers who either wanna come on your shows, or wanna sell you something - they just crawl all the indexes (Apple Podcasts requires this), and they grab all the email addresses of everybody who has a podcast, and they email you as if theyā€™re your best friend. And you know theyā€™re emailing 45 million people, or whatever it isā€¦

Until recently, Apple actually removed that requirement, and so the iTunes owner element is no longer required; it actually doesnā€™t do anythingā€¦ And so we can remove that, which means weā€™ve removed that email address from our feeds, which means less spam over the long-run, which is exciting. So little stuff like thatā€¦

Yeah, a lot of amazing improvements, I have to say. Even though they read ā€“ like, itā€™s one-liners, and I enjoyed very much seeing them in our dev Slack channel as they keep rolling in, thatā€™s very nice. I really enjoy that, by the way. Thatā€™s one of the things that weā€™ve improved for Kaizen 7. And seeing those improvements coming through was very nice. But itā€™s like 10 minutes, 15 minutes here, and they add up over 2,5 months; thereā€™s a lot of stuff there, so itā€™s great to see that. Amazing.

Talking about contributions and improvements, I wanna give a shout-out to Josep, Luis, Giralt, Glakost, Pep for short, maybe I shouldā€™ve started with thatā€¦ Pull request 415. That was opened since May 2022. So May earlier this year. Thatā€™s been open for a whileā€¦ He is a Ship It listener, and I think it was in response to one of the Ship It episodesā€¦ Thank you very much, Pep, for that pull request. And I noticed it in August, and I finally merged it for Kaizen 8. So that made me very happy to get it in.

Now, he was asking whether he can get a Kaizen T-shirt, and at the time we didnā€™t have them in the merch store, but now we do. Now, Iā€™m wondering - you two are already wearing them, so thatā€™s great. Iā€™m still waiting for mineā€¦ That is amazing. Iā€™m wondering if we can send maybe Pep a Kaizen T-shirtā€¦ What do you all think?

Letā€™s do it.

Letā€™s do it.

Ship it.

Okay. Yes, please.

Ship that shirt.

Alright, Pep, so hopefully youā€™re listening to this; and if not, thatā€™s okay. Weā€™ll get your posting address somehow; weā€™ll see how we get that T-shirt to you.

Well, itā€™s easier than that. We just give him a coupon code to the merch store, and they go there and use the code themselves, order whatever ā€“ they may not even get the Kaizen shirt, you know? He may get something else.

We have his email. We can spam him. Okay, so Pep, youā€™ll be spammedā€¦

[13:53] This is the kind of spam you wanna get, you know?

Oh, yes. Thatā€™s amazing. Okay. Cool. Now, someone else that I wish we could send a Kaizen T-shirt to is Noah [unintelligible 00:14:03.00] He has two pull requests. Pull request 428, we got it merged, where Elixir was bumped from 1.13 to 1.14.

Rightā€¦

And Erlang OTP from 24 to 25. Now, if this sounds familiar, thatā€™s exactly what we did for Kaizen 7, the previous Kaizen, and Noah did it for Kaizen 8. How amazing is that?

Now, I tried to merge that one and failedā€¦ I think thereā€™s more to the story here. Maybe weā€™re getting ahead of ourselvesā€¦ But you finally got it ā€“ I couldnā€™t do it; I wasnā€™t cool enough. Gerhard, youā€™re cooler than I am, so it ended up landing eventuallyā€¦ But I think we should definitely send Noah a T-shirt, too. I mean, if Pep gets one, Noahā€™s gotta get one, right?

Definitely. Those are the two noteworthy contributions. Great improvements, thank you both.

Now, there is a related pull request where I explained how to use the Docker that we have deployed on Fly.io to build images. But the gotcha is that if you docker login, and you docker login locally, targeting a remote Docker engine, the login, the auth gets configured locally, not on the remote engine. So what that means is even though you appear to be authenticated, the remote engine cannot push images, because remotely it doesnā€™t have that auth token, so it canā€™t talk to the registry, in our case Docker Hub. So all of that is going to improve; all of that. But I think weā€™re gonna leave it for a bit later.

Okay. So letā€™s focus in on this Fly/Docker situation, because this is very coolā€¦ This solved a painpoint in my life, which has been there for years, which is local Docker for Mac slowness, having to have it installed, having to upgrade it, deal with itā€¦ And just the disk space usage of that sucker has been significant. And I donā€™t have to do that anymore, because we run our Docker on Fly, and I just set like remote server or something; thereā€™s some sort of environment variable you setā€¦

Docker hostā€¦

Docker host.

Export Docker host, yeah.

And because itā€™s client server software, I just have to run the client, the server is on Fly, all of the nasty bits that I donā€™t want on my machine are on Flyā€¦ And thatā€™s just super-cool. Iā€™ve been using it a lot, because Iā€™ve been setting up some Mastodon stuff on Fly, which was using Docker containers, in addition to our stuffā€¦ And thatā€™s super-cool. Is this just common practice now, that I wasnā€™t aware of, Gerhard? Was this a grand idea of yours? Whatā€™s the deal here?

Yeah, I donā€™t think itā€™s common practice. This is actually, if anything, not common, because of issues like I mentionā€¦ Like, your client is authenticated locally, but the remote engine isnā€™t, and weird things like that; definitely edge casesā€¦ And the reason why I set this up was for our CI. So I think Kaizen 7 or even Kaizen 6 - we had the issue where we were targeting the same Docker engine; that was running in my shelf, I kid you not. Itā€™s a very beefy machine, but I just run it in my shelf.

Thatā€™s right. It was at your house.

And the GitHub Actions runner was connecting using Tailscale to that host. But the problem is I was using the same Docker engine for my local development, this, that and the otherā€¦ And then there issues when the CI was trying to use it, too. So again, thereā€™s a pull request; I will link it in the show notes.

So to fix that, to improve that situation, Iā€™ve configured a Docker engine on Fly. Again, I think it was in a previous Kaizen where this was mentionedā€¦ And I added pull request 426 - thatā€™s the last one, September 22nd; thatā€™s when it was merged - to basically capture how to use that Docker engine on Fly.

So not only our CI is using it, we are using it too, but again, we have to be careful with that, because we can create issues if we share the same Docker engine. So thatā€™s something to improve further, for sure.

[17:50] Well, itā€™s a cool hack, and I will say, itā€™s a better hack than the previous one, which was to run it from your shelf. You know, weā€™re pretty open here, and open to criticism, open to ridiculeā€¦ You know, our code has always been open, our infrastructure has always been openā€¦ We donā€™t get easily embarrassed; that one is like darn near ā€“ like, seriously, guys? Youā€™re running your production business websites and stuff with a ā€“

Just a CI. Itā€™s okay. [laughs]

ā€¦just a little bit of that infrastructure in Gerhardā€™s office?

Yeah, pretty muchā€¦

ā€¦or closet, or wherever that shelf existsā€¦ So Iā€™m glad ā€“ I think thatā€™s definitely a Kaizen. Weā€™ve continuously improved from there, and weā€™ve moved it off there into a Fly VM.

This is going to improve even further, but I think we should leave that to a bit later; Dagger has a lot to do with that, and all this year has been about Dagger for me, soā€¦ Anyways, thereā€™s something big coming, so letā€™s leave it for a bit later.

An even bigger improvement from Noah that is waiting to be reviewed and merged is pull request 430. He added support RSS feeds for specific topics, especially combining current feed and topic functionalityā€¦ Now, this was branched off pull request 390, that was opened last year by [unintelligible 00:18:58.24] So thereā€™s two PRs in one, and I think we mentioned this briefly, maybe offline. What do you think about that one, Jerod?

Iā€™m definitely down with the feature. I think I approved the feature conceptually years ago, when [unintelligible 00:19:16.05] began working on itā€¦ And I think he got some stuff working. It wasnā€™t quite thereā€¦ I think the implementation - I code-reviewed the implementation and said ā€œWhy donā€™t we do it this way? We already have a feed controller. Letā€™s just reuse that, versus putting it over in the topic controller area of the codebase.ā€ And then life happened, and he was just never able to get back around to it, never passed my threshold of things that Iā€™m gonna actively work on, so it just kind of sat there. So it was super-cool to see kind of a hand-off of this feature to somebody elseā€¦ And I havenā€™t looked at this PR recently, but spiritually, Iā€™m there. Iā€™m definitely gonna merge the feature when itā€™s ready to go. I think itā€™s cool. Certain people just care about certain topics, and it was like ā€œWhy not have a feed of ā€“ā€ We have feeds, theyā€™re just HTML feeds of topics. So why not have RSS feeds of those topics? Itā€™s a no-brainer, so - happy to have it in there.

Itā€™s a long-time WordPress feature, basically, to have your tag or your topic be feedable, essentially. So I agree with the usefulness of it. Because for a while there when we were on WordPress, we would encourage that. Thatā€™s [unintelligible 00:20:20.09] tracked Ruby by just tracking the tag via RSS, that way you donā€™t have to get the whole kit and caboodle; you can just get the one thingā€¦ Which is what kind of everybody wants, right? Just a slice, not the whole pie.

A similar feature is just author-basedā€¦ Now, we donā€™t post that many posts, but even news perhapsā€¦ Publications bigger than ours, Iā€™ve definitely had certain authors that I appreciate, but Iā€™m not gonna subscribe to The Vergeā€™s feed. Iā€™m just not gonna do it, because theyā€™re gonna publish 12, 15, 20 things a day, and overwhelm me. But this author - I wanna know when they write something new. And The Verge did not provide that as a feature. I think Feedbin actually allowed me to filter down a feed, which is kind of coolā€¦ But author-based feeds - I think thatā€™s something that WordPress also does by default, and I would definitely appreciate that, even on oursā€¦ Even if two people use it. Still, itā€™s a pretty easy feature once youā€™ve already done it for topics; itā€™s the same concepts on a different objectā€¦ So I would also merge something like that.

Nice. Does it mean that Kaizen can have its own feed?

Kaizen can have its own feed, because thereā€™s a Kaizen topic now, right?

Yes, there is.

Yup, so thatā€™s all it needs.

Okayā€¦ Thatā€™s really cool.

Yeah. Like, if you only like the Kaizen episodes - boom. You donā€™t like any of Gerhardā€™s interviewsā€¦ Whatā€™s wrong with you?

Exactly. I donā€™t know. I donā€™t know. But thatā€™s fine. [laughter] I run things on my shelf.

The consumer isnā€™t always the listener. It could be somebody whoā€™s publishing a web page. Similar to the way we did when we were on GitHub [unintelligible 00:21:49.05] forever ago. Like, they would just pull from the podcast tag only. There was one feed, so they would filter it. So there could be a usefulness, where itā€™s like, not not liking itā€¦

So Iā€™ve been dabbling into our Spotifyā€¦ This is a bit upstream from where we were, butā€¦ You know, Spotify has a lot of people who never discovered us before, so like, you wanna be there, you wanna be visibleā€¦ You know, new people listening to your stuff is coolā€¦ So they have playlists inside of Spotify, and Iā€™ve been over there creating certain playlists. Like, itā€™d be cool to have a Kaizen playlist in Spotify, but I donā€™t wanna go into Spotify and create a Kaizen playlist, and manage itā€¦ Itā€™d be cool ā€“ this is never gonna happen, but a Spotify robot is what we need to basically mirror our topic feeds into different indexesā€¦ And then you could have ā€“ for instance, our dev game shows is a manual playlist I created over there for our Frontend Feud episodes and our Go Panic episodesā€¦ And it exists over there because I go over there ā€“ we do them infrequently enough that I go add them. Iā€™m like ā€œHere it is. Itā€™s dev game shows.ā€ Well, that should be a topic on our platform that then just gets syndicated into places, and then weā€™d be cooking with gasā€¦ A Kaizen feed on-site is awesome, but if that could be mirrored into Spotifyā€¦

Oh, yes.

ā€¦even cooler.

We can dreamā€¦ We can dream.

I would definitely send a Kaizen T-shirt to a pull request that did something like that.

Yeah, the Kaizen is young; Ship It is young. Lots of things could happen, yeah.

Yeah. A publishing bot.

Cool. Now, all these improvements, all this talk of improvements - Iā€™m sure you, dear listener, are thinking ā€œOkay, I want to contribute. How do I do that?ā€ Pull request 434 was created for that. Itā€™s a new contributing guide, and it includes how to set up our app locally for our development. It does not involve Docker.

No Docker.

So Brew, thatā€™s all you need, and I did this on a clean Mac, macOS 12. Linux help is wanted. So if you want a Kaizen T-shirt, thereā€™s a hint for you.

[laughs] These are like ā€“ you put a Kaizen shirt on a stick and dangle it in front of people.

Pretty much. [unintelligible 00:25:56.09] [laughter] Well, by the way, you can also buy them. You know, this is a free one, just to make it clearā€¦ How many do we have on the store, Adam? Whatā€™s our stock level like? Do we still have any left, by the way?

[26:09] We doā€¦ Itā€™s not massive, but itā€™s enough.

But now that we have this all set up - I mean, re-stocking is easier than it was the first time, when we had to actually create the design, submit the designā€¦ Now itā€™s kind of in the system, and re-stocking is not a huge problemā€¦ But weā€™re definitely getting low, because theyā€™ve been on the store for maybe six weeks, and weā€™ve just started talking about it, and theyā€™ve been selling. Soā€¦

Okay. Nice

ā€¦either buy now, or code nowā€¦

Yeah. [laughs] Thatā€™s a good one. Contribute. Contribute to changelog.com, the repo.

So weā€™ve got ā€“ yeah, weā€™re getting kind of low already. Weā€™ve got seven each of large, medium and small, 11 of XL, and 15 of 2XL. But I think that 2XL is not correct. [unintelligible 00:26:55.16] so I think that could be more like 11 of that one, too. So weā€™re pretty shallow on our stock for it. So if you wanna get one, contribute, or if you wanna buy one, do it now.

Plus, Gerhard needs one.

One? I thought four. Iā€™m getting four. Five, actually. For every day of the week.

One for each.

Yeah. One for each. Cool. So coming back to 431ā€¦ I know that, Jerod, youā€™ve reviewed itā€¦ You had the question about the DCO, why do we need it and what it isā€¦ What did you think of that PR overall, apart from the DCO? We can go into that next.

Yeah. No, I think itā€™s great. I like the idea of putting all of our contributing stuff in its own file, its own place in the repoā€¦ The readme was getting excessively wrong. It was pretty outdated. It probably still kind of worked, but nobodyā€™s really actively using that flow. We all develop locally anyways, so why have a flow for contributions thatā€™s different from what we do?

I think the reason for that was cross-platform compat. We want everyone to be able to contribute. But in reality, it wasnā€™t good. So this is better. I do think itā€™s a step. I would rather have a better story, and I think thereā€™s lots of cool opportunities. Have you guys seen Devenv? Thatā€™s a Nix-based one that I saw just yesterday. I put it on Changelog News. Potentially a way that we could do itā€¦ Itā€™s cross-platform. Itā€™s native, itā€™s not inside containersā€¦

Now, all of our stuff is pretty stock. Like, okay, Elixir and Erlang is a little bit rare, but other than that, itā€™s Postgres, and itā€™s ImageMagick and Node, right?

I donā€™t think ā€“ do we even need ImageMagick? I donā€™t think we need it.

I mean, you donā€™t need it for development unless youā€™re actually working on that feature. The Apple run without it. But you do need it just for like avatar resizing and stuffā€¦ So to be 100% running the site you would need it, but for 99% you would not need it.

Cool. So what Iā€™m going to do now is do ā€œRemember to add ImageMagick.ā€ Iā€™m leaving myself a comment.

On the PR.

On the PR.

Put that in the contributors guide as a dependency.

Iā€™m looking at the readme that you changed, so Iā€™m excited about this, because Iā€™ve been waiting for a more thorough guide, because Iā€™m tired of asking Jerod, and heā€™s probably tired of asking me to help me set up my local setup, and itā€™s always like ā€œDo I have all the right steps? Help me understand what the specifics are like. Do I need ImageMagick, for example?ā€ What do I need to have to just contribute small things, like footer changes, or other changes that I can take care of, that I tend to push out, because I donā€™t have a dev environment set upā€¦

So Iā€™ve actually been digging into my own dotfiles and automation and stuff like that, because I just did a clean install of a Mac, just for the fun of it; so weā€™re kind of like right in the same spot, Gerhard. So Iā€™m gonna go through this myselfā€¦ And Iā€™ll be your first Guinea pig, because Iā€™m excited to do this.

You know what - Iā€™m going to add you as a reviewer on this pull request. See? Pull requests are really useful. Super-useful for stuff like thisā€¦ To get people to look at it, try it out, see what worksā€¦ Like have that conversation, which is around a specific issue.

[30:05] To be honest, I would have liked this to be already in; I would have preferred thatā€¦ Because if something is not right, thatā€™s okay; we can just iterate in the final place. We donā€™t have to keep it on the side, because itā€™s just the inventory.

Yeah, we can merge it, and then if you find changes, follow up.

But then we need to submit a follow-up pull request, so thatā€™s the most annoying partā€¦

Ah, come onā€¦

Oh, my goshā€¦ This is where I guess it would be nice to live in master or main, depending upon the age of your repo, for example, where your initial idea we talked about early in the show kind of plays [unintelligible 00:30:32.26] where itā€™s like, separate branch, separate pull requestā€¦ It kind of gets kludgy UX-wise.

A question on the DCOā€¦ Is that in line with or similar to being verified, like your commits being verified? I know that this is different, but is it in the same realm? Would we like to have verified commits? Because I know, Jerod, youā€™re not a verified committer. You commit code that ā€“ youā€™re not verified.

Iā€™ve got a bunch of open tabs right nowā€¦

Okayā€¦ I can help you with that. Itā€™s really easy. If you already have 1Password, itā€™s ever super-easier. Theyā€™ve made some cool stuffā€¦ I love the 1Password SSH integrations; if you have a TouchID Mac and youā€™re not doing 1Passwordā€¦ If youā€™re familiar with this, Gerhard, please praise it up, because I love it.

You literally biometrically sign your commits with your finger. And you can SSH into a machine, or commit to your Git repo even locally before you push it, your commit, locally. [unintelligible 00:31:25.25] is signed via a biometric press on 1Password. So cool. I donā€™t recommend it for machines that are remote, because you literally have to have the UIā€¦ Because if not, youā€™ll get stuck in the prompt. But then they have some instructions on how to sign your commits, which is like a couple of additions to your git configā€¦ And youā€™re off to the races. Itā€™s really, really easy.

Well, Iā€™m smelling a pairing session hereā€¦ And if not, then Iā€™m disappointed. [laughter]

Yeah, itā€™s super-easy, and Iā€™m loving it, honestly. Iā€™m just so stoked with the stuff they did. I have some improvements Iā€™d love to see them doā€¦ But theyā€™ve made some great strides already with biometrically signing Git commits, and SSH-ing and using your SSH key. So cool.

Now, canā€™t I just take my GPG key and add it to Keychain, and not have toā€¦?

Thatā€™s the old way. The new way is using the SSH key. So you can sign using the SSH key. GitHub supports it. GitHub will verify your signatureā€¦

So I already have an SSH key, is what youā€™re saying.

You do, exactly.

Iā€™ve got a 10-year old SSH key just sitting right there.

Thatā€™s it. You put it in 1Password, do the configurationā€¦

Oh, I was just setting up a GPG right now. So I didnā€™t need to set up this GPG key?

No. You donā€™t. No, no.

Doggone itā€¦

Thatā€™s the old way. Thatā€™s the old-school way.

Do you rotate your SSH keys often, Gerhard?

Sore subject. Sore subject.

Sore. Thatā€™s how we started. [laughs]

I was gonna say, thatā€™s a long ā€“

Six years ago. That was like ā€œHey, Jerod, what should we do with this old key? Youā€™ve gotta SSH.ā€ [laughs] Thatā€™s how old it is.

I told you, weā€™re not easily embarrassed. In fact, it was so oldā€¦ Was it Ubuntu - it wouldnā€™t let me SSH in.

Exactly.

It was like, ā€œNah. Your algorithm is too old.ā€

Itā€™s too old, yeah.

Itā€™s like, ā€œA 10-year old SSH key? Iā€™ve never seen such a thing. What is this thing?!ā€

I connect with so many machines. I donā€™t wanna have to rotate that thingā€¦ Not anymore. Now I connect to like nothing. But back thenā€¦

Tell me if this is overboardā€¦ So the one thing I donā€™t like about 1Password currently is that if you create the SSH key within it, it doesnā€™t let you create a password, which I think is super-weird. I like creating pass phrases for mine, but the user experience makes you put it in all the time. But I like to generate a 100-character passphrase for my SSH keyā€¦ Because whoā€™s ever gonna crack a 100-character passphrase? Itā€™s probably just impossible. But is that what you do, Gerhard? How do you deal with these little things like these? Do you passphrase every SSH key? Of course, right? Thatā€™s the best security. Because if not, you only have the keys themselves to authenticate; if you donā€™t have a passphrase, itā€™s just ā€“ pair them up.

[34:03] So I used to have my YubiKey part with the whole SSH, and the YubiKey would use to generate the same passphraseā€¦ So my passphrase from my SSH keys used to be something that I knew, joined with what my YubiKey was generating.

I see. So something memorable.

Exactly, yeah.

So my brain is 1Password, basically. Like, I skip all the memory and me; I donā€™t have a clue what my passphrase is.

So thatā€™s what I used to have. And the problem was there were issues with the YubiKey. It was not just the YubiKey, but it was like the software. The support was a bit patchy, and then there were like new upgradesā€¦ So I had this physical keyā€¦ Iā€™m still using it for 2-factor authentication as a back-up; I also have my phoneā€¦ So if my phone gets lost, my YubiKey has all the OTP tokens. So thatā€™s a separate issue. And also now 1Password. So I basically have triple redundancy for my [unintelligible 00:34:54.27]

Of course you do.

[laughs] As you would, right? Exactly.

As YOU would.

Yeah, as you would.

No, not meā€¦ As YOU would, Gerhard. [laughter]

Careful.

So coming back to 1Password, I do add my SSH keys there. You were mentioning not having passphrasesā€¦ I canā€™t remember that. That was something recent which I changed.

Yeah, if you create it within 1Password, there is no passphrase created. It just creates it.

Now, I prefer to do it via [unintelligible 00:35:26.10] myself and import it, because I can create the passphrase myself. That could be maybe I donā€™t understand how to use it, but I donā€™t think so; Iā€™m holding it wrong potentially, but I donā€™t think it creates a passphrase if you create the SSH key inside of 1Passwordā€¦ Which you can do. I think itā€™s super-cool, because ā€“ I mean, for those who donā€™t wanna touch the command line, thatā€™s a GUI, essentially, for creating the SSH key. And you could choose RSA, you could choose Curve25519, whatever you wanna choose; you can go either direction. But I like to do it via [unintelligible 00:35:55.09] because I know I can create a passphrase there, and that I just import it into 1Password.

Yeah, thatā€™s what I do, too. Thatā€™s exactly what I do, too. Yes.

Iā€™m curious about your OTP stuff with the YubiKey, which Iā€™ve never set upā€¦ But in terms of an app authenticator, Iā€™ve been using UI Verify; I donā€™t know if you use this or not, but UI Verify to me is the best. I donā€™t know, will that focus for you guys?

I can see it. Thatā€™s really cool.

Oh, thatā€™s my face. Sorry. It tried face ID firstā€¦

Yeah, I havenā€™t heard of UI Verify.

Thatā€™s all of the things I have in there, which you can sort of seeā€¦ That blue line up top will rotate throughā€¦ You can see how long you have to put the code inā€¦

Whereā€™s yours, Jerod? Whereā€™s yours?

For those on audio, Gerhard and Adam are holding their phones up to their cameras to show each otherā€¦

[laughs] The OTP tokens.

ā€¦their current OTP tokens.

Thatā€™s right.

Quick, type that in before it rotates.

Yeah. [laughter]

So hereā€™s my problem with those thingsā€¦ I upgraded my phone and I inevitably forgot that I need to back that sucker up, and I wiped my old phone, and then I lost access to like the six most important things in my life. And then Iā€™ve gotta go to that site and Iā€™ve gotta say ā€œHey, itā€™s me, and I lost my auth token verifier app.ā€ And theyā€™re like ā€œHere, send us a picture of your photo IDā€, and I do that six times. And I said to myself, ā€œThis might be worse [unintelligible 00:37:22.01]

Thatā€™s fine, you need a backup. You need a backup, seriously.

Aahhā€¦!

Yeah. So you have it on your phoneā€¦ 1Password. 1Password supports it, by the way. So you can have your OTP keys in 1Password, on top of your phone.

I saw that, but I didnā€™t know how it worked, and I was already using UI Verify, so I was like ā€œOh, Iā€™ll just use this.ā€ So I love that youā€™re ā€“ weā€™re gonna point to blog post or something, documentation that youā€™re aware ofā€¦ Because I was like ā€œOkay, cool, but I also have it here already, so Iā€™ll just keep using what Iā€™ve got.|

Yeah. Letā€™s share them post-show. But you definitely want ā€“ because as you mentioned, if you upgrade your phoneā€¦ If something goes wrong, your phone gets stolen, what do you do then?

[38:01] Yeah, youā€™re totally locked out. You donā€™t regularly back up your UI Verify, do you, Adam? If you donā€™t, youā€™re gonna be in the same boat Iā€™m in when I forgot to and upgraded. And Iā€™m like ā€œDang it, thereā€™s all the serious thingsā€¦ I canā€™t get into the most important things.ā€

Thatā€™s true. If I lost my phone and I wouldnā€™t be able to move the data overā€¦ You know, Iā€™ll have to test that theory. But for me, my most recent phone upgrade, I just moved the data from one phone to the other. I didnā€™t do it through the cloud, I went from phone to phone, waited three hours, was patientā€¦ Because I feel like thatā€™s the best way to get everything.

Yeah. I do a restore from backupā€¦

I donā€™t like backups. I like to go phone to phone, direct. Itā€™s problems with the cloud.

Thatā€™s yucky though, because any problems you had on that on that old phone are now on your new phone. Whereas if itā€™s just the backup, so the things that are good, itā€™s cleaner. But you donā€™t get that UI Verify, orā€¦ I was using Google Authenticator; same idea, right?

Well, it did come over. That was thing with this time; I was sharing that with you. It was like, when I moved my data from my phone to the new phoneā€¦

Well, phone to phone you would, yeah. But from the cloud you wouldnā€™t. Or I didnā€™t. And thatā€™s how I lost access to so many things.

Well, thereā€™s definitely some holes in the process, for sure. What I like about it, and what Iā€™m sad that you donā€™t embrace this because of all this pain you feel, is that really itā€™s like the best way to be authenticated. Because your face is your face, your phone is your phone, as you knowā€¦ And to me, things like GitHub, especially when youā€™ve got supply chain security issuesā€¦ To me, if more people verified their commits, took the time to do ā€“ maybe we should write this blog post, Gerhard, because I feel like it could really help.

I think so.

I feel like if more people did just the verification on the Git commits, which is tied back to your SSH key, and thereā€™s many ways you have to put that key into GitHub, it has to authenticate with themā€¦ Then they have the ghauth locally, which you can sort of authenticate to as well, which is all part of this processā€¦ It verifies your commitsā€¦ Same thing for your SSHā€¦ Itā€™s like, in 1Password itā€™s gonna be super-easyā€¦ But then also, this 2-factor authentication, these OTP codes - to me, thatā€™s the best way to ensure itā€™s you. And itā€™s worth the pain. I wouldnā€™t do it for every app in the world, but more and more supported, so Iā€™d just embrace it. GitHub in particular. Cal.comā€¦ Like, I donā€™t want people to not be me on Calendar; so Cal.com is there, Slack is there, GitHub is thereā€¦ Unify, Ubiquity, thatā€™s thereā€¦

Mm-hm. Thereā€™s still one pull request which I didnā€™t have time forā€¦ But between this episode recording and the episode shipping, thereā€™s a week. So a week is like a year in my world, so Iā€™m hoping that this PR will landā€¦ And thatā€™s basically the migration between LastPass and 1Password. Weā€™ve mentioned this in Kaizen 7, I think even before weā€™ve talked about it. So I think part of that we can also help everyone configure SSH via 1Password for those that use it. I use it, you use it, Jerod will use itā€¦ Yes, heā€™s noddingā€¦

Well, we have it, so he should use it. I mean, itā€™s too easy.

Exactly. Yeah. And then once we do that, all the secretsā€¦ By the way, they are already in 1Password. I migrated them from LastPass to 1Password. I just havenā€™t finished all the integration with our infrastructure. With Fly, all of that stuff. But I think part of this should be how to configure your SSH key, how to sign your Git commitsā€¦ Because itā€™s all this one tool. The problem is we are depending on a lot of things on 1Password. So what worries me is what is the backup plan; if that was to fail, what do we do?

Iā€™m just being paranoid, Iā€™m sureā€¦

Because it is cloud-based, right. If they were attacked, if they were compromisedā€¦ It is a centralized point of failure, and it is cloud-based, so there isnā€™t local copies. You have to authenticate to the cloud, and itā€™s all internet connection-required, right?

We do have local copies, but itā€™s all synced from the cloud. So if the cloud gets corruptedā€¦

Right, precisely.

ā€¦the vaults would get corrupted. All the local vaults.

[41:59] I think youā€™re knocking on the door of an episode on Ship It with them, because I think ā€“ Iā€™ve got questions for them. If you wouldnā€™t mind, Iā€™d join you on that show, if you were open to a guest host, or whateverā€¦ Because Iā€™ve got questions. Thereā€™s some things around the SSH authentication stuff, Iā€™ve got questions about improvementsā€¦ Iā€™m just curious about where theyā€™re going with it, because I think itā€™s great how it is currently; but if there is improvement (Kaizen), then Iā€™m just curious how we leverage it in our infrastructure, and then also individually as users.

I like it. Definitely. So thatā€™s coming up in PR form, and also in Ship It episode form.

If they say yes.

Well, someone will say yesā€¦

Someone will.

Someone that knows 1Password really well will say yes.

Weā€™ll take anybody. The janitor, you knowā€¦

Exactly.

ā€œWhat do you know about the SSH integration?ā€

ā€œWhat is SSH?ā€

One thing which I havenā€™t mentioned, and I wanted to; I was convinced that weā€™d start the episode with this - it was the GitHub discussion 433. A lot of things to be talked about, and other things are in that GitHub discussion. This is the first one of the kind. Itā€™s on our GitHub, and it contains all the links, all the references, all the comments for what we talked about in this episode.

Iā€™m trying something new; itā€™s an open-ended discussion. You can ask questions, you can even maybe suggest someone from 1Password that wants to join us on that episode. So everything is there. And itā€™s something that can be open for a while. We donā€™t have to close it. We can leave it open. And as we start preparing for Kaizen 9, we could create another discussion for Kaizen 9 ahead of time, just to give people a bit more visibility as to whatā€™s coming, for our most loyal Kaizen followers. Just an experiment; weā€™ll see where it goes. But thereā€™s a lot of stuff there.

Now, I must mention the diagramā€¦ But before I do so, does anyone want to mention anything else about this discussion?

Well, I wanna point out a few things for it. So when you go to this discussion, what Gerhardā€™s saying is like for each Kaizen weā€™re gonna have an open discussion; where normally these details were behind the scenes, sort of as preparatory show notesā€¦ They didnā€™t live in the wild, basically. So what heā€™s proposing and what weā€™re doing with this one as an experiment is to put it out there in the open, so we can use it, but then you can also follow along. But one thing I see there when you go to this page is this tag up there that says ā€œAlmost there.ā€ What is this UI? Tell us about what you see there to know, like, is this current, is it stale, is it old? How do you know the status of this discussion, so that you can sort of play a part?

Oh, I see. Okay. So thatā€™s using shields.io, and I think weā€™re also using the readme, which is basically those ā€“ how do you call them? Itā€™s not bubblesā€¦ What do you call them?

Badges.

Badges. Thatā€™s it. Thank you. Itā€™s the badges that usually people put in readmes, on GitHub you see them in repositoriesā€¦ So thatā€™s a badge, and itā€™s like of a certain style. Itā€™s like a traffic lights system, so itā€™s red when itā€™s work in progress, a Stop sort of thing; we are very heavily working on it. Orange - get ready; almost there. Weā€™re recording it, or we are very close to recording it. And green, once the episode ships. So thatā€™s what Iā€™m thinking.

[48:18] Isnā€™t there labels in there and stuff like that too, though? Couldnā€™t you do that with labels? Because thatā€™s like static in the ā€“

Yeah, I could.

Right? Like, youā€™ve gotta manually update. Youā€™re gonna have to go back there and say ā€œLast updated on November -ā€ not 15th, like it is nowā€¦ 17th, or whenever you next go up there, and then change it from saying ā€œAlmost thereā€ to ā€œShippedā€, orā€¦ So itā€™s a very manual thing. Is there a way to do some of this stuff with like labels, and other things, orā€¦?

Possiblyā€¦ I mean, Iā€™m adding a lot of content anyways to that description; so that is iterating, so changing a few characters is the least. Thereā€™s so many other things happening behind the scenesā€¦ Because Iā€™m editing that discussion, the first one; Iā€™m editing the first post, or the first comment, which starts the whole discussionā€¦ And a lot of stuff is being added, which is basically a summary of everything that we did in 2,5 monthsā€¦ And there tends to be a lot of stuff.

So labels - yes, maybe. It might work. But I think a badge - itā€™s good in that we are used to seeing badges on GitHub readmes: build is passing, build is failingā€¦ Things like that. So that is meant to signal the state of something. And I think a traffic light system is what we are familiar with from the real world, so I quite like that.

So if it was green, itā€™d be like ā€œThis is shippedā€ or ā€œThis is sort of done and baked. Feel free to comment, but whatā€™s planned here is now done.ā€

And again, I havenā€™t figured out all the stages that the discussion goes through. Iā€™m still figuring it out.

Right. These Mermaid flowcharts, though - Iā€™ve seen lots of thingsā€¦ Even GitHub Universe came up recently with blocks, and stuff; this is similar, a lot of interactive things you could put in here. This is a Mermaid flowchart, like youā€™re talking about. Itā€™s Markdown ā€“ itā€™s text.

That is one of my favorite features. Oh, yes. It is text. It is all text. So Iā€™m using Mermaid.live to have that live continuous feedback loop as I work on the chart. So you can see how the chart changes in real time as you change something in the graph itself. The chart that Adam is referring to - and youā€™ll see it in this discussion, GitHub discussion 433 - shows our current setup. All of it. Fly, GitHub Actions, Sentry, DockerHub, LastPassā€¦ Everything. All the services that we use, how the app runs, PostgreSQL replication, the whole thing. That took quite a while to build, but Iā€™m very happy with it because it captures a point in time, a snapshot of what we are currently running.

And OP3 was mentioned. Now, thatā€™s interestingā€¦ I didnā€™t know how that interacts. So things like that we can add to this diagram, and we can also have links. So that means some of these elements - you can click on them and all of a sudden youā€™re in Fly, looking at the metrics that are for the app, directly from this. Obviously, you canā€™t authenticate; if youā€™re not part of our team, you canā€™t access thatā€¦ But if you are, just click on that and there you are.

Also, when things change, we can submit a pull request that shows what changed in that chart. So this is something which hasnā€™t been open yet. This is a PR which weā€™ll ā€“ Iā€™m thinking InfrastructureMD. And InfrastructureMD will have this diagram to begin with, that will capture our setup. I think thatā€™s really nice. We can tag itā€¦ Kaizen 7, Kaizen 8ā€¦ We can see how these things change over time. I wanna try it out.

The end result of this is super-cool. Definitely check the show notes, or maybe we can put it even as the chapter image of right now, so people with chapters can look at it.

The text is complicated, manā€¦ How did you learn to write this? Or did something generate this based on drag and drop things, like draw.io, or you wrote this yourself by hand?

I wrote this myself, yeah.

Handcrafted Mermaid text.

Handcrafted, exactly. All of it. Yup.

No. Nothing.

[52:15] Well, if you consider your editor a GUIā€¦

So on one monitor I had Mermaid-js.github.io open, which is a reference for all things Mermaid. I had my flowchart open and I saw examples thereā€¦ And on the other monitor I had Mermaid.live, which was split in half. The left-hand-side half was the diagram, which I was writing; the actual code, if you wish, the Mermaid DSL. And on the right-hand-side I could see a live representation of every single character press. Itā€™s really nice. So I had instant live preview.

So as I was trying to capture the things which I was trying to put in this chart, I could see how it changes the graph. And once I was done, copy-paste it in our GitHub. That was it.

So is Mermaid.live a startup, is it an open source thing?

I donā€™t know, I just found it. ā€œOh, this looks interestingā€¦ā€ For hours and hours I was engrossed in writing Mermaid; learning and writing Mermaid.

Is that a third-party thing? I guess Iā€™ll just go to the web URL and see what is thereā€¦ But is that part of the flow of this? Or itā€™s just like they created a spec and some tooling, and then GitHub integrated their tooling? I donā€™t know what Mermaid.live is. Hold on, let me look it up.

Yeah, letā€™s look it up. I know itā€™s like ā€“ yeah, letā€™s check it. I used it, and it worked really well. It was like ā€“ I donā€™t know who created it; itā€™s like a live editor, Mermaid Live Editor. I think itā€™s hosted ā€“ this site is powered by Netlify, thatā€™s what I see. But I donā€™t know whoā€™s behind the service, to be honest.

Yeah, so when you load it up, it literally is the editor. Mermaid.live takes you to the editor that Gerhard just described. But thereā€™s no like About Us, or Pricing page, or anythingā€¦ Which I guess is kind of a good sign. Clicking through to the GitHub, itā€™s Mermaid-js thatā€™s in charge of this thingā€¦ And they donā€™t have much about them either. Thereā€™s a Twitter handle that says ā€œCreate diagrams and visualizations with text. Official handle of the MermaidJS open source project.ā€ So maybe that explains it right there. It is an open source project with a domain that hosts the editor, where you can get your Mermaid stuff. And then you just copy-paste from that into your markdown inside of GitHub, and it renders.

Thatā€™s right.

This is a blog post, Gerhardā€¦ This is a blog post.

I think it is, yeah. For sure. We used to do these every year, but I think thereā€™s another one coming, for sure. December will be a good month, thatā€™s what Iā€™m thinking. Reminiscing, capturing ā€“ a good retro month, and capturing all these things.

So the one thing which I wanna mention is that the Mermaid in GitHub ā€“ and by the way, this GitHub markdown has native support for Mermaid. So we have like the [unintelligible 00:54:58.08] and you define it as Mermaid, and then you start writing Mermaid. And itā€™s like code; like, any code that you would put. So thatā€™s really nice.

Mermaid.live has some extra features. So if you copy our Mermaid from this discussion and you put it in Mermaid Live, youā€™ll see some extra things, like for example icons. Fav icons for Fly, Phoenix, things like that. Yes.

Oh, snap.

So when youā€™re in the discussion you have that copy, so thereā€™s like the two arrows pointing in left and right; that expands the chart. And the button to the right is the copy button. So if you click on copy, you can go to Mermaid Live and you copy-paste that chart, see what happens.

Trying it, trying itā€¦ This should have a button that instead of Copy says ā€œOpen in Mermaidā€, or something.

That would be nice.

Uh-ohā€¦ It says ā€œLong diagram detectedā€¦ā€

Turn off autosync? Okayā€¦

Yeah, I think it disables autosync. So you need to enable autosync, which is the toggle just above the codeā€¦

ā€¦and then you refresh.

Cool. So for instance, DockerHub has the Docker icon.

Yeah. Pingdom has it, Grafanaā€¦ A bunch of things like that. So I really like that.

Fancy-schmancy, Gerhard. Youā€™re really going to the next level. You nerded out on thisā€¦

So some of these things can even be animated.

Oh, boyā€¦

I didnā€™t have timeā€¦ Exactly. Arrows that move and changeā€¦ And before you know it ā€“

You can show the path of a request through our systemā€¦

Yeah, exactly. That would be really nice, yeah. So thatā€™s coolā€¦ Iā€™m glad that you enjoy it.

I do enjoy it.

I enjoy how much you enjoy it.

You have such patience for things like this. I do not have the patienceā€¦ Well, sometimes I do. But I can see how you have good artifacts that come from obsessions. Thatā€™s what this is.

Isnā€™t this like a ā€“

Passion for detail, thatā€™s what I would call it. You wanna go further, and you keep refining and refining. I mean, you donā€™t do it in one session. Maybe you have three sessions. An hour, an hour-and-a-halfā€¦ ā€œOkay, so how can I improve this a little more?ā€ Thatā€™s what it is. And you keep refining it, and you see that finally, like ā€œWhoa! This thing is amazing.ā€ And I spent maybe six hours in total, but broken down in 3-4 sessions.

Right. And thereā€™s the little wins along the way, which gets you to the next spot, and then you realize you can [unintelligible 00:57:17.13] This is a quintessential yak-shave, isnā€™t it? Like, you just shaved this yak all the way down to its bareā€“

Pretty muchā€¦ [laughs] Itā€™s a nice yak.

Itā€™s a mermaid.

Itā€™s a mermaid, thatā€™s a good one. [laughter]

Cool. Very cool.

Now, letā€™s talk about something that in a way captures my whole year. I know this was not meant to be retrospectiveā€¦ This is the last Kaizen, and I saved the best for last. So one of the things that weā€™ve been using since November 2021 was Dagger for our pipelines. Why Dagger? Well, it allows you to run CI/CD locally. It allows you to run the pipeline locally. And the reason why thatā€™s nice is because you donā€™t have to commit and push to see what is the effect of this change. You can run the pipeline locally.

And then when it comes to commit and pushing, guess what - your CI/CD system, whatever it is, it will run exactly the same pipeline.

In November, episode 33, Ship It #33 - you can listen to why we did it, and a couple of other things. But we were using Dagger version 0.1. In the meantime, Dagger version 0.2 was shippedā€¦ That was end of March/April, and that was CUE-basedā€¦ And people liked it, but it was CUE, and it was weird for many, and some struggled. So Dagger version 0.3 shipped.

Nope. [laughter] Thatā€™s a step back, thatā€™s not an improvement, Jerodā€¦ Thatā€™s not how this works.

Oh. Okay, good. I thought you were gonna say Yaml.

No, no, no. Itā€™s actually the language that you use. So if you write Go, you can now declare your pipeline in Go, and you can run it in Go. If you use Node.js, TypeScript - you can do that. If you really want CUE, thereā€™s CUE still there. Now, thereā€™s no support currently for Elixir; we could add itā€¦ There are just SDKs.

Is that a transpiler? Does it transpile to CUE then?

No. Thereā€™s GraphQL behind the scenes. So the interface to the engine, the engine that runs the pipeline, and it orchestrates everything, puts up an API, which is GraphQL, and it does all the magic. So anything that can make GraphQL calls can declare a pipeline and can run a pipeline.

So the next thing, the next big thing - and I donā€™t know whether this will be finished by the time this Kaizen goes out, which is in this case next weekā€¦ But soon after, I want to add a pipeline written in Go, because itā€™s a cloud-native language, that would be the upgrade from 0.1 to 0.3. In this case, Go SDK 0.4. A Dagger Go SDK 0.4 that anyone can run locally. And then our CI - we can get tests locally, we can have a bunch of things, and this opens againā€¦ Itā€™s the next bits of Docker, but without Docker, in a nutshell, for pipelines.

[01:00:15.06] Is that your guysā€™ new marketing slogan? ā€œThe best bits of Docker, without Docker.ā€

Noā€¦ Thatā€™s just me saying it.

Well, submit that to the CMO, see what he thinks.

For those that listen and donā€™t know yet, I do work at Dagger since Januaryā€¦ So itā€™s been 11 months now, close to 12 months. Itā€™s close to being a year, by the way.

We wouldnā€™t normally run a 0.1 tool unless we had inside access, right? 0.1 is pretty earlyā€¦

Very early, yeah. I think we are one of the very few ones that run it in production. Now, thatā€™s the thing that has been running every single commit since November. And apart from the Docker engine - by the way, Docker is needed as a shim to get BuildKit, but BuildKit is the interesting one; it was the interesting componentā€¦ And that is slowly changing, because the Dagger engine, as itā€™s being built, itā€™s less and less BuildKit and more and more its own thing. It was just like a stepping stone, same as Docker is a stepping stone. But BuildKit is the interesting bit right now, and that will change soon.

So the reason why we want that is because we donā€™t want to be writing Yaml, and we donā€™t want to be writing anything else that looks weird. Now, we may not be able to write Elixir right now, but itā€™s an option in the future. If someone wants to contribute that SDK, why not? Larsā€¦Thatā€™s the first person that comes to mind.

So is this a move back? I mean, not back, but is this a move in the direction of imperative configs? Because you have a full programming language at your disposal; youā€™re coding now. Youā€™re not like doing some DSL, or declaring your infrastructure, your pipeline. Youā€™re coding it.

Youā€™re coding something that makes GraphQL calls. So theyā€™re all very functional. So you run this function, and it gives you back a response. And then what do you do next?

So because youā€™re thinking in terms of functions, in terms of artifacts, it is a real pipeline in the sense that it does operations, it gets results, and then it continues with those operations. The nice thing is you get to use your own language for the looping, for anything that you would do - templatingā€¦ Any weird things that you may want to do, you get to use your language.

If thereā€™s any library ā€“ for example, to integrate with 1Password, you get to use your language. You donā€™t have to find a CLI, or this weird thing to integrate with the service. If thereā€™s a library for your language, thatā€™s all you need. So that is one of the advantages.

Well, I happen to know a pretty good functional language thatā€™s good at pipelinesā€¦

I know, right? [laughs]

Elixir seems pretty good for this.

Pretty much, pretty muchā€¦

But Goā€™s a good, cool start for us, for sure.

I mean, itā€™s something really lightweight, and something thatā€™s supportedā€¦ So version 0.3 - that makes me really excited, because we can get rid of some CUE, and we can get rid of some makefiles as well. We can get rid of all the makefiles. And we can use a magefile, which is makefile but in Goā€¦ Rake? Remember Rake? Mage is like Rake; think of it like that.

Okay. For Go.

For Go. Exactly.

Alright.

So thatā€™s the next big thing.

It sounds pretty coolā€¦

Now, what else is coming for you, between this Kaizen and the next oneā€¦? Any last thoughts before we wrap up?

I didnā€™t prepare for this being the last one for the yearā€¦ Iā€™m excited about the improvements that have been happeningā€¦ I donā€™t know, thatā€™s about it for me. Iā€™m excited that these discussions are open now too for these episodes. I feel like itā€™s more of an integration for the listeners to play a part, and see whatā€™s going on behind the scenes.

Do you remember when we used to do this once per year?

I do, yeah.

Thatā€™s how we started. Once per year. Look at us; every two-and-a-half monthsā€¦ This is pretty good.

[01:03:58.28] Is this the right clip? Should we be doing it less, or more? Or just right? Something worth thinking about. Maybe ask you, the listener - these Kaizen episodes, do you like them less or more than all the other Ship-Its? Because if itā€™s a lot more - I donā€™t know, maybe we have to make adjustments. If itā€™s the same, then we just keep it as is. If itā€™s less, maybe we should stop doing these. I guess maybe we should Kaizen our Kaizen, and hear from the listener, and ask you if this is something we should do at the exact same rate, or more, or less. What do you all think? How can we make it better? And then weā€™ll know for next time.

I really like that.

In terms of things that I wanna work on - thereā€™s a long listā€¦ Lots of little things. I guess probably expect a lot more little commits from me. We have a couple of big initiatives that probably wonā€™t land by the next Kaizen. So more small changes. I would like to see some outside contributors, now that we have the new contributing guide.

Iā€™ve never been good at providing low-hanging fruit for people to reach for and grabā€¦ Maybe I can put some effort towards that with my list of little things; hang them out there as good first issues, or ways of getting involved for people. Maybe that can be a goal for me over the next couple months.

I think so, too. I mean, we already had two very nice contributions, external contributions for this Kaizen. That was very nice to see. So if we can get two for the next one, thatā€™s amazing. I mean, one we didnā€™t merge yet, so the RSS feeds - Iā€™m sure we will discuss in the next one. But Iā€™m sure there are other things. Again, I donā€™t know what they are, especially like the small onesā€¦ And Iā€™m mostly focused on the infrastructure side of things, and that requires access, and things like that. But from the app perspective, Iā€™m sure thereā€™s other things. Dark mode, especially in these dark, long nightsā€¦ I would appreciate dark mode.

Well, we might be working on something in that arena as wellā€¦

Oh, interestingā€¦

Letā€™s invite some folks in. So if youā€™re listening to this and youā€™re thinking ā€œGosh, Iā€™ve gone to the repo, I donā€™t see these lack of low-hanging fruit that Jerod is talking about. How can I get involved?ā€ The easiest way honestly is just changelog.com/community; you can hang out in the Dev channel, or I would say even the Ship It channel. [unintelligible 01:06:20.27] asking questions around this front. So you can go into Ship It and ask questions about the show, or whatever; or you can go into Dev, which we have pull requests that are there, and things like that happening; commits that are in there. But you can also ask questions. And if youā€™re thinking like ā€œHey, Iā€™d love a T-shirt, and Iā€™d love to contribute. Give me a directionā€, and if itā€™s worthy enough, obviously, then weā€™ll give you that direction and give you a shirt, a code, or whateverā€¦

But thatā€™s probably the easiest way, is be part of the community. Come on in, andā€¦ Thatā€™s how we wrap up the year, too. We have a state of the ā€˜log coming up too, where we go through the state of the Changelog over the year, which is sort of the show itself, and also sort of overarching of the entire CPU, the Changelog Podcast universe, so to speak, thatā€™s growingā€¦ But it begins with people. Itā€™s all about the people.

So come on in, youā€™re welcomeā€¦ Hang your hat, call it homeā€¦ Weā€™ll make you a tea, coffee, favorite water, give you some slippersā€¦ Chill out.

And if you donā€™t want to sign up for Slack, thatā€™s okay, too. We have these discussions on GitHub. Youā€™re already there, we know that. Discussion 433ā€¦ Drop us a comment. Weā€™re all subscribed, and getting notifications for that; so that works, too. I think itā€™s time to call itā€¦

I think so.

Thatā€™s it.

Happy New Year, Merry Christmasā€¦

Merry Christmas.

Not in that orderā€¦ And Kaizen.

Yes, Kaizen!

Kaizen!

Kaizen!

Get your shirt!

Changelog

Our transcripts are open source on GitHub. Improvements are welcome. šŸ’š

Player art
  0:00 / 0:00