Go Capture the Flag! 🚩

Hello, and welcome to another wonderful episode of Go Time. I’m your host, Angelica Hill, and in this episode we’re going to delve into the exciting world of Go Capture the Flag. My wonderful guests are going to share their experience organizing this wonderful game, and overseeing it as they implement it for the first time at GopherCon 2023 last year. For those of you who don’t know, Capture the Flag events involve teams vying for supremacy as they strive to gather digital flags, presented as strings, and successfully submit them to the competition organizers.

It’s a dynamic competition where the team who amasses the highest amount of points is victorious. So it’s essentially a thrilling kind of scavenger hunt for nuts. So join us today as we unravel the intricacies and the excitement surrounding this wonderful gaming experience, and we are extremely lucky to have not one, but both of the organizers of Go Capture the Flag with us today. So our first wonderful guest is Neil S Primmer, who is a principal architect at Rearc, a boutique cloud and data consulting firm. He lives in the San Francisco Bay Area with his wife, three children, and a multitude of cats, which - I’m told the correct noun for that is a clowder of cats. Is that right, Neil?

That’s what my research has found. I can’t back it up myself, and I certainly don’t refer to them that way.

Well, welcome to the show. How are you today?

I’m doing fantastic.

Awesome. Excited to dive in. Our next wonderful guest is Benji Vesterby, who is the CEO of CodePro, which is a cybersecurity and software consulting-based firm in North Carolina, where he lives with his wife, his daughter, two dogs and two cats. And he says he’s “not cool enough to have a clowder of cats”, so he’s going to have the two dogs and two cats. Maybe you need to up the cat number to get that classification. His background is in computer science, which eventually led him to information security, and everything from deep packet inspection, to application security, to industrial control system security. How are you doing today, Benji?

I’m doing pretty good.

Awesome. Excited to dive in.

I’m not sure my wife will let me get a third cat.

Really?

Yeah.

Well, one can hope to aspire to the clowder level that Neil has. So we’re going to start with the very basics. What is Go Capture the Flag?

So Capture the Flags are a type of competition where teams compete to get the most points by retrieving digital flags, in the form of strings, and submitting them to the competition organizers.

Yeah, the easiest way I’ve found to explain the Capture the Flag to people is that it’s essentially a scavenger hunt for nerds. The goal is to code or hack a given challenge in order to expose a flag, which in the majority of cases for the one that we developed last year was a uuidv4, that was surrounded by curly brackets, and prefixed with gc23. There were several flags that didn’t follow that pattern, where we used some hidden flags on the badges, or on the hotel key cards, things like that… But for the majority of cases, it was a uuid before.

And how did this kind of come together? I think we’ll go from the start and bring through till kind of where we’re going after this. So how did this come about? How did the idea of creating a Go Capture the Flag and bringing it to GopherCon happen?

Yeah, so the organizers of GopherCon wanted to have more community events last year. And having a background in security, I proposed a Capture the Flag. This is something that you normally see at security events, and it’s not exclusive to the security community. And engineers enjoy puzzles. So after figuring out a rough amount of effort and time involved, we made the decision to move forward and start looking for a team of people. Neil joined me pretty early on, and we decided to co-host the event.

Yeah, as soon as Benji asked me to join, I was all-in. CTFs are something I have been interested in since sometime in the late ‘90s, to early 2000s. I remember watching coverage of some of the earlier Def Cons on ZD TV and Tech TV, and just being really drawn into that idea of gamifying your own education. There’s more to it than just the CTFs, but at least to Neil in high school, the idea of building, improving your skills or doing something hands-on was really appealing. As my actual career developed, I had actually never expected to get the opportunity to participate in one. My roles have been far enough away from security as a primary focus that it didn’t make sense for me to attend security conferences. And the conferences I did attend didn’t have an official CTF as part of the programming, at least until this year.

So you’ve kind of talked about it being not specific to the security community… But a question that kind of just popped to mind was, am I right in assuming that this is something that is more prevalent within the security community? I’m kind of interested as to why that might be… And is the nature of how you build and how you create these puzzles innately security-based? I’m kind of interested in that kind of interaction there between - if there is even one.

[00:06:23.05] Yeah, so security as a whole - if we take and look at the breadth of the security community and everything that it entails, the scope is fairly large. If we just take a broad look, you have infrastructure security, you have application security, you have network security, you have… And when you start breaking down into smaller pieces, then you start looking at “Okay, well, what are the subcategories of infrastructure and network and application? And how are those things intertwined?” I mean, you start going down the rabbit hole of “Okay, we’ve got infrastructure that’s hosting virtual machines, which have containers running on them, that are hosting Kubernetes clusters”, or - well, not containers running Kubernetes clusters, but VMs that are running Kubernetes clusters… Which were then hosting pods, that are running containers, that these mesh networks inside of them… And each of these different layers has a different layer of security. You’ve got networks and networks, you have all of these different levels of different security.

So what’s happened is you have this really, really large scope of necessary, essentially training and knowledge that has become necessary to know in the security community. And one of the ways the security community has gone about trying to solve this problem of a lack of knowledge or training has been to develop these Capture the Flag events… Where people can challenge themselves to learn new skills in a more engaging way than just sitting down in a classroom - which most people in the security community are not great at. And it allows them to engage more with the actual knowledge, and do it in a more real-world scenario where they get to actually attempt to attack a system, or pivot into a network, or use a multi-layered vector in order to actually infiltrate a specific application, or something like that.

It’s not specific to security, because the idea is just problem solving. It’s just using a different mentality to problem solve. And in fact, a security expert dealing with something like a capture the flag actually falls more into the mindset of somebody that you would normally meet in the QA. Somebody whose primary role is to actually break an application, if you start comparing that to security. And then if you start looking at like challenges, or problem-solving type events like this, the Association of Computing Machinery has had this for a long time, through universities and things like that, where they’ll actually host these events. And you have students competing against each other, in competitions and in Academia, solving computer science-based challenges for awards. So it’s not that it’s a new idea, but the security community has kind of really popularized it, and made it more of a generalized conference type event. And I think it really pairs well with the software community.

That’s awesome. So before we dive into specifically what you all made happen at GopherCon this year, I’d love to hear a little bit about your – and it doesn’t have to take long, but I’m interested to hear kind of for you both how you first kind of came into either participating in, or have you organized a CTF before GopherCon? Have you used it as a learning tool in your own growth and development? I’m kind of interested to hear your personal I guess stories and experience prior to GopherCon with these kinds of either competitive spaces or learning tools.

[00:10:04.04] Unfortunately, I hadn’t had the opportunity to participate any real Capture the Flag prior to this year. I did have some experience with similar series of challenges that you need to progress through… I’ve had employers that used that as kind of a test for advancement in certain times. But an actual real full competition challenge I haven’t had the exposure to directly before.

So exciting. That’s awesome that you got to do it for the first time at GopherCon this year. How about you, Benji?

Yeah, so I’ve been to a few different conferences where I’ve been able to take part in a Capture the Flag. My first real introduction was actually at a company off site when I was at Contrast Security, and it was actually led by one of the organizers of Kernelcon, which is a security conference based in Omaha. His name is Adam Shaw, and he actually is the one that helped kind of guide us last year; he was kind of our mentor, actually, in the process, and he’s actually who I had originally approached and said “Hey, what does it normally look like to build one of these?” Because he’s done quite a few of them. So he helped kind of guide the initial parts of the process and give us direction… And he had built this one at Contrast. And it was something that they did as part of a company off site, the very first year I started with them.

After that, I went and had done some other ones at other security conferences and things like that, and one of the things that I’ve found most frustrating about it is that, as a software engineer, it required a lot of very niche security knowledge that you don’t necessarily normally have. Now, me specifically, coming into it, I have a security background. Before contrast, I was in an information security team working as a security engineer, building out security tooling at Symantec. And so I had a fair amount of that, with the training and everything that I had from previous experience. And even then, it was difficult for me to participate in this Capture the Flag event at Contrast… Because it required very specific knowledge in certain areas of security.

So what I wanted for the one that we built was to have it be more approachable for people who had never done one, to be able to actually come in the door and try it out and not have to feel intimidated, and actually be able to make progress in the event, without it being something where it’s like, you just want to throw your computer out the window and walk out of the conference as soon as you even just look at the first challenge.

Great. Which kind of goes into - I would love to kind of talk a little bit now, off the back of what Benji was talking about, as to how specifically did differentiate GopherCon CTF from the more traditional CTFs, and maybe think through some of these pain points that Benji has flagged around needing that niche knowledge.

Yeah… Like we’ve said a couple of times now, traditionally Capture the Flags started in security conferences, and have been played at security conferences. There’s actually a couple of different styles of play even within Capture the Flags. What we chose was called a Jeopardy style, where you have to kind of receive your challenges from the organizers, and take each of those challenges on its own. But there’s other types, such as an attack defense style, where you have a red team and blue team, and one of them attacks infrastructure or flags owned by the other team.

That being said, those competitions typically focus primarily on those cybersecurity skills, and since GopherCon isn’t a security conference, we had to make sure that the challenges didn’t require any security knowledge at all to solve, but could be solved through the application of code.

One of the examples is we had the Raging Red Storm challenge, where contestants had to connect to a WebSocket endpoint and quickly solve math problems that grew in length as they progressed, until they finally got the flag. In order to make sure that they were solving that programmatically, the backend server was awarding points for each solve, but then also taking points away for the time it took in between solves. So if you were trying to do it all by hand, you could get to a certain point, but then the points just started decaying too fast for you to catch up.

[00:14:04.16] Finally, we wanted to make some challenges that didn’t require any specialized knowledge at all to complete. Mostly just there to be fun mini games. The terminal Text Twister Challenge is one that is kind of in that category. We built a web app that duplicates the terminal hacking game from the Fallout series. And for anyone who hasn’t played Fallout, that mini game is supposed to simulate evaluating a memory dump with a number of plain text strings in it, where clicking the right string gives you access to the system. You’re kind of picking the password from a memory dump.

For our challenge, we incorporated a series of these memory dumps the players had to progress through to get to the flag. There was also a hidden bonus flag available during this challenge for players who were paying enough attention to their selves.

So I don’t know, Benji, whether – I would love to hear a little bit more about some of the other challenges, or kind of any other ways in which this differentiated, or you intentionally made it different from those prior Capture the Flags that you kind of were chatting about previously, having done them before.

So it was important to have challenges that would push participants to learn new skills. So there were some fun ones… There was one that Neil had implemented that was a one-time pad puzzle, and I think we’re planning on using that one as an example going into this next year. We’ll publish the solution I think for that one as we’re getting ready for GopherCon 24, essentially as a write-up to say “Hey, if you’re looking at trying out the Capture the Flag this next year, this is something you should definitely read through for the mindset of how you should approach puzzles in this event.”

The challenges were set up in a way that – some of them were security related, and some of them were just challenging in general. We had some really interesting ones that actually leveraged some – there was one specifically that was actually not implemented by Neil or I either; there were several others that contributed to the Capture the Flag. One of them was actually exploiting a vulnerability that was in one of the Go open source libraries. Not the Go standard library, it was actually a third party package… But that was quite an enjoyable one.

There was one that I was really sad nobody got. It was a multi-phase challenge where you were able to get access to an SSH key and use that to shell into a server… But nobody actually got to that one. But it seems like based on the final statistics, I think we did a really good job of having a solid distribution of fairly simple challenges, to a fairly complex challenges.

I think one of the really interesting things that we saw when we first got started, that I think we’re going to probably work around this year as we start developing out the event this year, is that everybody started - when they pulled up the site where they got the challenges from, everybody started from the top and went down. So they all started on the same challenge… Which I thought was kind of funny, because they were broken into different categories, with different levels… And they were randomly put in there. So they weren’t in any particular order.

So everybody seemed to start from top down, even though there were other challenges in other categories, that would have been easier to solve… If it were me, or if I had any recommendation for anybody, it would be if you get stuck, or you start feeling like you’re frustrated with a puzzle, move on to another one. Just randomly pick, start going through them. Find the flags and move on to the next puzzle, and then go look back to the ones you had difficulty with. Go solve the easy ones first. Get as many flags as you can, and then and then loop back to the hard ones.

Mostly because we want it to be something where you feel like you’ve made progress, we want it to be something where you feel like you learned at the end of the day, and that you had fun, and that it’s not just something where you’re frustrated, or anything like that. And then also, look for a team of people who are interested in solving challenges with you.

I’m kind of interested a little bit in the process of actually like picking what challenges are you going to create. You talk a lot about making sure people are learning, but also making it accessible to multiple levels. I would love it if you could just take a second, either one of you or both of you, to chat a little bit about that process of deciding “We’re gonna have this brainstorm, perhaps… Like, we have all these challenges that we could do. Which ones actually make it?” Let’s use GopherCon last year. How was that decision process to make sure it was kind of all the things you’ve said - accessible, people are learning, you have things that are going to challenge all levels, while keeping it interesting?

[00:18:18.15] Yeah, so this one was kind of interesting… So the idea to do the Capture the Flag last year was kind of last minute. I think we had a total time between deciding we were going to do this and the event actually taking place was less than two months of time. I think Neil and I first started putting code down with less than 60 days to the event. And we were able to pull in a couple of others who helped contribute to some of the code, and then there were some people who helped run it on site.

And so kind of what ended up happening was we ended up having a theme, we had the Fallout theme… I pulled a number of challenge ideas out of previous Capture the Flags that I had been part of. One of the more difficult ones had come out of that one that I did at Contrast security, that everybody really hated… And the one that Neil developed, the one-time pad, or - what was that one called, Neil?

The Minutemen Missive.

The Minutemen Missive. That’s the one. Yeah, the Minutemen Missive was the one that everybody hated. I think all of the teams got stuck on that one for almost an entire day, for the teams that actually got to it. But yeah, I mean, it was one of those things where it’s like “Hey, this could be interesting or fun.” Initially, I took different categories of like security vulnerabilities. I had a whole list of them. I was like “Hey, we’ve got application security, we have supply chain security, we have web app, network security”, things like that. The radio behind me was filled with Raspberry Pi’s… We had an onsite network that went live I think on the first day of the conference… I didn’t have it live on community day, but it was live on the first day of the conference. In order to get into that network, you had to actually go through a packet capture, and find the password for it, and then you were able to attack some of the Raspberry Pi’s inside of that network.

So initially we had a set of categories that they were broken out into, and then Neil and I were just kind of hitting up each other on Slack with new ideas as we would go. I mean, because of the lack of time, the original plan was we were just going to have like 10 challengers total, and it was going to be the team that completed all 10 challenges first. Because what happened is Adam who was a helping mentor, said “Look, you have a limited amount of time, so just do a 10-challenge plan, and then the first team to complete all 10 challenges is the winning team.” Normally, you would do with a Jeopardy style Capture the Flag you would have 40 to 50 challenges, and then not every team solves every challenge, but most teams solve most challenges. Or most challenges are solved by at least one team. And what ended up happening was Neil and I started hacking away, and we ended up with almost 50 challenges. Or I think we might have actually exceeded 50. I don’t remember the exact numbers.

I think we were over 50 flags, but a lot of those were like bonus flags.

Oh, okay.

I think we were somewhere around 40 total challenges.

40 total challenges. Yeah. And actually, the statistics ended up lining up pretty well with that balance too, as far as where the distribution of number of solves per team, how not every team solved every challenge… And yeah, it ended up being one of those things where it ended up being a full Capture the Flag unexpectedly.

[00:21:41.26] The process was just – I mean, it was kind of more organic. It was like “Hey, this sounds like a fun problem. Can I do this thing?” One of the more interesting ones, I think, was – it wasn’t a security challenge per se, but you could technically consider it as a security challenge… We use the context in Go to share values in like a REST API all the time; we pass around authentication data through it all the time… And one of the challenges I put together was “Can I set it up so that with a known key, somebody can pull an authentication value out? If somebody incorrectly uses the plugin system, can they create a plugin that then dumps this known key out of context on a REST endpoint?” And of course, they were able to do that.

And then I was like “Well, how difficult would it be to do the same thing, but actually use reflection, and pull all the values out of the context, and find all of the authentication data in that context that we don’t know the key for?” So I was able to add an extra level of complexity to the problem, that they had to actually go and find the hidden keys that they didn’t know the actual flag value for.

So there were some of them that stretched people’s Go knowledge. That one involved the plugin system, which most people hadn’t used… I mean, most people probably shouldn’t. And then this year - I mean, not to ruin anything for anybody, but I might throw in a tool exec thing (we’ll see), for anybody who’s familiar with that, which is probably not a lot… But that might be fun.

Start studying up now.

Yeah, start studying up now, if you know what I’m talking about. If you don’t know what I’m talking about, have fun.

[laughs]

The work that I did at Contrast - I saw a lot of really interesting things in the Go language that you should never do. And so I got to see a lot of those intricacies of the language that I would not normally have seen… And so I could play with some stuff that I thought would be fun.

So I would love to hear a little bit about how it went at GopherCon this year. I mean, Neil, it was kind of one of your first, as you said, experiences organizing and then running a Capture the Flag. I would love to hear from you, how did it go?

So we actually really had kind of high hopes even going into it, just because we hadn’t seen this at non-security conferences, really. But the participation actually exceeded our expectations. We had almost 20% of the attendees register, and 72% participation rate from those who registered. We also had a pretty good response rate for the Feedback Forum, just over 15%. Most of our participants were there for their first GopherCon ever, and about three quarters of them also had never participated in a Capture the Flag before. So this seems to confirm that we were pretty close to the mark with what we were aiming for regarding the approachability of the challenges.

One thing we did get surprised by was the feelings around the theme. We’d expected reactions somewhere between like “Meh” to being really excited, but the feedback was a lot more polarized. There were a number of people who disliked the theme. They hadn’t played the Fallout games at all, and felt like they were at a disadvantage because they didn’t pick up on context clues, or were worried about things just being hidden in like game lore.

Thus far, the overall feedback we’ve received on the theme can be summarized as “We’d love to see the theme rotate each year”, so long as we’re able to do it to the same depth. I think that we did have a lot of depth to the follow-up theme, which was, I think, really able to help shape a lot of those challenges in a way that they would have felt more generic without.

One other thing that we were well aware of, but the feedback confirmed, was that the hints for the challenges were kind of of inconsistent quality, particularly in regards to the cryptography challenges. Bringing it back to that one-time pad, we’re hoping that for next year we’re going to be able to incorporate some level of like contextual hints that will allow players to know which hints will actually help them, instead of giving them information they’ve already figured out.

I’m kind of interested to talk a little bit more about that kind of theme/story-base/having some kind of throughline between the challenges, especially given, as you noted, Benji, that you kind of should be able to take on the challenges in kind of any order that you see fit.

[00:25:55.11] I’m kind of intrigued thinking about the experience in GopherCon, or just like generally as you think about doing this particularly again, or past experience, how important is it to have that kind of throughline, overall theme that can tie everything together?

Yeah, so when I talked to Adam originally, that was actually his first thing; he’s like “You have to have a theme.” It’s one of the most important things for having a successful event, is to have a theme that is engaging for the participants. Otherwise it’s just a bunch of challenges with no direction.

So to what Neil said and what’s important, I think that the participants misunderstood that just because there was a theme, they assumed that they had to know about Fallout to do the challenges. And that wasn’t the case. The challenges, while they were themed for Fallout, had no Fallout-specific knowledge in them that was required in order to actually solve a challenge. It was all very much like security, or Go, or programming in general. In fact, most of the challenges didn’t even require Go.

So one thing that is important to make clear though, since not everybody who’s hearing this actually was there for the event - when the challenges went live, not every challenge was available in the system. What it was is actually a list of challenges, and each one of those - there were some of them that were part of a series of challenges. Those ones, as you would go through each step, would unlock the next step.

So let’s say for example - we had one called the Nuka-World Challenge. Nuka-World is this area in the Fallout game series, it’s kind of like a messed up theme park that’s based on Disney, essentially. And that one was an application security set of challenges. Now, there were five or six challenges in that set, but they would unlock in sequence. So the first challenge was the easier of the five or six, but as you progressed through that set of challenges, it would unlock each one afterwards. So in that specific series you would have to do the first one, and it would unlock the next one. But the first one in each of the series all the way down the page were available. So at the beginning there were maybe 25 available challenges. By the end of it though, you had 40-50 challenges available to the whole team, assuming they unlocked the other challenges. So it was one of those things where as you would progress, you would expand the available challenges.

The theme though allows us to – one of my favorite things was hiding Easter eggs in different parts of the challenges. One of the challenges… So Fallout, if you haven’t played the game, one of the things about Fallout that’s really quite enjoyable for me, just because I have a weird sense of humor, is that it’s based in this post-apocalyptic world, that’s kind of like stuck in the 1950s, but it’s kind of like this demented sense of humor. And so there’s this – if you saw the trailer that we put out on the GopherCon channel last year, you can kind of see some of this in the actual 1950s footage that is in that trailer from here in the States, that they showed children about nuclear fallout… That kind of humor is actually in the game. And so I took some of the cartoon clips from the game, and actually streamed them inside of the network that was inside the radio. And one of the challenges actually made it to where if you successfully found that stream inside the radio and viewed it, the flag was actually across the bottom of the broadcast. And you could actually play it. It was an actual streamed video on an RTSP stream, and it would have sound and everything. One of the actual systems was just over an RTSP stream.

So that was kind of a fun Easter-eggy type – like, this is just fun for me. It was a simple network security challenge in the sense of it required more security knowledge than software knowledge. You would normally – at a security conference, you would network scan, you would see “Hey, this port is open. It’s normally used for this specific protocol”, and then you would go view it. So it was a little less of the “I’m a Go programmer”, that kind of thing, and more of a security challenge. But it was fun for me, and it was kind of like a little Easter-eggy type thing.

[00:30:05.25] This year’s theme - we’ll make an announcement as to that later on, but it’s gonna be fun. We’ll have some good Easter eggs and good linking between challenges on that one, I think. Yeah, so it’s nice, because it creates this cohesion between challenges, that - it’s not just fun for participants, but also it’s fun for people building the challenges. Like, it keeps us engaged as well.

I feel like one of my core questions was exactly that, is are we going to do this again? We obviously have GopherCon coming up this year, so I’d love to hear one - I think I already know the answer, but I need confirmation… Are you going to be doing it again this year? And if so, what are the things you’re going to take from this last year? What are the things that you’re thinking you’ll do differently?

Yeah, we’ll definitely be bringing the event back next year, and we’re already planning for next year’s event at GopherCon 2024. We do have some exciting additions and changes for next year, that it’s a little too early to spoil right now… But definitely expect to hear more about it throughout the time leading up to GopherCon 2004.

And if we want to stay posted, where am I looking to see these little spoilers?

Keep an eye on the GopherCon social - Twitter, YouTube, all of the different GopherCon social accounts. Slack - we have a GopherCon Slack channel… Or sorry, an actual Capture the Flag GopherCon channel. So you would [unintelligible 00:31:28.29] GopherCon-CTF.

And it’ll be in the episode notes, wherever you’re listening to this podcast.

Yeah. I do think we’re – we’re planning on changing the name of it this year. One of the things that Neil and I discussed was that the first two days of the conference we spent the majority of our time explaining what a CTF or capture the flag was. I think I explained it no less than 60 times on the first day of community day, and just more the first day of the conference. Neil, do you want to tell them what we’re thinking?

Yeah, so we’re actually thinking that to better emphasize what it is that we’re trying to do, as well as the right mindset for people to come into it with, is we’re going to rebrand from Capture the Flag, instead to Challenge Series. The format is going to largely remain the same, but we felt that there was definitely some missing clarity in the Capture the Flag moniker. Some folks actually said they thought it was going to be like physically taking physical capture flags and capturing them. So that’s definitely something we took into consideration when making that discussion.

For sure. I will say, when I first heard about it, it was like bringing me back to middle school. “Where is the flag that I have to try and [unintelligible 00:32:42.21] people to get to?”

Yeah, and I realized that, when I was trying to explain it too, that first day, even when I tried to reference a scavenger hunt as an example of what a Capture the Flag was, even that is still potentially something that’s very cultural in nature. I think that anyone who didn’t grow up here had difficulty knowing what a Capture the Flag was, or what a scavenger hunt was. And so with the challenge series, not only does it separate it from that very specific cultural reference, it also allows us to move more away from – not move away from necessarily, but it gives us more access to a broader problem space. So it’s not just about – it was never about just security, right? That’s not what we wanted from it; that’s what generally Capture the Flags are at a security conference. And not all of our challenges were security-related.

The nice thing about calling it a challenge series is that it allows us to kind of like bridge the gap between, I don’t know more of the ACM style challenges, where we have somebody delete a node from a B plus tree, and hash the data values, and that’s your flag. Versus go crack a password.

[00:34:02.24] And it allows for people who have a more computer science background or a more software engineering background to have a more comfortable time actually attempting a challenge. It also allows us to do things that are more specific to our industry and our field, and encourage people to try new things that are Go-specific or software-specific, and challenge them in new and interesting ways that are not just security-related. Because at the end of the day, it’s meant to help people try new things and learn new things. It’s not about “Hey, here’s this new security thing” or “Try this really interesting niche security attack.” It’s really meant to engage people in the community to try something you haven’t tried before. Learn something about Go you don’t know about. Or maybe you learn new algorithm that you’ve never seen, or something like that. And I think a lot of the people that participated last year, I think – I don’t remember if we had this question on the survey, Neil, but I think if you asked them, they would agree that they all learned something in the event.

Yeah, because I think the thing that I’ve found most interesting and I’d love to kind of dig in a little more with you both on is that learning aspect. I see how this could be like a fun, challenging kind of game experience, a chance to flex different technical muscles, or solve problems in different ways… I’m kind of interested to dig a little bit deeper into that kind of learning aspect of it. Is it predominantly coming through exposing them to new technologies, new approaches to problem solving, and that’s what they’re learning? Is there a world in which there might be, I don’t know, hints and clues for people who perhaps come into a challenge and they look at it and have no idea what it is? I’m kind of interested in how you strike that balance between challenge and helping or facilitating that learning in a way that feels challenging enough to be exciting, but not too challenging that it’s like “Oh, I can’t do this?”

Yeah, that was something we had a little bit of a discussion about as we were building out these challenges… And there’s definitely somewhere we dialed back the difficulty a little bit as we went through and did testing on them after we started developing them. It’s kind of hard to know how difficult the challenge is going to be for someone though, because I actually expected that the one-time pad challenge was going to be significantly easier for people to solve than it was, because it’s just – one-time pads have been a thing I’ve known about for decades, because you read spy novels as a kid or something, it gets brought up all the time, and you’re aware of how it works even if you’ve never actually implemented it… But just missing that one piece of knowing how it works makes that challenge infinitely more difficult< and it just didn’t cross my mind. So then when we started actually digging into it, seeing how high of a difficulty that was for people - we’re going to have a lot of feedback from the people who participated in the serious challenges to help us be a lot better at understanding that balance going into next year.

Yeah. And I think there’s a lot that we didn’t know that we didn’t know. To Neil’s point, there was a challenge that had to do with a robots text file last year that I thought was going to be the easiest challenge of the whole thing. Like, it was literally like - I expected it to be like “This is the simplest challenge of the entire event.” And I sat there the first day, and every team – and because it was at the very top of the list, it was the first one that everybody started with, and every team struggled with it for the entire day. And I quickly realized that just because of my background, and I know what a robots text file is, and I know to go look there as a security practitioner… And in the hints they very much directed people at that; people in the community, or who have a different background, who haven’t necessarily worked in frontend development or haven’t been in the industry as long, who didn’t have a GeoCities site when they were a teen… Maybe they didn’t know to look at a robots.txt file. I’m dating myself a little bit there… That was a little less intuitive for them.

[00:38:15.02] And so it gives you a different perspective from a – because I mean, essentially, at the end of the day we’re kind of like puzzle makers. That’s what we’re doing. We’re building puzzles for others. We have complete knowledge of the puzzle though, and we don’t know what they don’t know, and we don’t know what we don’t know about who’s going to be playing. And so Neil had the idea of sending the survey out at the end last year, and we got a lot of information out of that. And we also have some of the participants from last year who want to help with this year’s challenges. And I think that that’s going to be good, to help direct us on a better path to make sure that more people get enjoyment out of it. And I hope that in this year the people who came by and might have been intimidated by it, or didn’t know what it was and walked on by, give it a try. Even if it’s something where – because the cool thing is we’ll do exactly what we did this last year. We’ll have challenges for anybody who solves at least one of the puzzles. So if you’re at GopherCon, and you come by, and even if you just sit down and you solve even just one of the puzzles, you’ll be in the random drawing for one of the prizes at the closing party… But I really hope more people try it out this year, because it was really a lot of fun… And I think this year’s challenges are going to be - not broader in scope, but they’re going to go beyond security, and they’ll be a little bit more in everybody’s wheelhouse, a little bit more approachable, I think, for most people.

Yeah, I would highly encourage anyone who is coming to GopherCon this year to check it out… As someone who is a fellow organizer of the conference, and was running around like a headless chicken, one of the things that I’ve said this year is “I’m not going to do that, because I have to spend at least one day doing the Capture the Flag.”

Kind of along those lines, and just thinking through people who maybe will be coming for the first time - as you said, you had a very high percentage of first-time GopherCon attendees… And as someone personally who is like a serial over-preparer, what are some things that if I want to be super-prepped… I know it’s supposed to challenge me, but I would like to get that pre-knowledge in. Is there any websites, technologies, things that I can do to really set myself up to really enjoy it, and bloodily have the best chance of winning?

So one thing that we would really recommend is reading a book called “The Red Team Field Manual.” That was actually one of the prizes we gave away, I think three or four copies of, last year, to various participants. The other is just CTF101.org. I actually use that as a resource when building some of the challenges this year. So that’s a definitely good place to start with just understanding a little bit more about how Capture the Flags work, and what some of the mindset that goes into those challenges are.

Yeah. And then we haven’t done it yet, we’ll try and get the blog out for the one-time pad sometime between now and when 2024 GopherCon launches. I’m not sure where that’s going to be at the moment, to be honest. And then there’s also a number of YouTube tutorials, things like that, that you can look up. I think HackMyBox is another example of a place you could probably find some training, things like that. Look it up on Google, or whatever. Don’t get in trouble, though…

But the Red Team Field Manual is a good place to start though. It talks a lot about a lot of tools. Play around with some Kali Linux, drop a note in the CTF channel, ask last year’s participants; there’s a lot of them in there. I’m sure that they’d be willing to share ideas or thoughts. This year is gonna be a lot more fun. I mean, not that last year wasn’t a lot of fun. This year is gonna be a lot more fun, because we’re going to bring a lot more challenges.

You have more time this year as well; you’re not having to rush to get it –

Yeah, we’ll see. We’ll see.

Well… I say that now.

Yeah.

[00:41:51.22] My kind of final question before I’ll kind of turn over to you all in case there’s any final closing thoughts is we’ve kind of talked a little bit about the types of challenges, we’ve talked about what people can do to prep… But for those who might be either attending GopherCon for the first time, with no software engineering experience whatsoever, are there any actual baseline requirements? I know you all do all you can to make sure it’s as inclusive as possible. But for those true newbies to the space… So my first Go conference, I hadn’t written a line of code in my life. It was something where I went to the conference to just see what this community was about, and then I was like “Oh my gosh, I love this. Let’s go! Now I’m gonna learn.” Two questions, you can choose which flavor to answer. One is how can people who have never written a line of code in their life participate or get some kind of enjoyment, or maybe come over in shadow and just watch it? Or for those who maybe are just getting into software engineering, and Go is their first language potentially, are there any - like, “You need to know how to use the terminal”? Are there any baseline requirements? I just want to make sure the true newbies have your guidance.

We definitely made an effort this year to make sure that at least one of the challenges was accessible to someone with no code knowledge whatsoever. The Terminal Text Twister, like I mentioned before - fun little mini game; you can just go through and you can brute-force it by just clicking words until you figure out what the right word for each stage was, and then move on to the next one.

There was also a crossword, which was – you could get the answers pretty easily without even needing to know anything about programming or about Go. You could just use Google and figure out the answers yourself. I think someone actually threw the answers into ChatGPT and got their answers that way… So we’re going to make sure that we have at least a couple of those challenges for next year as well.

The idea is that everyone who has any interest should be able to participate and should be able to pass at least one challenge. I think those are the people we want to reward the most, are the people who are taking a bigger step in coming in there and being able to say “I did it. Now what’s next?” and being able to be part of the group. And community surrounding just the competition is going to help them kind of take that next step beyond that.

Yeah. And the thing is too, is - some of the challenges too, we had several people that came by last year that didn’t have their computers with them. There were some challenges that you technically could do without a computer, but they were much more difficult. If you’re coming to GopherCon, bring a laptop with you. Just in general, it’ll make things a lot easier on you. But if you can’t, that’s fine. There’ll probably be some challenges you can do without it.

If you’re new to coding, if you’re intro to Go, things like that, and you want to do the challenge series - which we’re gonna call the Challenges Series - come hang out with the hosts at the table. One of the best things about GopherCon - and I’ll tell anybody who ever asked me; I’ve been to a lot of conferences over the years, and GopherCon’s my favorite. And the reason being is because when I went to GopherCon the first time, I think it was the first time that I can ever say that I felt like truly accepted as a person. So I sat down and I was able to sit with other people who spoke the same language, and enjoyed the same things. And it was able to learn from others in an environment where it didn’t matter who I was.

So I learned more in my first GopherCon about Go than I had in like the year and a half that I had been coding Go before I came. So if you’re wanting to learn Go, or you’re wanting to learn to program, or you’re wanting to get into software, come to GopherCon, come to the Challenge Series table… I will personally help you with the challenges. I mean, I’ll show you how to set up the terminal, if you don’t know… Any of that stuff. The Go community is incredible. It’s welcoming, and inclusive, and it is very much worth being part of. There has never been a development community that I have been more proud to be a part of.

[00:45:51.05] And I’ll jump on top of that as well… I think the community I’ve seen around Go is much more like a community than any other programming community I’ve been part of… At least at that scale. Like, I’ve been part of smaller, location-based communities that feel more community-ish, but for the tens of thousands of people surrounding the Go language and ecosystem, to feel so much like you know people, and you have friends, and you get to really know the other people involved… I keep saying that one of the biggest things and most important parts of GopherCon is community day. And for me at least, being part of the Capture the Flag felt like the entire conference was community day.

Yeah, that’s for sure. Yeah. It definitely felt like the whole conference was community day with the Capture the Flag.

That’s amazing. That’s awesome. Great. So what I’m hearing is whether you’re a complete newbie, or you’re a Go genius, there will be a challenge for you, and Challenge Series is for you.

Yes.

Great. I’m very excited. And I will see you at the table in a few months. And without further ado, we’re now going to move over to a completely unrelated - although you can make it related if you’d like - section of the episode, which is unpopular opinions.

So I’m gonna come to you first, Neil. I would love to hear your unpopular opinion.

Okay. Benji and I coordinated on this a little bit… I took the non-tech unpopular opinion.

Okay…

I believe the most appropriate cheese for pizza is Romano cheese. That is my unpopular opinion.

Say more. Why?

So the area I grew up in, in Eastern Ohio, actually has this as like kind of a regional dish. You take a pizza, you put like kind of a sweeter sauce, with green peppers, bell peppers baked into it, and then you don’t do any of the regular pizza cheeses. You do just Romano cheese. Shake your cheese over it. It doesn’t melt quite the same way, it’s not nearly as heavy, and it really meshes well with kind of a little bit sweeter sauce… And it’s just like the right type of pizza.

I cannot say that I have ever tried that in my entire life… But I want it in my mouth right now.

[laughs] I feel the same. I’m very intrigued. I need to go buy some now and make my own pizza at home for dinner.

Yeah, it’s a type of pizza called Brier Hill pizza.

Okay.

The only problem is that it would mean that I have to go to Ohio.

Oh, no…!

Just kidding. Just kidding. To anybody who’s listening in Ohio, you guys are great.

I’ve actually never been to Ohio. I need to check it out.

I used to work for a company that’s based in Ohio, in Perrysburg, actually…

Awesome.

I actually don’t mind Ohio. It’s very nice.

Okay, so we have an unpopular - or potentially popular, once people get a chance to try it - opinion. Benji, over to you.

Are you sure you want me to go before you go? You don’t want to go before everybody rage-quits?

No, I’m ready. I’m ready. We are not quitters on this show.

Alright. So my unpopular opinion - and you may or may not agree. That’s fine. I don’t care…

Okay.

…is that anyone, at any time, should be able to develop and release a net new project to the open source community, regardless of the number of pre-existing projects of the same kind, without being concerned for contributing to those existing projects. Open source shouldn’t be gatekept to new projects simply because a similar one exists, and people should be able to contribute code however they want, and be proud of their work.

There’s an XKCD for this.

[00:49:53.14] Is there an XKCD for this? Yeah? Because there has been so many times that I’ll build something and I’ll release – because I do most of my stuff in open source. Well, lately I haven’t been, but most of the time I’ll build something and I’ll open-source it just because “Hey, I use this”, and it’s easier for me to import a Go package if it’s open source. So I’ll throw it out on open source, right? And because I’m proud of it, I’ll share it in the Go Slack, or I’ll share it on Reddit or something like that, and everybody goes up in arms, “Oh, this already exists. Why aren’t you contributing to this other package?” And then I’ll see it happen to others, where somebody else will share their library, and things like that… “Oh, why aren’t you contributing to this other thing instead? Why did you build your own?” I don’t think that it should happen. I think that people should be able to contribute to the open source community and build what they want, whether or not it exists already or not.

I mean, maybe I’m preaching to the choir here… I would love to hear a little bit about why, like, devil’s advocate, it might be suboptimal to have loads of different open source projects that do the same or similar things… Because I can see that being a [unintelligible 00:50:55.00] argument, if there’s already something, and if the ethos of one of the core goals of open source is to make something better and better. Like I put something out there, and Neil contributes, makes it better; Benji, you come in, you make it so we build together a better… Surely, that may be a better path to overall community service than me putting something out there, Neil putting something out there, Benji putting something out there.

Yeah, I mean, I think you can make the argument that if you feel inclined to contribute to another project… Let’s say, for example, you are like “Hey, I really enjoy this open source project, and I work in the same language”, or whatever. Let’s say for example I enjoy working with the Gin router, right? Let’s say I prefer that one. I don’t want to go out and build my own router, and I find a bug and I want to contribute to that project - yeah, sure, I’ll go and fix a bug in the project, and I’ll PR into it. That’s fine. If you want to contribute to the project, that’s fine.

But let’s say, for example, I want to go down the rabbit hole of, I don’t know, creating my own logger. I’ve done that. I did it a while ago, years ago. Then I found Zap, and I threw mine away. But now Zap is thrown away because of Slog. So it’s one of those things where as time progresses, things change.

We see this in the Go community… A good example of this is Go Modules; it was a good example of this. We saw a number of different package repository options in the Go community, and then the Go team was like “Hey, let’s look at Dep”, and then - I forget the other ones. I only ever used Dep. And then they’re like “Let’s consolidate them, but let’s look at them and see what the benefits of each of these different ones are, and then build something that we can kind of all coalesce on.”

The same thing with the new Slog library. I don’t you need Zap anymore, I’ve got Slug. It’s part of the standard library in Go; I can use that. It’s no longer in beta, it’s in the standard library, it comes with Go 1.21. Go generics has come out, I don’t have to worry about using some weird third party generics library that’s using the empty interfacing [unintelligible 00:52:58.09] I can use the standard Go implementation of generics.

So in some respects, I can understand where contributing to an existing project might be more beneficial. In some respects, it helps push the community forward. In other respects though, it helps me develop as an engineer. If I need to go out and learn how to develop a logging library, I learn things that I didn’t know before. With the logging library, I learned how to better understand different levels of logging. What’s important to collect in a log that’s useful at different levels. How do I handle formatting, how do I handle lots of these different things. How do I identify where a log is taken inside of a specific file, how do I handle stack traces, things like that. And that’s only one example.

[00:53:46.19] I think that one of the most important things is that as an engineer, or as a person in general, is that you learn from what you do. And so whether or not it’s you putting out something into the open source just because that’s the easiest way for you to import a package, or you want to share it with somebody and be proud of it, or share it with the community and be proud of it, you should be able to do that without having others say “Well, why aren’t you contributing to this other project that somebody else created?” That seems rather immediately dismissive of somebody else.

At the same time, I think that the community can coalesce on certain technologies, and we have done that in the Go community. We have continued to evolve as a community, and we’ve gotten better in different places. And so I think that - yeah, I mean, I can see the argument either way, but at the end of the day, I think it’s important to encourage people to learn, however they do that. And I don’t think that people learn best by being shut down for being excited about something they did. I think that people learn best by being encouraged to try something that they want to do. And I think that that’s the difference.

So if I’ve [unintelligible 00:54:57.14] unpopular opinion, which actually I think is maybe quite popular, it’s less about whether you should either contribute to an existing, or put your own workout to open source, it’s more about the response, and the attitude towards being able to make that choice for yourself, based on your goals, what you want to do, what’s important to you, whether it’s learning, adding to a larger project, working with other maintainers etc. It’s less about which you do, and more about the attitude towards it, and having the community be more supportive of that individual’s choice, regardless of what that motivation is. If they want to do it, then that’s what they want to do. I’m intrigued. I feel like this might instigate a really great discussion. It’s getting my cogs going.

Awesome. Well, we are pretty much at time… So are there any closing thoughts, final remarks of any nature, before we close out?

I’ve gotta fill my gas tank, so I can go to Ohio.

[laughs] And on that note, thank you both. It was an absolute pleasure chatting to you both. As we’ve noted, if you’re not on Gophers Slack, join and then you can look within Gophers Slack and join the channel. Can you remind me again what the channel name is? Are you going to make a change to it, given the challenge name change, or is it going to remain as is?

We probably are going to change it this year. And actually, we probably are going to be more active in the Discord this year. Because I know that GopherCon covers the subscription for Discord, and we can have a lot more – there’s a lot more flexibility in there. And so we’ll probably be more active in Discord this year for the challenge series. So if you’re not in Slack or you’re not in the Discord, jump into both of those… But yeah, we’ll probably work on something to synchronize the two of them. Some kind of bot… I don’t know. We’ll see.

Great. And then on general updates on GopherCon and the Challenge Series, you can follow GopherCon on Twitter, all the places, to get updates. And cheeky reminder, call for papers for GopherCon are about to open. So if you’re noodling on an idea, please start formulating it into a concise proposal, so that you can get it in ASAP.

Yeah. And if you’re second-guessing submitting, don’t second guess it. Submit it.

Just do it. Just do it. Awesome. Well, pleasure as always talking to you both, and have a great rest of your day.